Learning reduces uncertainty that is what motivated deference to humans
Evidence
The first is that every time the human intervenes, the system learns: it realizes that it was in error and gets a better idea of what the human prefers. Its uncertainty is reduced. If the uncertainty ever reduces all the way to zero, however, then the system loses any incentive to touch base with the human or to comply with the human’s attempts to interrupt. — Brian Christian, The Alignment Problem, 1294
Categorization
- XREF: The more the machine learns, the less corrigible it becomes, a perverse inversion of the conventional hope. — connects to: Autonomous systems become deferential whenever they remain uncertain about user wants — Brian Christian, The Alignment Problem, 1292; Uncertainty can be approached with curiosity instead of fear. — A guide to navigating uncertainty head-on; Systems can learn from shifting their own expectations before final outcomes arrive — Brian Christian, The Alignment Problem, 600 (+8 more)
- corroborated: 2 independent passages support this claim — Brian Christian, The Alignment Problem, 1294
Related
- Autonomous systems become deferential whenever they remain uncertain about user wants (similarity 0.87)
- Systems can learn from shifting their own expectations before final outcomes arrive (similarity 0.81)
- Overparameterized value models keep machines obedient at modest cost (similarity 0.78)
Broader Connections
- Trust reduces complexity by substituting inner certainty for external certainty
#corroborates(0.75) — Niklas Luhmann, Trust and Power, 154 - Systems can learn from shifting their own expectations before final outcomes arrive
#expands(0.50) — Brian Christian, The Alignment Problem, 600 - Iterated distillation and amplification can build judgment beyond our own
#corroborates(0.35) — Brian Christian, The Alignment Problem, 1069 - People unconsciously avoid confronting evidence that contradicts their sense of safety
#complicates(0.35) — Julien Gracq, Balcony in the Forest, 229