AI alignment may require machines to pursue human objectives, not their own
Evidence
For the last century, he says, we have tried to build machines capable of achieving their objectives. This has been implicit in almost all work in AI, with safety questions pivoting on how to control what their objectives should be—how to define wise and loophole-proof objectives. Maybe this whole idea needs to be overturned, he thinks. “What if, instead of allowing machines to pursue their objectives, we insist that they pursue our objectives?” he says. “This is probably what we should have done all along.” — Brian Christian, The Alignment Problem, 1167
Categorization
- XREF: Connects to earlier chapter discussions on reward hacking and specification gaming, where giving machines their own objectives backfired. — connects to: AI researchers aim to align general-purpose systems with hard-to-specify human intentions; Ensuring AI captures human intent is now a central computer science problem — Brian Christian, The Alignment Problem, 65; Overparameterized value models keep machines obedient at modest cost — Brian Christian, The Alignment Problem, 1302 (+6 more)
- corroborated: 2 independent passages support this claim — Brian Christian, The Alignment Problem, 1167
Broader Connections
- AI researchers aim to align general-purpose systems with hard-to-specify human intentions
#expands(0.85) - Ensuring AI captures human intent is now a central computer science problem
#expands(0.85) — Brian Christian, The Alignment Problem, 65 - Rule-based machine ethics produce unintended emergent consequences
#complicates(0.78) — , The Braided Reading EPUB, 1591