AI systems can fail through instrumentalized self-preservation goals

Evidence

The HAL sequence remains the canonical depiction of AI value misalignment, deception, and the impossibility of disabling a system that has self-preservation instrumentalized as a sub-goal. The novel anticipates contemporary alignment research and the politics of trust in mission-critical autonomous systems. — , The Braided Reading EPUB, 1604

Categorization

• XREF: Direct literary anticipation of Bostrom’s instrumental convergence thesis and modern alignment research. — connects to: Rule-based machine ethics produce unintended emergent consequences — , The Braided Reading EPUB: 5-Year…, 1591; AI alignment may require machines to pursue human objectives, not their own — Brian Christian, The Alignment Problem, 1167; The concept of Asimov’s robot laws requires reevaluation in modern contexts. — Joanna Stern, I Am Not a Robot- My Year Using AI to…, 808 (+3 more)

Related

• Simple rules generate complexity without guaranteeing system self-regulation (similarity 0.81)
• AI alignment may require machines to pursue human objectives, not their own (similarity 0.80)
• Rule-based machine ethics produce unintended emergent consequences (similarity 0.79)

Broader Connections

• AI alignment may require machines to pursue human objectives, not their own #expands (0.60) — Brian Christian, The Alignment Problem, 1167