Iterated distillation and amplification can build judgment beyond our own

Evidence

Perhaps, Christiano believes, this very same schema—what he calls “iterated distillation and amplification”—can be used to develop systems with complicated judgment, beyond, and yet aligned with, our own. … We might ask one copy of our system to give us an assessment of expected wait times. We might ask another to give us an estimated budget. A third we might ask for a report about accessibility. We, as the “boss,” would make the final determination—“amplifying” the work of our machine subordinates. Those subordinates, in turn, would “distill” whatever lessons they could from our final decision and become slightly better urban planners as a result: faster-working in sum than we ourselves, but modeled in our own image. We then iterate, by delegating the next project to this new, slightly improved version of our team, and the virtuous circle continues. — Brian Christian, The Alignment Problem, 1069

Categorization

• SEED: The ‘amplification’ loop that lets AI outstrip our own competence while staying aligned is the hopeful arc of the whole alignment problem.
• corroborated: 2 independent passages support this claim — Brian Christian, The Alignment Problem, 1069
• XREF: connects to: Learning reduces uncertainty that is what motivated deference to humans — Brian Christian, The Alignment Problem, 1294; Even randomly weighted linear models beat human expert judgments — Brian Christian, The Alignment Problem, 441; Permanent creations like geometrical theorems exceed ordinary human powers — G. H. Hardy, A Mathematician’s Apology, 153 (+2 more)