Complex behavior emerges from rewarding successive small approximations

Evidence

The point was they had discovered shaping: a technique for instilling complex behaviors through simple rewards, namely by rewarding a series of successive approximations of that behavior. “This makes it possible,” Skinner wrote, “to shape an animal’s behavior almost as a sculptor shapes a lump of clay.” — Brian Christian, The Alignment Problem, 660

Categorization

• DEFINE: Defines shaping via Skinner’s sculptor-of-clay metaphor, which grounds the whole chapter’s technical discussion.
• corroborated: 2 independent passages support this claim — Brian Christian, The Alignment Problem, 660
• XREF: connects to: Machine learning divides into unsupervised, supervised, and reinforcement learning — Brian Christian, The Alignment Problem, 58; End-goal-only rewards make learning impractically slow — Brian Christian, The Alignment Problem, 672; Rewarded actions get stamped in and repeated while punished ones fade — Brian Christian, The Alignment Problem, 538 (+9 more)

Related

• End-goal-only rewards make learning impractically slow (similarity 0.78)

Broader Connections

• Rewarded actions get stamped in and repeated while punished ones fade #corroborates (0.90) — Brian Christian, The Alignment Problem, 538
• Trust forms sequentially by reducing social complexity over discrete steps #expands (0.78)
• Seen-through trust reduces complexity less efficiently than spontaneous trust #qualifies (0.72) — Niklas Luhmann, Trust and Power, 256
• Self-imitation can bootstrap a machine to superhuman skill #expands (0.70) — Brian Christian, The Alignment Problem, 1057