Dopamine encodes errors in expectation, not reward or surprise itself

Evidence

Once the monkey learned that the cue reliably preceded the juice, however, Schultz tried something new: he gave it a false alarm. He triggered the cue; as usual, the monkey’s dopamine activity spiked above the neurons’ normal baseline hum. The monkey pressed the lever. No juice came. The monkey’s dopamine neurons fell—briefly, but unmistakably—silent. “Then I said, Well. That is something different than surprise.” — Brian Christian, The Alignment Problem, 585

Categorization

• XREF: Resonates with the earlier discussion of how expectations shape perception; also connects to behavioral economic findings on loss aversion. — connects to: Gambling addiction may stem from intrinsic surprise rewards overriding negative expected value. — Brian Christian, The Alignment Problem, 909; Rewarded actions get stamped in and repeated while punished ones fade — Brian Christian, The Alignment Problem, 538; Copying same-sex models activates dopamine reward circuitry like receiving money — Joseph Henrich, The Secret of Our Success: How Culture…, 170
• corroborated: 2 independent passages support this claim — Brian Christian, The Alignment Problem, 585

Broader Connections

• Gambling addiction may stem from intrinsic surprise rewards overriding negative expected value. #complicates (0.60) — Brian Christian, The Alignment Problem, 909
• Rewarded actions get stamped in and repeated while punished ones fade #qualifies (0.50) — Brian Christian, The Alignment Problem, 538