End-goal-only rewards make learning impractically slow

Evidence

Reinforcement-learning researchers know this as the problem of sparse rewards, or more succinctly as the problem of sparsity. If the reward is defined explicitly in terms of the end goal, or something fairly close to it, then one must essentially wait until random button-pressing, or random flailing around, produces the desired effect. The mathematics show that most reinforcement-learning algorithms will, eventually, get there—but eventually, in practical terms, might arrive long after the sun has exploded. — Brian Christian, The Alignment Problem, 672

Categorization

• XREF: Connects to the book’s broader theme of alignment: reward design determines what machines (and people) learn. — connects to: Complex behavior emerges from rewarding successive small approximations — Brian Christian, The Alignment Problem, 660; Educational technology personalization revives behaviorist learning theories long rejected in psychology — , The Braided Reading EPUB: 5-Year…, 1062; Optimizing for stated goals produces monstrous outcomes — , The Braided Reading EPUB: 5-Year…, 655 (+4 more)
• corroborated: 2 independent passages support this claim — Brian Christian, The Alignment Problem, 672

Broader Connections

• Complex behavior emerges from rewarding successive small approximations #complicates (0.85) — Brian Christian, The Alignment Problem, 660