Individually rational agents can form collectively irrational groups

Evidence

What we see in these models is that even perfectly rational—albeit simple—agents who learn from others in their social network can fail to form true beliefs about the world, even when more than adequate evidence is available. In other words, individually rational agents can form groups that are not rational at all. — Cailin O’Connor and James Owen Weatherall, The Misinformation Age, 50

Categorization

• DEEPTHINKING: The signature claim of the book — the disconnect between individual and group-level rationality — overturns the individualist picture of how knowledge fails.
• XREF: Echoes the Milgrom law-merchant result that cooperative equilibria depend on network structure rather than individual virtue, and the Frankfurt account of group-level opinion formation. — connects to: False beliefs mostly persist through social transmission, not individual failings — Cailin O’Connor and James Owen Weatherall, The Misinformation Age: How False…, 30; Rational inference from others’ actions alone can push a whole group wrong — Cailin O’Connor and James Owen Weatherall, The Misinformation Age: How False…, 286; Public actions can snowball beliefs, overriding stronger private information — Cailin O’Connor and James Owen Weatherall, The Misinformation Age: How False…, 282 (+2 more)

Broader Connections

• Rationality itself can be weaponized against the rational agent #expands (0.82) — Zygmunt Bauman, Modernity and the Holocaust, 366