Anti-fake-news measures will be gamed once they become targets

Evidence

Perhaps more important, we need to recognize that fake news stories—and propaganda more generally—are not fixed targets. These problems cannot be solved once and for all. Economist Charles Goodhart is known for “Goodhart’s law,” which has been glossed by anthropologist Marilyn Strathern as, “When a measure becomes a target, it ceases to be a good measure.” In other words, whenever there are interests that would like to game an instrument of measurement, they will surely figure out how to do it—and once they do, the measurement is useless. A classic example occurred in Hanoi, Vietnam, under French colonial rule. In the early 1900s, the city was so overrun with rats that the colonial government offered a bounty for every rat tail returned to them. The problem was that people began cutting off rats’ tails and then simply releasing the rats to breed and make more rats, with more tails. — Cailin O’Connor and James Owen Weatherall, The Misinformation Age, 581

Categorization

• CANDY: The Hanoi rat-tail bounty reverse-incentive story is perfect elevator material.
• DEFINE: Goodhart’s law named and glossed.
• XREF: connects to: Social platforms are culpable for fake news but culpability is not remediation — Cailin O’Connor and James Owen Weatherall, The Misinformation Age: How False…, 577; Deception in warfare escalates against counter-deception at heavy cost — Emlen, Douglas J, Animal Weapons_ The Evolution of Battle, 454; Correcting conspiracy misinformation can backfire and entrench the false belief. — Russell Muirhead and Nancy L. Rosenblum, A Lot of People Are Saying: The New…, 119 (+3 more)

Broader Connections

• Deception in warfare escalates against counter-deception at heavy cost #exemplifies (0.85) — Emlen, Douglas J, Animal Weapons_ The Evolution of Battle, 454
• Correcting conspiracy misinformation can backfire and entrench the false belief. #exemplifies (0.80) — Russell Muirhead and Nancy L. Rosenblum, A Lot of People Are Saying, 119
• Credible threats can require revealing information that defeats the threat #exemplifies (0.80) — Thomas C. Schelling, The Strategy of Conflict, 801
• Pre-exposing people to weakened persuasive attacks cultivates mental resistance against later misinformation. #qualifies (0.78) — Sander van der Linden, Foolproof, 633