More sampling of a rare risk gives false confidence right before catastrophe

Evidence

One way to think about the problem is to imagine playing a game of Russian roulette. Most people would prefer not to play this game when offered the option because the risk-reward ratio is too high, which is correct if we assume the classic “one bullet and six chambers in the gun” setup. But what if I have information you don’t have, derived from a mathematical model called Brains at Risk (BaR), that tells me the gun has over a billion chambers and only one bullet, and that I can figure out where the bullet is by sampling (pulling the trigger millions of times)? Let’s also say that each time I pull the trigger I receive 100. I click once and am 100 richer, so I click again. By lunchtime, I am a millionaire and have grown confident. In terms of estimating the risk that I face, each “click without a bang” is a piece of information about the probability distribution. As I sample (click) more, I grow more confident about the shape of the distribution. I think that I am generating a more accurate prediction of where the bullet is with each piece of information (click), right up until the moment when I blow my brains out. I have just run into a black swan: a low probability (given the sample and assumed distribution) and (very) high-impact event. — Mark Blyth, Austerity, 113

Categorization

• XREF: The Russian roulette model is Blyth’s pedagogical version of Taleb’s black swan idea, adapted for the financialization critique. — connects to: A probabilistic threat stays credible despite failures because one trial proves little — Thomas C. Schelling, The Strategy of Conflict, 422; Assessing unprecedented threats through old experience leads to catastrophe. — Shoshana Zuboff, The Age of Surveillance Capitalism, 605; Under a Gaussian distribution extreme tail events become effectively impossible — Mark Blyth, Austerity: The History of a Dangerous…, 106 (+1 more)
• CANDY: A vivid, self-contained reverse-martingale parable that lands well as an elevator explanation of model-based risk blindness.

Broader Connections

• A probabilistic threat stays credible despite failures because one trial proves little #qualifies (0.78) — Thomas C. Schelling, The Strategy of Conflict, 422
• Assessing unprecedented threats through old experience leads to catastrophe. #expands (0.72) — Shoshana Zuboff, The Age of Surveillance Capitalism, 605