No single statistical definition of fairness can satisfy all reasonable criteria simultaneously

Evidence

Indeed, different base rates across different legally protected groups are major complication for fairness in predictive analytics (see Berk et al., 2018). It is a primary factor in the so-called “group fairness impossibility theorem” (see Chouldechova, 2017), which arises from the fact that different groups nearly always have different base rates, and different base rates, absent perfect prediction, will lead to different error rates (we can think of false positives and false negatives). Pursuing this line of thought, Kleinberg, Mullainathan, and Raghavan (2017) outlined various mathematical definitions of fairness that could reasonably be used in the criminal justice context—and they showed that these three definitions could not all be operationally satisfied at the same time. Likewise, Verma and Rubin (2018) discussed twenty different definitions of fairness. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1155

Categorization

• DEEPTHINKING: The group fairness impossibility theorem is a profound result: in a disparate-base-rate world, fair criteria conflict irreducibly — hugely relevant to the reader’s legal-bias research.
• corroborated: 2 independent passages support this claim — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law: A Definitive Look at…, 1155
• XREF: connects to: No single all-inclusive theory of environmental and ecological justice is required — David Schlosberg, Defining Environmental Justice:…, 44