Competing statistical fairness definitions can conflict irreconcilably within one algorithm

Evidence

Our first definition of fairness is “group fairness.” We can say that a classifier satisfies this definition if both protected (e.g., Black defendants) and unprotected (White defendants) groups have equal probabilities of being assigned to the positive predicted class (Verma & Rubin, 2018). … But group fairness considers only predicted outcomes; it fails to include a consideration of actual outcomes. Our second definition of fairness is “predictive parity” that considers both predicted and actual outcomes. A classifier would satisfy this definition if both Black and White defendants have equal positive predictive value (PPV); that is, for both Black and White defendants, the probability that a defendant who received a good score will actually refrain from additional crimes and show up for subsequent court dates should be the same. … While this definition is appealing, the problem is that it may be in conflict with the group fairness definition. It is possible to have predictive parity but fail to satisfy group fairness—and vice versa. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1157

Categorization

• DEFINE: Defines group fairness versus predictive parity as two distinct, mutually conflicting statistical definitions.
• XREF: Connects to the broader fairness-in-ML impossibility literature (e.g., Kleinberg et al. showing group fairness, predictive parity, and calibration cannot all hold together).
• corroborated: 2 independent passages support this claim — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law: A Definitive Look at…, 1157