Algorithms that correct documented bias may themselves violate equal protection law

Evidence

Some might argue that, even if the steps outlined above are followed, there are insurmountable problems with using legal technology to decrease bias. For instance, we know that when there are similarly situated male and female defendants, male defendants receive more severe sentences. If a prosecutor’s office built out their data collection and analysis capabilities, they would be able to quantify the extent of this pro-female bias. But if they installed an algorithm that corrected for this bias, this could be considered a violation of the Equal Protection Clause (see Starr, 2014; Oleson, 2011). If we imagine parallel and analogous algorithmic uses, we might find violations of Title VII. Indeed, it seems that algorithmic correction would be a form of discrimination—or so goes the argument. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1207

Categorization

• XREF: Connects to the reader’s interest in whether rational/moral correction of bias backfires — here correction is legally proscribed, a legal-institutional instance of the conviction-vs-practice divergence theme. — connects to: US law may criminalize researchers running platform fairness tests — Nick Couldry & Ulises A. Mejias, The Costs of Connection: How Data Is…, 441
• corroborated: 3 independent passages support this claim — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law: A Definitive Look at…, 1207