US law requires governmental decisions be explainable, constraining opaque AI use

Evidence

In short, it can be hard to understand the machine’s decision process. This raises deep questions relating to the imperative—long established in U.S. law—that governmental decision-making be explainable (see Crawford & Schultz, 2014; Selbst & Powles, 2017). Thus, offices must hesitate to use fully opaque technologies (see Citron & Pasquale, 2014; Edwards & Veale, 2017); and, if any such technology is used, it should be analyzed by researchers working on model explanations, and the analyses should be relayed to interested or affected parties. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1199

Categorization

• XREF: Builds a direct line from the COMPAS controversy (Loomis) to a general legal explainability doctrine — connects to the earlier chapter on algorithmic sentencing bias. — connects to: US law may criminalize researchers running platform fairness tests — Nick Couldry & Ulises A. Mejias, The Costs of Connection: How Data Is…, 441; Meaningful contestation of algorithms requires new machine-grade regulatory power. — Shoshana Zuboff, The Age of Surveillance Capitalism, 1547