Offices deploying legal AI should publish quarterly bias metrics like financial reports

Evidence

If the goal is to rid the office of racial bias in case outcomes, then the question undergirding the metrics is clear: Are similarly situated defendants being treated similarly? While there are a few different ways of assessing this, the office must decide which ones to take (and using more than one is certainly appropriate), and it must produce quarterly reports, similar to a financial accounting, that describes how the office is doing in terms of these metrics. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1201

Categorization

• SEED: An essay angle: the proposal to treat algorithmic bias auditing like financial accounting — what happens when bias metrics become regulated, standardized disclosures?