Omitted-variable critiques of bias studies have weakened with richer data

Evidence

This is a troublesome result, and it is understandable that objections can be raised. For instance, some might argue that it is impossible for researchers to control for every single factor that can matter in a judge’s decision; there could be additional features of the case that can be observed by the judge but unobserved by researchers, and these omitted variables could be driving the findings. This objection, while reasonable, does not have the power it used to have thirty years ago when empirical analyses were much less sophisticated and big and detailed data sets were not available. This “omitted variable bias” argument has less credibility now because researchers apply increasingly sophisticated statistical techniques to large data sets which include rich information on the case: most, if not all, of the information that is available to the judge. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 778

Categorization

• XREF: Parallels the general debate over observational vs. experimental inference; echoes Pearl’s omitted variable treatment in causal inference. — connects to: Critics charged that sampling only high and low scorers biased the results — Martin Jay, The Dialectical Imagination: A History…, 744; Failing to control for population density produces spurious no-disparity findings in waste-siting studies.