Human plus algorithm outperforms either alone on judgment tasks

Evidence

Further, the leadership team should emphasize that legal technology will not be used to replace attorneys, nor will it be used to replace attorney judgment. As Fountaine, McCarthy, and Saleh (2019) have written for the private sector, “When AI is adopted broadly, employees up and down the hierarchy will augment their own judgment and intuition with algorithms’ recommendations to arrive at better answers than either humans or machines could reach on their own” (p. 65). Indeed, voluminous research has found that professional judgment improves when it is coupled with data-based analyses and guidance (Dawes, Faust, & Meehl, 1989; Gendreau, Little, & Goggin, 1996; Kuncel, Klieger, Connelly, & Ones, 2013). This combination of human decision-making informed by cognitive computing is sometimes called “collaborative intelligence” (Wilson & Daugherty, 2018), and the term rightly captures how legal technology should be used: to empower attorneys by placing them in a working partnership with AI. As the CEO of a multi-billiondollar company wrote in the Harvard Business Review, “A good person plus a good algorithm is far superior to the best person or the best algorithm alone” (Lake, 2018). For example, researchers employed a deep-learning algorithm to detect metastatic breast cancer from slides of biopsies, and the algorithm was accurate 92.5 percent of the time (Kleinberg et al., 2018). A physician who reviewed the same slides was accurate 96.6 percent of the time. However, when the two systems were combined—algorithm and physician—the accuracy rate rose to a near-perfect 99.5 percent. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1194 (#corroborates, via linked claim)

Given that there is significant bias in U.S. criminal justice2 and almost no interventions have successfully curtailed it, we think the time is right to introduce technology as a way to correct for human error. For some years now, it has been known that big data and AI often outperform human judgment alone (Berk, 2008; Lazer et al., 2009). — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1171 (#complicates, via linked claim)

As a new media mogul, Jonah Peretti was idiosyncratic. He mostly obsessed over cracking the secrets of virality. As former colleague Ben Smith noted, “He wasn’t concerned with taste or quality or brand or consistency. He just wanted to know what would get traffic.” Peretti used this data-driven approach to revitalize The Huffington Post, creating a sophisticated system of search engine optimization (SEO) that catered to Google’s algorithms. They engineered articles with keyword-rich headlines, strategically named URLs, and hidden HTML elements to ensure high visibility in search results. This formula drastically increased the site’s traffic, solidifying its place as a leading digital media outlet. And here began the practice of prioritizing content to satisfy Google’s “machines” more than to cater to the long-term needs of human readers. — W. David Marx, Blank Space, 197 (#complicates, via linked claim)

Categorization

• XREF: connects to: Human judgment combined with algorithms outperforms either alone — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law: A Definitive Look at…, 1194; Even randomly weighted linear models beat human expert judgments — Brian Christian, The Alignment Problem, 441; Big data and AI outperform human judgment in high-stakes decisions. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law: A Definitive Look at…, 1171 (+1 more)

Broader Connections

• Human judgment combined with algorithms outperforms either alone #corroborates (1.00) — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1194
• Big data and AI outperform human judgment in high-stakes decisions. #complicates (0.75) — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1171
• Search-engine algorithms displaced human judgment in content prioritization #complicates (0.70) — W. David Marx, Blank Space, 197