Machine learning produces knowledge no one, even its engineers, can explain.

Evidence

How does Machine Learning work, in broad terms? The calculative power involved in today’s social knowledge is massively greater than that employed in the nineteenth and twentieth centuries; it also operates on different principles. Whereas the earlier social knowledge was built from specific calculations performed on so-called structured data (for example, entries in statistical tables and databases), today’s social knowledge can be built from unstructured data, drawn directly from the traces left in the flow of everyday life. The structure of the data has to be found through repeating and gradually refining initial “rules” (guesses) for how to categorize those traces. New rules of categorization are generated in the course of applying those initial rules. As a result, the significance of the analysis is nonlinear, “emergent … , [and] consisting of previously unknown patterns and relationships.” The end product (the knowledge generated) is not explicable in terms of rules at all; it is just a complex correlation that emerges from a calculative process operating at many different levels. As such, the operations of data analytics are necessarily opaque, and no one—not even the engineers who run the process—can account exactly for how that knowledge was generated. — Nick Couldry & Ulises A. Mejias, The Costs of Connection, 393

Categorization

• QUESTION: Worth chasing: how do the authors reconcile ‘necessarily opaque’ ML with accountability demands in the colonialism critique — is opacity itself the extractive mechanism?
• XREF: Connects to O’Neil’s ‘Weapons of Math Destruction’ algorithmic opacity critique and to attention/surveillance school’s treatment of black-box systems. — connects to: Machine-learning data operations have grown too complex for humans to comprehend or audit. — Shoshana Zuboff, The Age of Surveillance Capitalism, 1546; Machine learning increasingly intertwines technical problems with human and social questions

Broader Connections

• Machine-learning data operations have grown too complex for humans to comprehend or audit. #expands (0.92) — Shoshana Zuboff, The Age of Surveillance Capitalism, 1546
• Machine learning increasingly intertwines technical problems with human and social questions #complicates (0.75)