Algorithms’ behavior is opaque even to their own programmers.

Evidence

How exactly they do so is entirely opaque. Even the people who write these algorithms do not understand it: once the algorithm is in motion, the scale of the data involved and the speed at which it is processed would make it impossible for any human to trace its route through such a vast tree of ever-proliferating decisions, even if they did have full access to a full record of its activity. But left to their own devices, constantly monitoring and incessantly reacting to the outcomes of their own actions, and then to the outcomes of their reactions, these ‘algos’, as they’re known, have acquired some astonishing capacities that their own coders and programmers find hard to understand. There is nothing new here, however: remember how the financial engineers of the 1990s and 2000s used algorithms to create derivatives of such enormous complexity that they themselves had no way of knowing what was inside those derivatives? Similarly, the engineers coding Alexa-like, cloud-based devices for the purpose of creating automated systems that modify our behaviour are building so much complexity into these systems that they don’t really understand exactly why their systems do what they do. — Yanis Varoufakis, Technofeudalism, 313

Categorization

• QUESTION: If even coders cannot trace or explain these adaptive systems, what are the regulatory and accountability implications for platform power? Worth chasing.
• XREF: Direct parallel to the 2008 financial crisis derivatives opacity described in Varoufakis’s financialization work; also resonates with Black-box AI critiques from elsewhere. — connects to: Algorithms obscure power asymmetries more opaquely than print ever did.; Dark pools and algorithmic trading remain opaque even to most traders — Aris Komporozos-Athanasiou, Speculative Communities: Living with…, 92; Apparent complexity masks hidden relational code rules — Almo Farina, Soundscape Ecology: Principles,…, 685

Broader Connections

• Algorithms obscure power asymmetries more opaquely than print ever did. #corroborates (0.90)
• Dark pools and algorithmic trading remain opaque even to most traders #expands (0.85) — Aris Komporozos-Athanasiou, Speculative Communities, 92
• Apparent complexity masks hidden relational code rules #expands (0.80) — Almo Farina, Soundscape Ecology, 685