Big data correlations alone cannot deliver causal knowledge to prediction systems

Evidence

Remember that Google economist Hal Varian extolled the “new uses” of big data that proceed from ubiquitous computer-mediated transactions. Among these he included the opportunity for “continuous experimentation.” Varian noted that Google has its engineering and data science teams consistently running thousands of “A/B” experiments that rely on randomization and controls to test user reactions to hundreds of variations in page characteristics from layout to buttons to fonts. … Varian endorsed and celebrated this self-authorizing experimental role, warning that all the data in the world “can only measure correlation, not causality.” Data tell what happened but not why it happened. In the absence of causal knowledge, even the best predictions are only extrapolations from the past. — Shoshana Zuboff, The Age of Surveillance Capitalism, 930

Categorization

• XREF: Correlates to the Milgrom law-merchant material about information costs and institutions filling gaps; here Zuboff shows prediction markets needing experiment to bridge correlation-causality gap. — connects to: Machine learning underlies Big Data and drives today’s massive investment. — Nick Couldry & Ulises A. Mejias, The Costs of Connection: How Data Is…, 392; Big Data logic rejects theory-driven hypothesis testing in favor of raw pattern extraction — Nick Couldry & Ulises A. Mejias, The Costs of Connection: How Data Is…, 394; Guaranteed prediction products require causal knowledge gained through experiments — Shoshana Zuboff, The Age of Surveillance Capitalism, 931 (+10 more)
• SEED: Essay angle: the correlation/causality gap as the economic engine of the experimenter’s right — why data corporations are structurally pushed into running experiments on users.

Broader Connections

• Guaranteed prediction products require causal knowledge gained through experiments #corroborates (0.93) — Shoshana Zuboff, The Age of Surveillance Capitalism, 931
• Base-rate training data can mislead models about causality #corroborates (0.78)
• Large samples produce significant correlations that explain little variance #expands (0.78) — Roderick M. Kramer, Geoffrey J. Leonardelli, Robert W. Livingston, Social Cognition, Social Identity, and…, 1036
• Social prediction depends on social rationality, not just method #expands (0.75) — Martin Jay, The Dialectical Imagination, 300
• Shared acquiescence can create spurious correlations between any two measures #exemplifies (0.75)
• A liberatory universalism opposes Big Data’s totalizing reduction of life to data #expands (0.72) — Nick Couldry & Ulises A. Mejias, The Costs of Connection, 637