Algorithmic discrimination affects real people despite targeting only probabilistic data doubles.

Evidence

Even though the new social knowledge is produced through operations that bypass human beings, it is actual human beings who are tethered to the discriminations that such knowledge generates. Based on algorithmic reasoning, it is a real person who gets sent to jail, refused health insurance, or offered a favorable price in the supermarket or an opportunity for social housing. Indeed, the gap between the uncertainty of cause and the certainty of personal impact is what makes algorithmic power so unnerving. — Nick Couldry & Ulises A. Mejias, The Costs of Connection, 408

Categorization

• XREF: The ‘uncertainty of cause, certainty of impact’ paradox parallels the surveillance school’s opacity/black-box literature (e.g., Pasquale’s black box society). — connects to: Algorithmic risk prediction individualizes insurance and erodes social solidarity — Aris Komporozos-Athanasiou, Speculative Communities: Living with…, 421; Most US job applicants face automated personality screening before any human contact — Nick Couldry & Ulises A. Mejias, The Costs of Connection: How Data Is…, 228; Machine learning systems are not inherently fair or just by default (+7 more)

Broader Connections

• Automated recommendation systems reproduce and intensify offline racial discrimination. #expands (0.85) — Grant Bollmer, Katherine Guinness, The Influencer Factory, 36
• Machine learning systems are not inherently fair or just by default #expands (0.80)
• Algorithmic systems deepen racial inequality by rendering rather than erasing racialized bodies #expands (0.80) — Aris Komporozos-Athanasiou, Speculative Communities, 93
• Privately controlled data extraction produces social classifications that entrench inequality #expands (0.80) — Nick Couldry & Ulises A. Mejias, The Costs of Connection, 443
• Machine-built data doubles work like human doppelgangers built from others’ perceptions #expands (0.80) — Naomi Klein, Doppelganger, 273
• Bias-correction algorithms can equalize treatment while masking latent power to discriminate #qualifies (0.75)