Automated personality profiling can threaten well-being, freedom, or even life.

Evidence

The authors appeared increasingly ambivalent about the social implications of their work. On the one hand, they announced that these new predictive capabilities could be used to “improve numerous products and services.” They concluded that online businesses can adjust their behavior to match each user’s personality, with marketing and product recommendations psychologically tailored to each individual. But the authors also warned that automated prediction engines run by companies, governments, or Facebook itself can compute millions of profiles without individual consent or awareness, discovering facts “that an individual may not have intended to share.” The researchers cautioned that “one can imagine situations in which such predictions, even if incorrect, could pose a threat to an individual’s well-being, freedom, or even life.” — Shoshana Zuboff, The Age of Surveillance Capitalism, 856

Categorization

• QUESTION: The ‘even if incorrect’ caveat is striking — false predictions about an individual can still be harmful. What mechanisms make an incorrect prediction dangerous rather than merely wrong?
• XREF: connects to: Most US job applicants face automated personality screening before any human contact — Nick Couldry & Ulises A. Mejias, The Costs of Connection: How Data Is…, 228; Automated surveillance threatens the open-ended self-transformative capacity that defines selfhood — Nick Couldry & Ulises A. Mejias, The Costs of Connection: How Data Is…, 489; Knowing more about a person makes controlling them easier, threatening autonomy — Shoshana Zuboff, The Age of Surveillance Capitalism, 599

Broader Connections

• Automated surveillance threatens the open-ended self-transformative capacity that defines selfhood #expands (0.85) — Nick Couldry & Ulises A. Mejias, The Costs of Connection, 489
• Knowing more about a person makes controlling them easier, threatening autonomy #expands (0.80) — Shoshana Zuboff, The Age of Surveillance Capitalism, 599