Survey questions deemed comparable still shift meaning across waves

Evidence

As noted earlier, the ANES cumulative file includes questions from its individual year files if those questions are deemed reasonably comparable and were repeated with sufficient frequency. The only question related to civil rights that spans our preand post-periods that the ANES deems comparable over time asks whether the federal government should ensure school integration. It covers only a single pre-period year (1962) and is then asked most years from 1964 through 2000. … Even though the ANES deems the question comparable from 1962 onward, non-trivial differences arise year to year. For example, in 1962 supporting integration but “not by force” is an option (and coded as support), whereas in 1964 that option is not offered. In 1964, the justification of it not being the “government’s business” is introduced, but this wording is not included in 1962. — Ilyana Kuziemko and Ebonya Washington, Why Did the Democrats Lose the South?…, 1325

Categorization

• DEFINE: Concrete example of how a ‘comparable’ survey item changes its meaning year to year through response options.
• XREF: Relevant to methodological caveats about using repeated cross-sectional survey items as panel-like evidence in the threatened-privilege research agenda. — connects to: Identical survey responses can stem from opposite interpretations — Bob Altemeyer, The Authoritarian Specter, 3977; One-directional survey items make scales falsely look valid and consistent — Bob Altemeyer, The Authoritarian Specter, 776; Question wording and historical context make cross-year race-attitude survey items non-comparable. — Ilyana Kuziemko and Ebonya Washington, Why Did the Democrats Lose the South?…, 1330 (+2 more)

Broader Connections

• Question wording and historical context make cross-year race-attitude survey items non-comparable. #corroborates (0.90) — Ilyana Kuziemko and Ebonya Washington, Why Did the Democrats Lose the South?…, 1330
• Identical survey responses can stem from opposite interpretations #expands (0.85) — Bob Altemeyer, The Authoritarian Specter, 3977
• Vague survey wording can make racial prejudice undetectable in response data #corroborates (0.80)
• One-directional survey items make scales falsely look valid and consistent #exemplifies (0.75) — Bob Altemeyer, The Authoritarian Specter, 776
• Survey research is vulnerable to systematic measurement errors that skew results #expands (0.75)