One-directional survey items make scales falsely look valid and consistent

Evidence

So unbalanced scales can easily “work” (produce statistically significant connections with something else), whether they have much construct validity or not. In addition, direction-of-wording effects would provide gravy for this meat loaf they serve up. Like most response biases, acquiescence would raise the correlations among the items on a unidirectionally worded test, making it look more internally consistent than the content really is. — Bob Altemeyer, The Authoritarian Specter, 776

Categorization

• XREF: Methodological warning relevant to evaluating scale validity in the vault’s empirical sources. — connects to: Survey questions deemed comparable still shift meaning across waves — Ilyana Kuziemko and Ebonya Washington, Why Did the Democrats Lose the South?…, 1325; Treating Likert responses as interval data distorts estimates when aggregating across disparate items — Ruiqian Li and Paul Froese, The Duality of American Christian…, 96