Unbalanced scales can show spurious significance without true construct validity
Evidence
So unbalanced scales can easily “work” (produce statistically significant connections with something else), whether they have much construct validity or not. In addition, direction-of-wording effects would provide gravy for this meat loaf they serve up. Like most response biases, acquiescence would raise the correlations among the items on a unidirectionally worded test, making it look more internally consistent than the content really is. — Bob Altemeyer, The Authoritarian Specter, 776
Categorization
Category: Causal Relationships
- DEFINE: Concisely explains why unidirectional wording makes a test look more reliable than its content warrants.
Related
- One-directional survey items make scales falsely look valid and consistent (similarity 0.92)
- Balanced scales understate a test’s real internal consistency (similarity 0.88)
- Unbalanced scales inflate reliability alphas and correlations through acquiescence (similarity 0.86)
- Unbalanced survey scales allow response bias to inflate apparent reliability (similarity 0.81)
- Unidimensionality enables causal explanation rather than mere correlation (similarity 0.80)
- Balanced scales make agreement rates independent of trait strength in the sample (similarity 0.79)