Balanced scales understate a test’s real internal consistency

Evidence

By the same token, yea-saying (and naysaying) would reduce the correlations between protrait and contrait items on balanced tests (for example, +3 on a protrait makes you look high on the trait; +3 on a contrait makes you look the opposite). Thus balanced tests would look Jess internally consistent than the content really is. And the reduction in correlation between protrait and contrait items would tend to separate them onto different factors in a factor analysis, even if they were largely tapping the same thing. — Bob Altemeyer, The Authoritarian Specter, 779

Categorization

Category: Causal Relationships

• DEFINE: Clarifies protrait/contrait wording and its measurement consequences.

Related

• Unbalanced scales inflate reliability alphas and correlations through acquiescence (similarity 0.89)
• Unbalanced scales can show spurious significance without true construct validity (similarity 0.88)
• Unbalanced survey scales allow response bias to inflate apparent reliability (similarity 0.85)
• One-directional survey items make scales falsely look valid and consistent (similarity 0.85)
• Response bias makes balanced surveys appear spurious multifactor structures (similarity 0.84)
• Balanced scales make agreement rates independent of trait strength in the sample (similarity 0.82)
• A shared psychological trait connects RWA Scale items despite diverse content (similarity 0.79)
• Improving a questionnaire’s internal consistency substantially strengthens its measured correlations (similarity 0.79)
• Unidimensionality enables causal explanation rather than mere correlation (similarity 0.78)