Long scales achieve acceptable reliability without valid unidimensional measurement
Evidence
The only reason the Dogmatism Scale achieves reliabilities around .80, even with response sets producing some artificial internal consistency, is because it has so many items. You can produce “acceptable reliability” among a set of barely connecting statements that measure gobs of different things if you simply assemble enough of them. — Bob Altemeyer, The Authoritarian Specter, 4244
Categorization
Category: Factual Claims
- XREF: Connects to the general critique of psychometric instruments like the F Scale and RWA scale central to this whole book.
- SEED: An essay on how raw item count manufactures fake reliability in personality inventories, with examples from bad corporate tests.
Related
- High reliability coefficients can be manufactured by item count regardless of construct coherence (similarity 0.91)
- The 40-item Dogmatism Scale has weak internal consistency (similarity 0.82)
- Shortened versions of the Dogmatism Scale failed to represent the full scale (similarity 0.82)
- Empirically-selected dogmatism scale items fail to replicate across samples, performing at chance level (similarity 0.82)
- Poor measurement reliability doomed dogmatism research to weak, unstable findings (similarity 0.82)
- The Dogmatism Scale lacked construct validity due to low consistency and yea-saying bias. (similarity 0.80)
- Scale-shortening item selections are unstable across different samples (similarity 0.80)
- Rokeach’s famous Dogmatism Scale lacked construct validity from its inception (similarity 0.79)
- A flawed psychological test can stay popular if it correlates with interesting variables. (similarity 0.79)
- Unidimensional attitude scales allow reliable cross-scale inference from single survey items (similarity 0.79)
- Unbalanced scales inflate reliability alphas and correlations through acquiescence (similarity 0.78)
- Rokeach’s D Scale lacked clean evidence supporting its core premise from the start. (similarity 0.78)
- A single-page study with no reliability data was treated as validating the D Scale. (similarity 0.78)