Long scales can show acceptable reliability while measuring many unrelated things

Evidence

The only reason the Dogmatism Scale achieves reliabilities around .80, even with response sets producing some artificial internal consistency, is because it has so many items. You can produce “acceptable reliability” among a set of barely connecting statements that measure gobs of different things if you simply assemble enough of them. — Bob Altemeyer, The Authoritarian Specter, 4244 (#expands, via linked claim)

So unbalanced scales can easily “work” (produce statistically significant connections with something else), whether they have much construct validity or not. In addition, direction-of-wording effects would provide gravy for this meat loaf they serve up. Like most response biases, acquiescence would raise the correlations among the items on a unidirectionally worded test, making it look more internally consistent than the content really is. — Bob Altemeyer, The Authoritarian Specter, 776 (#complicates, via linked claim)

Categorization

• XREF: connects to: Scales with high Cronbach-α may still conceal several distinct latent factors. — Ruiqian Li and Paul Froese, The Duality of American Christian…, 84; Long scales achieve acceptable reliability without valid unidimensional measurement; High reliability coefficients can be manufactured by item count regardless of construct coherence — Bob Altemeyer, The Authoritarian Specter, 4244 (+2 more)

Broader Connections

• High reliability coefficients can be manufactured by item count regardless of construct coherence #expands (0.95) — Bob Altemeyer, The Authoritarian Specter, 4244
• Long scales achieve acceptable reliability without valid unidimensional measurement #expands (0.92)
• One-directional survey items make scales falsely look valid and consistent #complicates (0.72) — Bob Altemeyer, The Authoritarian Specter, 776