Inconsistent research findings often signal underlying measurement error rather than real variation

Evidence

Overall, the quest construct has considerable promise as a “moderator variable” for separating nonprejudiced from prejudiced religious people. But when you see such inconsistency in the data, it often means the numbers are almost buzzing with measurement error. I would opine that the construct needs to be measured better. The scale should be balanced against response sets and have its internal consistency boosted. In the studies reported above, the alpha of the scale varied from . (yes!), to .69. This is like fastening big lead weights to a brick and telling it to swim. — Bob Altemeyer, The Authoritarian Specter, 2057

Categorization

• CANDY: The brick-swimming metaphor for poor scale reliability is a delightfully vivid, self-contained image.
• XREF: connects to: Measurement instrument choice drastically changes forgiveness findings — Nyla R. Branscombe and Bertjan Doosje (eds.), Collective Guilt: International…, 1375; Contradictory siting studies differed mainly over geographic unit and comparison scope — Luke W. Cole and Sheila R. Foster, From the Ground Up: Environmental…, 300