Correlations between measures overstate similarity because extremes dominate.

Evidence

When comparing various concepts and measures one usually finds that correlation coefficients are used to assess similarity. This procedure often dramatically underestimates the dissimilarity of measures. One reason for this is that observations at the ends of the spectrum usually have more weight (in statistical terms more leverage; Belsley et al. 1980) than those in the middle. It is often the case that concepts and measures agree on the extreme cases since they are clear-cut and easy to code, while at the same disagreeing frequently on cases in the middle. Points in the middle often have a “half fish, half fowl” character that makes them hard to categorize and classify. — Gary Goertz, Social Science Concepts and Measurement, 1089

Categorization

• DEEPTHINKING: A quietly devastating point about a ubiquitous method: the number everyone trusts is inflated by the easy cases at the tails.