High correlations can conceal total disagreement in the middle of a scale.

Evidence

When there is a significant number of cases in the gray zone, using the correlation coefficient as a measure of similarity can wildly underestimate discrepancies between measures. For example, using the Polity democracy data, if one takes the cases at extreme values (i.e., − 10 and 10) as given, which consist of 23 percent of the data, and then replaces all the observations in between with independent, random, and uniform data, one still gets a correlation coefficient of almost 0.50. In short, there can exist extensive disagreement between measures in the gray zone and one can still get quite respectable correlation coefficients. — Gary Goertz, Social Science Concepts and Measurement, 1100

Categorization

• DEEPTHINKING: The randomized-middle simulation inverts the standard validation instinct: a 0.50 correlation survives when the gray zone is pure noise.