Post-hoc category derivation need not invalidate converging findings.

Evidence

There is only one criticism that applies to all four studies: the derivation of scoring categories from prior examination of data. What the authors ought to have done in their first study is to examine a part of their data in search of discriminating categories and use the remainder to test. The four studies taken together suggest that the categories would have survived such a test. Suppose we consider one of the studies, for example the main interview study, as the preliminary examination of data in search of categories. Since the other studies employ some categories that are the same as or closely similar to those used with the main interviews it would seem that the authors could have used these studies as tests and need not have made preliminary examinations of data. The fact that they did make such examinations does not completely vitiate the force of the convergence in the findings. — John T. Jost and Jim Sidanius, Political Psychology, 648

Categorization

• DEFINE: The single criticism common to all four Berkeley studies is the derivation of scoring categories from prior examination of the same data — a case of researcher degrees of freedom.