Field realism and participant diversity strengthen an experiment’s external validity

Evidence

Over 20 years ago, Carl Hovland urged that the study of communication be based on field and experimental research (Hovland, 1959; also see Converse, 1970). We agree. Of course, experimentation has problems of its own, which our studies do not fully escape. That our participants represent no identifiable population, that our research setting departs in innumerable small ways from the natural communication environment, that the news programs we created might distort what would actually be seen on network newscasts—each raises questions about the external validity of our results. … We took care to avoid a standard pitfall of experimentation—the so-called college sophomore problem— by encouraging diversity in experimental participants. We undertook extra precautions to recreate the natural communication environment: participants watched the broadcasts in small groups in an informal and relaxed setting. And we were careful not to tamper with standard network practice in constituting our experimental presentations. — John T. Jost and Jim Sidanius, Political Psychology, 1504

Categorization

• DEFINE: Names the ‘college sophomore problem’ — overreliance on undergraduates as experimental participants, threatening generalizability.
• XREF: Connects to methodological pluralism debates in social science, mirroring the triangulation arguments common in mixed-methods literature.