Publication bias inflates the apparent effectiveness of prejudice-reduction programs

Evidence

Finally, it is probable that studies finding no effects or negative effects were less likely to have been published than studies finding positive effects, which may lead to overestimating the favorable effects of these programs. — , Rudman, Laurie A - On the Nature of…, 2670

Although the results of this meta-analysis are encouraging, they must be regarded as tentative. All of the programs were quite different from one another, making it difficult to compare across programs. The programs employed an amazing array of different measures of intergroup relations. A number of these measures are not well-established and their reliability and validity remain in question. Nearly all of these studies used self-report measures of attitudes which are subject to social desirability biases. Although many of these studies employed pretest/post-test designs with experimental and control groups, few of them employed random assignment and a number of studies employed weaker designs (pre/post designs or post-test/ control group designs). Thus, nearly all of them are subject to alternative explanations of one type or another. In addition, the sample sizes were sometimes rather small. Moreover, many of these programs were conducted by inexperienced personnel or were evaluated during their inaugural application. It seems likely that these programs would have had a greater impact if they had been conducted by experienced personnel or if they had been evaluated after they had been in use for a considerable period of time. Finally, it is probable that studies finding no effects or negative effects were less likely to have been published than studies finding positive effects, which may lead to overestimating the favorable effects of these programs. — , Rudman, Laurie A - On the Nature of…, 2669

Categorization

• XREF: The file-drawer problem in meta-science connects to replication crisis literature and to Bern’s classic work on publication bias. — connects to: Industry cherry-picking is continuous with ordinary scientific publication bias — Cailin O’Connor and James Owen Weatherall, The Misinformation Age: How False…, 359; Cross-cutting exposure fails because outgroup posts are predominantly derogatory; Group esteem threat effects may reflect publication bias — Blake M. Riek, Eric W. Mania, and Samuel L. Gaertner, Intergroup Threat and Outgroup…, 94
• corroborated: 2 independent passages support this claim — , Rudman, Laurie A - On the Nature of…, 2670; , Rudman, Laurie A - On the Nature of…, 2669