Meeting critics’ methodological demands strengthened rather than weakened the adverse finding
Evidence
Second: the 2016 meta-analysis’s design was shaped by the 2002 critics’ objections. This is the correct way for science to work: not the dismissal of critics but the engagement with their specific methodological claims in a study designed to test them. The critics asked for a narrower definition of the behavior, a more rigorous statistical method, and more longitudinal data. Gershoff and Grogan-Kaylor provided all three. The result was not a more equivocal finding. It was a stronger adverse association and the elimination of the only beneficial outcome. The critics’ demands, when met, produced a result that more completely refuted their position than the 2002 paper had. This is the dialectical process of scientific progress working as it is supposed to work, and it is worth marking as such. — , The Rod and the Child, 9593
Categorization
- XREF: A template of scientific dialectic that generalizes to other contested research fields like gun violence and vaccine efficacy mentioned in this chapter at 9569. — connects to: Partial debiasing can make bias problem worse by hiding it — Brian Christian, The Alignment Problem, 210; Critics charged that sampling only high and low scorers biased the results — Martin Jay, The Dialectical Imagination: A History…, 744
- SEED: Angle for an essay: when critics’ demands, when actually met, produce results that more completely refute the critics’ position — the rare case where rigor rewards the original finding.
Related
Broader Connections
- Partial debiasing can make bias problem worse by hiding it
#complicates(0.35) — Brian Christian, The Alignment Problem, 210