Self-reported training satisfaction cannot measure behavioral change.

Evidence

Our experience working with stakeholders in the CJS has suggested that insights from cognitive science have the potential to catalyze transformation of policies and practices linked to racial equity. We conduct post-workshop evaluations that indicate an understanding of the concepts, receptivity to engaging in the interventions suggested, and an internal motivation to address discrimination in general. While we find those measures generally positive, in our view, they are not sufficient to ensure that this approach is truly having outcome-relevant effects. Going forward, we are planning to engage in methodologically rigorous empirical study of this work rather than relying upon anecdotal information and the self-reports of evaluations. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1413 (#exemplifies, via linked claim)

I think we can lay to rest the suspicion that the students did not really change their attitudes in all these studies, but merely pretended to, to please the Experimenter. For subjects here were still affected by an article that the Experimenter condemned—not as much as when the Experimenter insisted he be considered neutral, of course, but almost as much. — Bob Altemeyer, The Authoritarian Specter, 3256 (#complicates, via linked claim)

Categorization

• XREF: connects to: Self-reported receptivity to anti-discrimination training fails to measure real behavioral change. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law: A Definitive Look at…, 1413; Bias trainings change self-reported knowledge but rarely alter behavior — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law: A Definitive Look at…, 1415; Congruence and responsiveness scores cannot distinguish responsiveness to wanted change from unwanted change. — Mikael Persson and Anders Sundell, Blocking the Poor: Status Quo Bias in…, 46 (+9 more)

Broader Connections

• Self-reported receptivity to anti-discrimination training fails to measure real behavioral change. #exemplifies (0.95) — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1413
• Attitude changes in experiments are genuine, not mere compliance #complicates (0.74) — Bob Altemeyer, The Authoritarian Specter, 3256