Consumer Reports speaker ratings were negatively correlated with blind-test fidelity ratings

Evidence

In 1985 I hosted a loudspeaker measurement seminar at the NRCC, where we showed and demonstrated our measurement capabilities and had lectures on things that were believed to be measurable and important to what we heard and preferred. The ultimate test was a double-blind subjective evaluation—ears and brains determined what was good, not the appearance of a curve. We also demonstrated those. The audience was a who’s who of loudspeaker designers and researchers from the US and Canada, and there were animated discussions. Among them were two engineers from Consumer Reports. Knowing they were coming, I had an overhead transparency prepared. For four loudspeakers that both of us had evaluated, it showed the subjective “fidelity” rating from our double-blind listening tests, and the “accuracy score” from their publication. The correlation was −0.7 (negative 0.7!). I still have the transparency. There was discussion, some puzzlement on their part, but nothing changed at the magazine. Based on this comparison, readers would have been well advised to invert the page of ratings. — , Toole, Floyd E - Sound reproduction-…, 583 (#corroborates, via linked claim)

Categorization

• XREF: connects to: Consumer Reports loudspeaker ratings lacked rigorous subjective validation; Consumer Reports loudspeaker ratings inversely predicted perceived sound quality. — , Toole, Floyd E - Sound reproduction-…, 583

Related

• Consumer Reports loudspeaker ratings lacked rigorous subjective validation (similarity 0.95)
• Consumer Reports loudspeaker ratings inversely predicted perceived sound quality. (similarity 0.95)
• Consumer Reports’ accuracy scores inversely correlated with actual listener preferences. (similarity 0.90)
• Consumer loudspeakers rated higher than professional monitors in blind tests (similarity 0.90)
• Sighted loudspeaker evaluations are demonstrably unreliable (similarity 0.84)
• Audible resonances drove low sound-quality ratings in early blind tests (similarity 0.83)
• Blind listener tests reliably favor the most neutral loudspeakers (similarity 0.83)
• Speaker cables and devices show near-zero audible differences in blind tests (similarity 0.82)
• Sound power alone cannot predict listener preference for loudspeakers (similarity 0.82)
• Consumer Reports rated loudspeakers using sone-converted sound power against an ideal target (similarity 0.82)
• Roughly 75 percent of people have sufficiently normal hearing to make reliable loudspeaker judgments (similarity 0.81)
• Objective measurements routinely expose flaws that subjective listening reviews miss (similarity 0.80)
• Historically, published loudspeaker specifications were often artistic rather than factual. (similarity 0.80)
• High-end audio forfeited credibility by refusing blind testing (similarity 0.80)
• Blind listeners largely agreed on loudspeaker preferences, matching measurement quality (similarity 0.80)

Broader Connections

• Consumer Reports loudspeaker ratings inversely predicted perceived sound quality. #corroborates (0.95) — , Toole, Floyd E - Sound reproduction-…, 583
• Consumer Reports loudspeaker ratings lacked rigorous subjective validation #corroborates (0.78)