Reliable panel listeners show low variance and wide rating ranges

Evidence

If one is establishing a population of listeners from which to draw over the long term, it will be necessary to monitor their decisions, looking for those who (a) exhibit small variations in repeated judgments and (b) differentiate their opinions of products by using a large numerical range of ratings (Toole, 1985; Olive, 2001, 2003). — Toole, Floyd E, Sound reproduction- the acoustics and…, 1164

Categorization

• SEED: A method-oriented essay on how to build and curate a reliable listening panel for subjective product evaluation.