Consensus-based sensory panels risk leader bias but training can counter it

Evidence

Flavour profile method (Cairncross and Sjöström, 1950): A major component of the flavour profile method is a highly trained panel of four to six members, who individually evaluate the stimuli and then work in discussion as a group to determine a consensus profile. This consensus leads to data that act as a representative value; this it is not an average of the panellists; scores, it is a single score agreed upon by all panel members. This component of the profile method was criticized in the 1960s and 1970s as offering too much potential for the panel leader or an opinionated panellist to introduce bias. It is also claimed that the appropriate selection of panellists, extensive training, and the blind nature of the testing can protect against bias. — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 938

Categorization

• XREF: Group consensus vs. averaged scores parallels the reputation systems and coordination problems from the law-merchant readings.