Auditory-model quality metrics fail on acoustically captured signals

Evidence

As the method is trained with a particular set of examples of reference and impaired audio samples, the method has a specific area of applicability. PEAQ has been used successfully to analyse audio quality with masking-based audio codecs without the effect of the listening room. For example, PEAQ estimates low MOS values for signals containing jitter or a slight deviation in the sampling frequency. However, PEAQ cannot operate with acoustically captured signals. It is probable that PEAQ would estimate a degraded MOS if the ear canal signals of a listener in a domestic environment were compared to the ear canal signals of the audio engineer in the mastering studio, although the listener would be satisfied with the audio quality. — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 2373

Categorization

• DEFINE: Clarifies what PEAQ can and cannot do, demarcating its limited area of applicability.
• XREF: Limits of instrument-based quality measures parallel the Milgrom point that full-information institutions are too costly — bounded applicability of tools.