Vision and sound expectations must concord for convincing audio-scene perception

Evidence

Psychoacoustic impressions depend on sight as well as sound. This is not just a matter of viewing the performers. When we enter a soundspace, we expect certain characteristics of it (concordance of vision and sound characteristics). No one knows this better than the sound engineers in Hollywood, who very carefully match the reverberation characteristics to location. Often, much of the voice recording needs to be redone post-production in a sound studio and later dubbed over the moving picture. Applying the correct reverberations to the “dry” voice signal coming from the studio is crucial, and quite an art. This is achieved using either a measured or computed impulse response for the source and microphone positions in a given space. — Eric J. Heller, Why You Hear What You Hear- An…, 476

Categorization

Category: Causal Relationships

• XREF: Connects to multimodal perception literature; reverberation-matching is the inverse of ventriloquism effect research.

Related

• Matching audio to a visual scene requires scene-appropriate impulse responses. (similarity 0.90)
• Visual cues override directional sound cues in film dialog perception (similarity 0.85)
• Visual size cues override matching audio reverberation judgments (similarity 0.85)
• Auditory expectation of reverberation is calibrated against visual room size. (similarity 0.83)
• Conflicting visual and audio size cues make hybrid reverberation sound wrong. (similarity 0.83)
• Vision dominates hearing in localization, identification, and size judgment (similarity 0.82)
• Room acoustics and presentation method alter how listeners perceive sounds (similarity 0.82)
• Recording fidelity matters less than room acoustics and head movement for perceived liveness (similarity 0.82)
• Direct sound determines localization, loudness, and perceived spectrum quality (similarity 0.82)
• Physical accuracy of a sound field does not predict perceived accuracy (similarity 0.82)
• Reverberation multiplies perceived speech loudness well beyond measured level gains (similarity 0.82)
• Listeners continuously re-normalize distorted HRTFs using visual and motion cues (similarity 0.81)
• Room acoustics smear directional perception and mask recorded reverberation (similarity 0.81)
• Auditory cortex achieves noise-and-reverberation-invariance that brainstem processing lacks. (similarity 0.81)
• Listeners perceptually normalize voices across changing room acoustics via direct sound and visual cues (similarity 0.81)

Broader Connections

• Auditory expectation of reverberation is calibrated against visual room size. #expands (0.90) — David Howard, Jamie Angus, coustics and Psychoacoustics, 1876
• Matching audio to a visual scene requires scene-appropriate impulse responses. #expands (0.85) — Eric J. Heller, Why You Hear What You Hear- An…, 2526
• Visual and contextual cues bias human judgments of sound quality #corroborates (0.82) — Toole, Floyd E, Sound reproduction- the acoustics and…, 39
• Context-dependent compensation shapes sound perception much as with color vision #expands (0.78) — Eric J. Heller, Why You Hear What You Hear- An…, 2563
• Room acoustics and presentation method alter how listeners perceive sounds #qualifies (0.75) — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2382
• Vision dominates hearing in localization, identification, and size judgment #exemplifies (0.72) — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 873
• Physical accuracy of a sound field does not predict perceived accuracy #qualifies (0.72) — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 1902