Synchronized visual motion captures perceived sound direction within 30 degrees

Evidence

A well-known example where vision modifies the auditory perception is ventriloquism (Alais and Burr, 2004), where listeners perceive the sound to be coming from a direction other than the one it actually comes from, if the visual movements associated with the sound generation are synchronized with the sound. The synchronized movements then ‘capture’ the spatial perception of sound. The effect is most powerful if the separation between the sound and visual sources is less than about 30◦. However, the auditory cue may still be more salient than the visual cue in some cases. If the visual image is severely blurred, the perceived direction matches that of the sound source, and if both of them are blurred, an average direction is perceived. — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 1466

A well-known example where vision modifies the auditory perception is ventriloquism (Alais and Burr, 2004), where listeners perceive the sound to be coming from a direction other than the one it actually comes from, if the visual movements associated with the sound generation are synchronized with the sound. The synchronized movements then ‘capture’ the spatial perception of sound. The effect is most powerful if the separation between the sound and visual sources is less than about 30◦. However, the auditory cue may still be more salient than the visual cue in some cases. If the visual image is severely blurred, the perceived direction matches that of the sound source, and if both of them are blurred, an average direction is perceived. — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 1466

Categorization

• SHARE: The 30-degree capture window and the blur experiment — a precision nugget for anyone working on audiovisual VR/AR reproduction or theater sound.
• XREF: Directly relevant to binaural/augmented-reality audio applications where loudspeaker placement and visual content must align within ~30 degrees. — connects to: Angular range dominates hearing-driven motion detection over speed and duration.; Auditory motion perception is not computed from endpoint comparisons; Motion direction perception requires a much larger angular sweep than static localization (+2 more)
• CANDY: Blur the visual enough and the sound ‘wins’ the localization contest; blur both and perception averages the two — a compact, quotable ventriloquism fact.
• corroborated: 4 independent passages support this claim — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 1466

Broader Connections

• Motion direction perception requires a much larger angular sweep than static localization #complicates (0.78)
• Angular range dominates hearing-driven motion detection over speed and duration. #complicates (0.75)
• Auditory motion perception is not computed from endpoint comparisons #complicates (0.72)