Syllabic speech produces far stronger audio-visual correlation than sentences do

Evidence

This supports the AVSSA hypothesis, in which a first speech scene analysis process would group together the adequate audiovisual pieces of information and estimate the degree of audiovisual coherence. The effect of context type (larger McGurk effect in the “Video syllables” condition) could be due to the differences in audiovisual correlations for syllables and sentences. Indeed, correlation analysis between audio (full band envelope) and video (mouth opening area) material for syllables and sentences provides a mean correlation value of 0.59 for “Video syllables” and 0.10 for “Video sentences” (Fig. 3 ). Another factor could increase binding with syllables, i.e. the presence of a streaming mechanism in which the syllabic target would be associated to the syllables stream rather than to the sentences stream. — Dijk, Pim van, Physiology, Psychoacoustics and…, 5178

Categorization

• DEFINE: Introduces the AVSSA hypothesis as a first speech scene analysis stage that evaluates audiovisual coherence before fusion.
• XREF: McGurk effect and its relation to audiovisual correlation connects to classic McGurk illusion literature; 0.59 vs 0.10 correlation numbers are a concrete anchor. — connects to: Words emerge from whole speech, not from combination of words — Herbert Marcuse, One-Dimensional Man: Studies in the…, 749