Syllable stimuli show far higher audiovisual correlation than sentences do

Evidence

Audiovisual fusion has long been considered as an automatic fusion process (e.g. Massaro 1987 ). However Exp. A. confirms that the contextual stimulus may modulate fusion as in our previous experiments (Nahorna et al. 2012 ; Ganesh et al. 2013 ; Nahorna et al. 2015 ) and extends the concept to the case of competing sources. This supports the AVSSA hypothesis, in which a first speech scene analysis process would group together the adequate audiovisual pieces of information and estimate the degree of audiovisual coherence. The effect of context type (larger McGurk effect in the “Video syllables” condition) could be due to the differences in audiovisual correlations for syllables and sentences. Indeed, correlation analysis between audio (full band envelope) and video (mouth opening area) material for syllables and sentences provides a mean correlation value of 0.59 for “Video syllables” and 0.10 for “Video sentences” (Fig. 3 ). — Dijk, Pim van, Physiology, Psychoacoustics and…, 5176

The effect of context type (larger McGurk effect in the “Video syllables” condition) could be due to the differences in audiovisual correlations for syllables and sentences. Indeed, correlation analysis between audio (full band envelope) and video (mouth opening area) material for syllables and sentences provides a mean correlation value of 0.59 for “Video syllables” and 0.10 for “Video sentences” (Fig. 3 ). Another factor could increase binding with syllables, i.e. the presence of a streaming mechanism in which the syllabic target would be associated to the syllables stream rather than to the sentences stream. — Dijk, Pim van, Physiology, Psychoacoustics and…, 5178

Categorization

• QUESTION: Why do audiovisual correlations differ so dramatically between syllables (0.59) and sentences (0.10) — is this a general acoustic-visual property or specific to this stimulus set?
• XREF: The 0.59 vs 0.10 correlation gap between speech units is a concrete number relevant to any account of why McGurk is stronger for syllables - useful alongside Mcgurk & MacDonald original findings.
• corroborated: 2 independent passages support this claim — Dijk, Pim van, Physiology, Psychoacoustics and…, 5176; Dijk, Pim van, Physiology, Psychoacoustics and…, 5178