Scene analysis principles apply to cross-modal audiovisual speech mixtures.

Evidence

Auditory scene analysis (ASA) was presented in Section 11.7.2. Computational auditory scene analysis (CASA) is defined by Wang and Brown (2006) as ‘the field of computational study that aims to achieve human performance in ASA by using one or two microphone recordings of the acoustic scene.’ Note that the definition does not say that the human auditory system should be imitated in the processing, only that the performance of the system should match that of a human. Human performance seems difficult to achieve, as we are very good at analysing complex acoustic scenes containing sound from multiple sources through reverberant rooms. The current CASA methods hardly achieve human performance levels in such tasks. — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 2574 (#expands, via linked claim)

Audiovisual fusion has long been considered as an automatic fusion process (e.g. Massaro 1987 ). However Exp. A. confirms that the contextual stimulus may modulate fusion as in our previous experiments (Nahorna et al. 2012 ; Ganesh et al. 2013 ; Nahorna et al. 2015 ) and extends the concept to the case of competing sources. — Dijk, Pim van, Physiology, Psychoacoustics and…, 5177 (#corroborates, via linked claim)

Categorization

• XREF: connects to: Auditory scene analysis organizes sound events into perceptual streams; Audiovisual speech fusion is not an automatic process but context-modulated. — Dijk, Pim van, Physiology, Psychoacoustics and…, 5137; Computational auditory scene analysis targets human-level performance without imitating the ear — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 2574 (+1 more)

Related

• Audiovisual speech fusion is not an automatic process but context-modulated. (similarity 0.89)
• Auditory scene analysis organizes sound events into perceptual streams (similarity 0.88)
• Computational auditory scene analysis targets human-level performance without imitating the ear (similarity 0.88)
• Audiovisual fusion is context-modulated, not automatic, and scales with audio-video coherence (similarity 0.86)
• Temporal audiovisual coherence strength determines visual weighting in fusion. (similarity 0.84)
• Audiovisual context coherence strengthens perceptual binding of speech syllables (similarity 0.84)
• Selective attention modulates fusion only for low-coherence video sentence contexts. (similarity 0.84)
• A coherence index computed before fusion decides whether audiovisual inputs bind (similarity 0.82)
• Exposure to low-coherence audiovisual contexts reduces subsequent multisensory fusion (similarity 0.82)
• Top-down instructional attention modulates audiovisual speech perception in unimpaired listeners (similarity 0.82)
• Voluntary attention modulates how audiovisual contexts affect subsequent fusion (similarity 0.81)
• Brief audiovisual context can sharply weaken the McGurk fusion effect. (similarity 0.81)
• Speech perception begins with an audiovisual scene analysis stage (similarity 0.81)
• Multisensory binding decisions depend on a prior coherence estimate between auditory and visual streams. (similarity 0.79)
• Two-stage scene analysis extends from auditory to audiovisual speech scenes. (similarity 0.79)

Broader Connections

• Computational auditory scene analysis targets human-level performance without imitating the ear #expands (0.85) — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 2574
• Audiovisual fusion is context-modulated, not automatic, and scales with audio-video coherence #corroborates (0.72) — Dijk, Pim van, Physiology, Psychoacoustics and…, 5177