Phonemic and articulatory-feature speech representations can be encoded as multivariate EEG-regression predictors

Evidence

The phonemic (Ph) representation was computed using forced alignment (Yuan and Liberman 2008 ), given a speech file and the correspondent orthographic transcription broken into 26 phonemes in the International Phonetic Alphabet (IPA). A multivariate time-series composed of 26 indicator variables was then obtained. … The phonetic features (Fea) encoding is a linear mapping of the phonetic representation into a space of 18 features (Mesgarani et al. 2014 ), which describe specific articulatory and acoustic properties of the speech phonetic content (Chomsky and Halle 1968 ). In particular, the chosen features are related to the manner and place of articulation, to the voicing of a consonant, and to the backness of a vowel. — Dijk, Pim van, Physiology, Psychoacoustics and…, 4371

Linear regression ( http://​sourceforge.​net/​projects/​aespa ) was used to create a mapping between the EEG and five different representations of the speech stimuli: 1. The broadband amplitude envelope (Env) of the speech signal, which was calculated as where is the complex analytic signal obtained by the sum of the original speech and its Hilbert transform 2. The spectrogram (Sgram) was obtained by partitioning the speech signal into three frequency-bands logarithmically spaced between 70 and 5000 Hz according to Greenwood’s equation (70–494–1680–5000 Hz, the same used for the vocoder), and computing the amplitude envelope for each band. 3. The phonemic (Ph) representation was computed using forced alignment (Yuan and Liberman 2008 ), given a speech file and the correspondent orthographic transcription broken into 26 phonemes in the International Phonetic Alphabet (IPA). A multivariate time-series composed of 26 indicator variables was then obtained. 4. The phonetic features (Fea) encoding is a linear mapping of the phonetic representation into a space of 18 features (Mesgarani et al. 2014 ), which describe specific articulatory and acoustic properties of the speech phonetic content (Chomsky and Halle 1968 ). In particular, the chosen features are related to the manner and place of articulation, to the voicing of a consonant, and to the backness of a vowel. 5. Finally, we propose a model that combines Fea and Sgram (FS ) by applying a concatenation of the two representations. — Dijk, Pim van, Physiology, Psychoacoustics and…, 4388

Categorization

• DEFINE: Explains how phonemic and phonetic-feature representations are constructed from speech for temporal response function modeling.
• corroborated: 3 independent passages support this claim — Dijk, Pim van, Physiology, Psychoacoustics and…, 4371; Dijk, Pim van, Physiology, Psychoacoustics and…, 4388
• XREF: connects to: Phonetic models predict EEG better, but only when speech is intelligible. — Dijk, Pim van, Physiology, Psychoacoustics and…, 4371; Envelope-insensitive phonetic EEG indices may monitor cochlear-implant patients’ phonetic discrimination.; Categorical phonetic features encode EEG information the speech envelope lacks — Dijk, Pim van, Physiology, Psychoacoustics and…, 4429 (+2 more)

Broader Connections

• Categorical phonetic features encode EEG information the speech envelope lacks #corroborates (0.85) — Dijk, Pim van, Physiology, Psychoacoustics and…, 4429
• Temporal response functions predict EEG from speech representations with cross-validated correlation #expands (0.80)
• TRF regression weights reflect cortical activation at specific speech delays #expands (0.75) — Dijk, Pim van, Physiology, Psychoacoustics and…, 4422