Audiovisual spoken word training can promote or impede auditory-only perceptual learning: prelingually deafened adults with late-acquired cochlear implants versus normal hearing adults - Bernstein, Lynne E.; Eberhardt, Silvio P.; Auer, Edward T. (Frontiers in Psychology, 2014)
Training with audiovisual (AO) speech can promote auditory perceptual learning of vocoded acoustic speech by adults with normal hearing. Pre-/perilingually deafened adults rely on visual speech even when they also use a cochlear implant. This study investigated whether visual speech promotes auditory perceptual learning in these cochlear implant users. In Experiment 1, 28 prelingually deafened adults with late-acquired cochlear implants were assigned to learn paired associations between spoken disyllabic C(=consonant)V(=vowel)CVC nonsense words and nonsense pictures (fribbles), under AV and then under auditory-only (AO) (or counter-balanced AO then AV) training conditions. After training on each list of paired-associates (PA), testing was carried out AO. Across AV and AO training, AO PA test scores improved as did identification of consonants in untrained CVCVC stimuli. However, whenever PA training was carried out with AV stimuli, AO test scores were steeply reduced. Experiment 2 repeated the experiment with 43 normal-hearing adults. Their AO tests scores did not drop following AV PA training and even increased relative to scores following AO training. Normal-hearing participants' consonant identification scores improved also but with a pattern that contrasted with cochlear implant users’: Normal hearing adults were most accurate for medial consonants, and in contrast cochlear implant users were most accurate for initial consonants. The results are interpreted within a multisensory reverse hierarchy theory, which predicts that perceptual tasks are carried out whenever possible based on immediate high-level perception without scrutiny of lower-level features. The theory implies that, based on their bias towards visual speech, cochlear implant participants learned the PAs with greater reliance on vision to the detriment of auditory perceptual learning. Normal-hearing participants' learning took advantage of the concurrence between auditory and visual speech.