Seeing to hear? Patterns of gaze to speaking faces in children with autism spectrum disorders. - Irwin, Julia R; Brancazio, Lawrence (Frontiers in Psychology, 2014)
Using eye-tracking methodology, gaze to a speaking face was compared in a group of children with autism spectrum disorders (ASD) and a group with typical development (TD). Patterns of gaze were observed under three conditions: audiovisual (AV) speech in auditory noise, visual only speech and an AV non-face, non-speech control. Children with ASD looked less to the face of the speaker and fixated less on the speakers' mouth than TD controls. No differences in gaze were reported for the non-face, non-speech control task. Since the mouth holds much of the articulatory information available on the face, these findings suggest that children with ASD may have reduced access to critical linguistic information. This reduced access to visible articulatory information could be a contributor to the communication and language problems exhibited by children with ASD.
Keywords: Audiovisual speech perception; Autism spectrum disorders; Communication development; Eyetracking; Lipreading; Speech in noise; dept.hls; dept.hms; dept.psy; peer-reviewed

The development of gaze to a speaking face. - Irwin, Julia; Brancazio, Lawrence; Volpe, Nicole (The Journal of the Acoustical Society of America, 2017)
When a speaker talks, the visible consequences of what they are saying can be seen. Listeners are influenced by this visible speech both in a noisy listening environment and even when auditory speech can easily be heard. While visible influence on heard speech has been reported to increase from early to late childhood, little is known about the mechanism that underlies this developmental trend. One possible account of developmental differences is that looking behavior to the face of a speaker changes with age. To examine this possibility, the gaze to a speaking face was examined in children from 5 to 10 yrs of age and adults. Participants viewed a speaker's face in a range of conditions that elicit looking: in a visual only (speech reading) condition, in the presence of auditory noise (speech in noise) condition, and in an audiovisual mismatch (McGurk) condition. Results indicate an increase in gaze on the face, and specifically, to the mouth of a speaker between the ages of 5 and 10 for all conditions. This change in looking behavior may help account for previous findings in the literature showing that visual influence on heard speech increases with development.
Keywords: *Facial Expression; *Fixation, Ocular; *Speech Acoustics; *Speech Perception; *Visual Perception; *Voice Quality; Acoustic Stimulation; Acoustics; Age Factors; Audiometry, Speech; Child; Child, Preschool; Cues; Early-to-late; Facial Expression; Female; Fixation, Ocular; Humans; Male; Noise; Noise/ae [Adverse Effects]; Perceptual Masking; Photic Stimulation; Physics; Speech; Speech Acoustics; Speech Perception; Speech in noise; Speech reading; Video Recording; Visible speech; Visual Perception; Voice Quality; Young Adult; age; association; auditory stimulation; child; dept.hls; dept.psy; eye fixation; facial expression; female; human; male; masking; noise; photostimulation; preschool child; speech; speech audiometry; speech perception; videorecording; vision; voice; young adult

Can children with autism spectrum disorders “Hear” a speaking face? - Irwin, Julia; Tornatore, Lauren A; Brancazio, Lawrence; Whalen, D H (Child Development, 2011)
This study used eye-tracking methodology to assess audiovisual speech perception in 26 children ranging in age from 5 to 15 years, half with autism spectrum disorders (ASD) and half with typical development. Given the characteristic reduction in gaze to the faces of others in children with ASD, it was hypothesized that they would show reduced influence of visual information on heard speech. Responses were compared on a set of auditory, visual, and audiovisual speech perception tasks. Even when fixated on the face of the speaker, children with ASD were less visually influenced than typical development controls. This indicates fundamental differences in the processing of audiovisual speech in children with ASD, which may contribute to their language and communication impairments.
Keywords: Adolescent; Child; Child Development Disorders, Pervasive; Child, Preschool; Communication; Female; Humans; Language Development Disorders; Lipreading; Male; Pattern Recognition, Visual; Perceptual Masking; Reference Values; Speech Perception; Verbal Behavior; adolescent; article; autism; child; dept.hls; dept.psy; female; human; interpersonal communication; language disability; lip reading; male; pattern recognition; perception; preschool child; psychological aspect; reference value; speech perception; verbal behavior

Development of an audiovisual speech perception app for children with autism spectrum disorders - Irwin, Julia; Preston, Jonathan; Brancazio, Lawrence; D'Angelo, Michael; Turcios, Jacqueline (Clinical Linguistics & Phonetics, 2015)
Perception of spoken language requires attention to acoustic as well as visible phonetic information. This article reviews the known differences in audiovisual speech perception in children with autism spectrum disorders (ASD) and specifies the need for interventions that address this construct. Elements of an audiovisual training program are described. This researcher-developed program delivered via an iPad app presents natural speech in the context of increasing noise, but supported with a speaking face. Children are cued to attend to visible articulatory information to assist in perception of the spoken words. Data from four children with ASD ages 8-10 are presented showing that the children improved their performance on an untrained auditory speech-in-noise task.
Keywords: Attention; Audiovisual Aids; Audiovisual app; Autism spectrum disorder; Child; Child Development Disorders, Pervasive; Comprehension; Humans; Lipreading; Male; Mobile Applications; Pattern Recognition, Visual; Phonetics; Semantics; Speech Perception; Speech perception; attention; audiovisual aid; child; comprehension; dept.cmd; dept.hls; dept.psy; human; lip reading; male; mobile application; pattern recognition; phonetics; psychology; semantics; speech perception

Electrophysiological Indices of Audiovisual Speech Perception in the Broader Autism Phenotype. - Irwin, Julia; Avery, Trey; Turcios, Jacqueline; Brancazio, Lawrence; Cook, Barbara; Landi, Nicole (Brain sciences, 2017)
When a speaker talks, the consequences of this can both be heard (audio) and seen (visual). A novel visual phonemic restoration task was used to assess behavioral discrimination and neural signatures (event-related potentials, or ERP) of audiovisual processing in typically developing children with a range of social and communicative skills assessed using the social responsiveness scale, a measure of traits associated with autism. An auditory oddball design presented two types of stimuli to the listener, a clear exemplar of an auditory consonant-vowel syllable /ba/ (the more frequently occurring standard stimulus), and a syllable in which the auditory cues for the consonant were substantially weakened, creating a stimulus which is more like /a/ (the infrequently presented deviant stimulus). All speech tokens were paired with a face producing /ba/ or a face with a pixelated mouth containing motion but no visual speech. In this paradigm, the visual /ba/ should cause the auditory /a/ to be perceived as /ba/, creating an attenuated oddball response; in contrast, a pixelated video (without articulatory information) should not have this effect. Behaviorally, participants showed visual phonemic restoration (reduced accuracy in detecting deviant /a/) in the presence of a speaking face. In addition, ERPs were observed in both an early time window (N100) and a later time window (P300) that were sensitive to speech context (/ba/ or /a/) and modulated by face context (speaking face with visible articulation or with pixelated mouth). Specifically, the oddball responses for the N100 and P300 were attenuated in the presence of a face producing /ba/ relative to a pixelated face, representing a possible neural correlate of the phonemic restoration effect. Notably, those individuals with more traits associated with autism (yet still in the non-clinical range) had smaller P300 responses overall, regardless of face context, suggesting generally reduced phonemic discrimination.
Keywords: Article; Audiovisual speech perception; Broader autism phenotype; Development; ERP; audiovisual aid; audiovisual equipment; audiovisual speech perception; autism; behavior; broader autism phenotype; child; clinical article; dept.cmd; dept.hls; dept.psy; development; electrophysiology; female; human; male; participant observation; phenotype; social adaptation; social competence; speech perception; stimulus; vision test

Electrophysiological Indices of Audiovisual Speech Perception: Beyond the McGurk Effect and Speech in Noise. - Irwin, Julia; Avery, Trey; Brancazio, Lawrence; Turcios, Jacqueline; Ryherd, Kayleigh; Landi, Nicole (Multisensory research, 2018)
Visual information on a talker's face can influence what a listener hears. Commonly used approaches to study this include mismatched audiovisual stimuli (e.g., McGurk type stimuli) or visual speech in auditory noise. In this paper we discuss potential limitations of these approaches and introduce a novel visual phonemic restoration method. This method always presents the same visual stimulus (e.g., /ba/) dubbed with a matched auditory stimulus (/ba/) or one that has weakened consonantal information and sounds more /a/-like). When this reduced auditory stimulus (or /a/) is dubbed with the visual /ba/, a visual influence will result in effectively 'restoring' the weakened auditory cues so that the stimulus is perceived as a /ba/. An oddball design in which participants are asked to detect the /a/ among a stream of more frequently occurring /ba/s while either a speaking face or face with no visual speech was used. In addition, the same paradigm was presented for a second contrast in which participants detected /pa/ among /ba/s, a contrast which should be unaltered by the presence of visual speech. Behavioral and some ERP findings reflect the expected phonemic restoration for the /ba/ vs. /a/ contrast; specifically, we observed reduced accuracy and P300 response in the presence of visual speech. Further, we report an unexpected finding of reduced accuracy and P300 response for both speech contrasts in the presence of visual speech, suggesting overall modulation of the auditory signal in the presence of visual speech. Consistent with this, we observed a mismatch negativity (MMN) effect for the /ba/ vs. /pa/ contrast only that was larger in absence of visual speech. We discuss the potential utility for this paradigm for listeners who cannot respond actively, such as infants and individuals with developmental disabilities.
Keywords: Audio-visual speech; Audio-visual stimulus; Audiovisual; Audiovisual speech perception; Auditory stimuli; Developmental disability; ERP; Electrophysiology; Enterprise resource planning; Mismatch negativity; Phonemic restoration; Potential utility; Restoration; Speech; Speech recognition; Visual information; dept.psy; peer-reviewed; phonemic restoration