Audio-visual integration in noise: influence of auditory and visual stimulus degradation on eyemovements and perception of the McGurk effect

Stacey, J ORCID logoORCID: https://orcid.org/0000-0003-4035-712X, Howard, C ORCID logoORCID: https://orcid.org/0000-0002-8755-1109, Mitra, S ORCID logoORCID: https://orcid.org/0000-0001-7620-4809 and Stacey, P ORCID logoORCID: https://orcid.org/0000-0002-6018-8979, 2020. Audio-visual integration in noise: influence of auditory and visual stimulus degradation on eyemovements and perception of the McGurk effect. Attention, Perception, & Psychophysics. ISSN 1943-3921

[thumbnail of 39555_a572_Stacey.pdf]
Preview
Text
39555_a572_Stacey.pdf - Post-print

Download (983kB) | Preview

Abstract

Seeing a talker's face can aid audiovisual (AV) integration when speech is presented in noise. However, few studies have simultaneously manipulated auditory and visual degradation. We aimed to establish how degrading the auditory and visual signal affected AV integration. Where people look on the face in this context is also of interest; Buchan, Paré, and Munhall (2008) found fixations on the mouth increased in the presence of auditory noise whilst Wilson, Alsius, Paré, and Munhall (2016) found mouth fixations decreased with decreasing visual resolution. In Experiment 1, participants listened to clear speech, and in Experiment 2, participants listened to vocoded speech designed to simulate the information provided by a cochlear implant. Speech was presented in three levels of auditory noise and three levels of visual blurring. Adding noise to the auditory signal increased McGurk responses, while blurring the visual signal decreased McGurk responses. Participants fixated the mouth more on trials when the McGurk effect was perceived. Adding auditory noise led to people fixating the mouth more, while visual degradation led to people fixating the mouth less. Combined, the results suggest that modality preference and where people look during AV integration of incongruent syllables varies according to the quality of information available.

Item Type: Journal article
Publication Title: Attention, Perception, & Psychophysics
Creators: Stacey, J., Howard, C., Mitra, S. and Stacey, P.
Publisher: Springer
Date: 12 June 2020
ISSN: 1943-3921
Identifiers:
Number
Type
1313545
Other
10.3758/s13414-020-02042-x
DOI
Divisions: Schools > School of Social Sciences
Record created by: Linda Sullivan
Date Added: 03 Apr 2020 15:37
Last Modified: 12 Jun 2021 03:00
URI: https://irep.ntu.ac.uk/id/eprint/39555

Actions (login required)

Edit View Edit View

Statistics

Views

Views per month over past year

Downloads

Downloads per month over past year