Explaining face-voice matching decisions: the contribution of mouth movements, stimulus effects and response biases

Lavan, N, Smith, H ORCID: https://orcid.org/0000-0003-2712-5527, Jiang, L and McGettigan, C, 2021. Explaining face-voice matching decisions: the contribution of mouth movements, stimulus effects and response biases. Attention, Perception, and Psychophysics. ISSN 1943-3921

Preview

Text 1424926_Smith.pdf - Post-print Download (712kB) | Preview

Abstract

Previous studies have shown that face-voice matching is more consistently above chance for dynamic (i.e. speaking) faces than for static faces. This suggests that dynamic information can play an important role in informing matching decisions. We initially asked whether this advantage for dynamic stimuli is due to shared information across modalities that is encoded in articulatory mouth movements. Participants completed a sequential face-voice matching task with (1) static images of faces, (2) dynamic videos of faces, (3) dynamic videos where only the mouth was visible, and (4) dynamic videos where the mouth was occluded, in a well-controlled stimulus set. Surprisingly, after accounting for random variation in the data due to design choices, accuracy for all 4 conditions was at chance. Crucially, however, exploratory analyses revealed that participants were not responding randomly, with different patterns of response biases being apparent for different conditions. Our findings suggest that face-voice identity matching may not be possible with above-chance accuracy but that analyses of response biases can shed light upon how people attempt face-voice matching. We discuss these findings with reference to the differential functional roles for faces and voices recently proposed for multimodal person perception.

Item Type:	Journal article
Publication Title:	Attention, Perception, and Psychophysics
Creators:	Lavan, N., Smith, H., Jiang, L. and McGettigan, C.
Publisher:	Springer (part of Springer Nature)
Date:	April 2021
ISSN:	1943-3921
Identifiers:	Number Type 10.3758/s13414-021-02290-5 DOI 1424926 Other
Divisions:	Schools > School of Social Sciences
Record created by:	Linda Sullivan
Date Added:	16 Mar 2021 16:41
Last Modified:	01 Apr 2022 03:00
URI:	https://irep.ntu.ac.uk/id/eprint/42522

Actions (login required)

Edit View

Statistics

Views

Downloads