Memory for multimodal picture-sound items: influences of congruency, identifiability and cognitive ageing

Badham, S ORCID: https://orcid.org/0000-0002-6890-102X, Atkin, C ORCID: https://orcid.org/0000-0001-5086-1173, Allen, H, Stacey, J, Henshaw, H and Roberts, KL ORCID: https://orcid.org/0000-0002-8735-2249, 2026. Memory for multimodal picture-sound items: influences of congruency, identifiability and cognitive ageing. Acta Psychologica. ISSN 0001-6918 (Forthcoming)

Text 2608024_Badham.pdf - Post-print Restricted to Repository staff only Download (909kB)

Abstract

Multimodal perception can often evoke performance that is greater than the sum of its unimodal parts in cognitive tasks. For example, choice reaction time research has found responses to multimodal stimuli that are faster than could be explained by responses to its individual unimodal components. This has led to a growth in multimodal perceptual research to explain behavioural observations, but less is known about multimodal influences on more cognitively demanding tasks such as semantic activation and memory. Across 3 well-powered pre-registered memory experiments (total n = 806), this study tested if multimodal encoding could reduce perceptual demands, evoke semantic processing, and encourage episodic recollection. In Experiment 1 the presence of congruent visual information aided later unimodal auditory retrieval (e.g., a dog picture and a barking sound), but multimodal manipulations had similar effects across young and older adults. Experiment 2 equated auditory only and visual only memory for picture-sound items by utilising blurring of the visual pictures, generating a novel shared stimulus set. Here there was no benefit to multimodal encoding, and items analysis showed that identifiability of a given stimulus was more strongly related to memory following multimodal encoding than memory following unimodal encoding. Experiment 3 utilised the remember-know paradigm and found no specific multimodal encoding effects on episodic remembering, but did find that item identifiability was more strongly linked to recollection than to familiarity. Our data highlight how multimodal processing is nuanced and supports a general need for multimodal research to explain wholistic sensory processing.

Item Type:	Journal article
Publication Title:	Acta Psychologica
Creators:	Badham, S., Atkin, C., Allen, H., Stacey, J., Henshaw, H. and Roberts, K.L.
Publisher:	Elsevier
Date:	15 April 2026
ISSN:	0001-6918
Identifiers:	Number Type 2608024 Other
Divisions:	Schools > School of Social Sciences
Record created by:	Jonathan Gallacher
Date Added:	20 Apr 2026 14:13
Last Modified:	20 Apr 2026 14:13
URI:	https://irep.ntu.ac.uk/id/eprint/55573

Actions (login required)

Edit View

Statistics

Views

Downloads