Parallel feature fusion for multi-modal scene recognition on dual-core MCUs

Woodward, K ORCID: https://orcid.org/0000-0003-3302-1345 and Eiman, K ORCID: https://orcid.org/0000-0002-1720-0661, 2025. Parallel feature fusion for multi-modal scene recognition on dual-core MCUs. IEEE Pervasive Computing. ISSN 1536-1268 (Forthcoming)

Text 2415007_Woodward.pdf - Accepted version Restricted to Repository staff only Download (973kB)

Abstract

Multi-modal sensing promises more robust environmental understanding for pervasive computing applications, but implementing sophisticated sensor fusion on resource-constrained devices remains challenging. We present a novel approach that leverages dual-core microcontrollers to enable parallel processing of visual and audio data for scene recognition. Our system concurrently executes specialized neural networks across separate cores while efficiently fusing their intermediate features, achieving 93% classification accuracy across nine different environments. This represents a 12.27% improvement over single-modality approaches while reducing latency by 48ms compared to sequential processing. The entire system operates within just 258KB of memory, demonstrating that complex multi-modal AI is achievable even on highly constrained devices. By enabling sophisticated environmental understanding without cloud connectivity, this work advances privacy-preserving edge intelligence for a wide range of future applications.

Item Type:	Journal article
Publication Title:	IEEE Pervasive Computing
Creators:	Woodward, K. and Eiman, K.
Publisher:	Institute of Electrical and Electronics Engineers
Date:	24 March 2025
ISSN:	1536-1268
Identifiers:	Number Type 2415007 Other
Divisions:	Schools > School of Science and Technology
Record created by:	Jonathan Gallacher
Date Added:	25 Mar 2025 10:38
Last Modified:	25 Mar 2025 10:40
URI:	https://irep.ntu.ac.uk/id/eprint/53298

Actions (login required)

Edit View

Statistics

Views

Downloads