Parallel feature fusion for multi-modal scene recognition on dual-core MCUs

Woodward, K ORCID: https://orcid.org/0000-0003-3302-1345 and Eiman, K ORCID: https://orcid.org/0000-0002-1720-0661, 2025. Parallel feature fusion for multi-modal scene recognition on dual-core MCUs. IEEE Pervasive Computing, 24 (3), pp. 54-62. ISSN 1536-1268

Preview

Text 2415007_Woodward.pdf - Accepted version Download (973kB) | Preview

Abstract

Multi-modal sensing promises more robust environmental understanding for pervasive computing applications, but implementing sophisticated sensor fusion on resource-constrained devices remains challenging. We present a novel approach that leverages dual-core microcontrollers to enable parallel processing of visual and audio data for scene recognition. Our system concurrently executes specialized neural networks across separate cores while efficiently fusing their intermediate features, achieving 93% classification accuracy across nine different environments. This represents a 12.27% improvement over single-modality approaches while reducing latency by 48ms compared to sequential processing. The entire system operates within just 258KB of memory, demonstrating that complex multi-modal AI is achievable even on highly constrained devices. By enabling sophisticated environmental understanding without cloud connectivity, this work advances privacy-preserving edge intelligence for a wide range of future applications.

Item Type:	Journal article
Publication Title:	IEEE Pervasive Computing
Creators:	Woodward, K. and Eiman, K.
Publisher:	Institute of Electrical and Electronics Engineers
Date:	July 2025
Volume:	24
Number:	3
ISSN:	1536-1268
Identifiers:	Number Type 10.1109/MPRV.2025.3555334 DOI 2415007 Other
Rights:	© 2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Divisions:	Schools > School of Science and Technology
Record created by:	Jonathan Gallacher
Date Added:	25 Mar 2025 10:38
Last Modified:	13 Apr 2026 12:13
URI:	https://irep.ntu.ac.uk/id/eprint/53298

Actions (login required)

Edit View

Statistics

Views

Downloads