An incremental cross-modal transfer learning method for gesture interaction

Zhong, J, Li, J, Lotfi, A ORCID: https://orcid.org/0000-0002-5139-6565, Liang, P and Yang, C, 2022. An incremental cross-modal transfer learning method for gesture interaction. Robotics and Autonomous Systems, 155: 104181. ISSN 0921-8890

Preview

Text 1563595_Lotfi.pdf - Post-print Download (2MB) | Preview

Abstract

Gesture can be used as an important way for human–robot interaction, since it is able to give accurate and intuitive instructions to the robots. Various sensors can be used to capture gestures. We apply three different sensors that can provide different modalities in recognizing human gestures. Such data also owns its own statistical properties for the purpose of transfer learning: they own the same labeled data, but both the source and the validation data-sets have their own statistical distributions. To tackle the transfer learning problem across different sensors with such kind of data-sets, we propose a weighting method to adjust the probability distributions of the data, which results in a more faster convergence result. We further apply this method in a broad learning system, which has proven to be efficient to learn with the incremental learning capability. The results show that although these three sensors measure different parts of the body using different technologies, transfer learning is able to find out the weighting correlation among the data-sets. It also suggests that using the proposed transfer learning is able to adjust the data which has different distributions which may be similar to the physical correlation between different parts of the body in the context of giving gestures.

Item Type:	Journal article
Publication Title:	Robotics and Autonomous Systems
Creators:	Zhong, J., Li, J., Lotfi, A., Liang, P. and Yang, C.
Publisher:	Elsevier
Date:	September 2022
Volume:	155
ISSN:	0921-8890
Identifiers:	Number Type 10.1016/j.robot.2022.104181 DOI 1563595 Other
Divisions:	Schools > School of Science and Technology
Record created by:	Jonathan Gallacher
Date Added:	11 Jul 2022 08:16
Last Modified:	24 Jun 2023 03:00
URI:	https://irep.ntu.ac.uk/id/eprint/46573

Actions (login required)

Edit View

Statistics

Views

Downloads