An incremental cross-modal transfer learning method for gesture interaction

Tools

Zhong, J., Li, J., Lotfi, A. ORCID: 0000-0002-5139-6565, Liang, P. and Yang, C., 2022. An incremental cross-modal transfer learning method for gesture interaction. Robotics and Autonomous Systems, 155: 104181. ISSN 0921-8890

Preview

Text
1563595_Lotfi.pdf - Post-print
Download (2MB) | Preview

Official URL: https://doi.org/10.1016/j.robot.2022.104181

Abstract

Gesture can be used as an important way for human–robot interaction, since it is able to give accurate and intuitive instructions to the robots. Various sensors can be used to capture gestures. We apply three different sensors that can provide different modalities in recognizing human gestures. Such data also owns its own statistical properties for the purpose of transfer learning: they own the same labeled data, but both the source and the validation data-sets have their own statistical distributions. To tackle the transfer learning problem across different sensors with such kind of data-sets, we propose a weighting method to adjust the probability distributions of the data, which results in a more faster convergence result. We further apply this method in a broad learning system, which has proven to be efficient to learn with the incremental learning capability. The results show that although these three sensors measure different parts of the body using different technologies, transfer learning is able to find out the weighting correlation among the data-sets. It also suggests that using the proposed transfer learning is able to adjust the data which has different distributions which may be similar to the physical correlation between different parts of the body in the context of giving gestures.

Item Type:

Journal article

Publication Title:

Robotics and Autonomous Systems

Creators:

Zhong, J., Li, J., Lotfi, A., Liang, P. and Yang, C.

Publisher:

Elsevier

Date:

September 2022

Volume:

155