Adaptive segmentation and sequence learning of human activities from skeleton data

Adama, DA ORCID: https://orcid.org/0000-0002-2650-857X, Lotfi, A ORCID: https://orcid.org/0000-0002-5139-6565 and Ranson, R ORCID: https://orcid.org/0000-0002-4619-3982, 2020. Adaptive segmentation and sequence learning of human activities from skeleton data. Expert Systems with Applications: 113836. ISSN 0957-4174

Preview

Text 1351946_Lotfi.pdf - Post-print Download (3MB) | Preview

Abstract

Discovering underlying patterns for predicting future actions from spatio-temporal human activity information is a fundamental component of research related to the development of expert systems in human activity recognition and assistive robotics. Current research focuses on classification or learning representations of activities for various applications. However, not much attention is given to the pattern discovery of activities which have a major role in the prediction of unseen actions. This paper proposes a novel Adaptive Segmentation and Sequence Learning (ASSL) framework which aims at segmenting unlabelled observations of human activities from extracted 3D joint information. Learning from these obtained segments provides information about the underlying patterns of activity sequences needed in predicting subsequent actions. In the proposed method, the temporal accumulated motion energy of body parts in an activity is utilised in the segmentation process to obtain key actions from unlabelled activity sequences since body parts show changes in acceleration and deceleration during an activity. Based on the segments obtained, the temporal sequence of transitions across activity segments are learned by employing a Long Short-Term Memory Recurrent Neural Network. This ASSL technique has been evaluated using both an experimental human activity dataset and a public activity dataset, and achieved a better performance when compared with other techniques including an Auto-regressive Integrated Moving Average, Support Vector Regression and Gaussian Mixture Regression Models in learning to predict patterns of activity sequences.

Item Type:	Journal article
Publication Title:	Expert Systems with Applications
Creators:	Adama, D.A., Lotfi, A. and Ranson, R.
Publisher:	Elsevier
Date:	9 August 2020
ISSN:	0957-4174
Identifiers:	Number Type 10.1016/j.eswa.2020.113836 DOI S0957417420306461 Publisher Item Identifier 1351946 Other
Divisions:	Schools > School of Science and Technology
Record created by:	Linda Sullivan
Date Added:	12 Aug 2020 13:23
Last Modified:	09 Aug 2022 03:00
URI:	https://irep.ntu.ac.uk/id/eprint/40442

Actions (login required)

Edit View

Statistics

Views

Downloads