Adaptive segmentation and sequence learning of human activities from skeleton data

Tools

Adama, D.A. ORCID: 0000-0002-2650-857X, Lotfi, A. ORCID: 0000-0002-5139-6565 and Ranson, R. ORCID: 0000-0002-4619-3982, 2020. Adaptive segmentation and sequence learning of human activities from skeleton data. Expert Systems with Applications: 113836. ISSN 0957-4174

Preview

Text
1351946_Lotfi.pdf - Post-print
Download (3MB) | Preview

Official URL: https://doi.org/10.1016/j.eswa.2020.113836

Abstract

Discovering underlying patterns for predicting future actions from spatio-temporal human activity information is a fundamental component of research related to the development of expert systems in human activity recognition and assistive robotics. Current research focuses on classification or learning representations of activities for various applications. However, not much attention is given to the pattern discovery of activities which have a major role in the prediction of unseen actions. This paper proposes a novel Adaptive Segmentation and Sequence Learning (ASSL) framework which aims at segmenting unlabelled observations of human activities from extracted 3D joint information. Learning from these obtained segments provides information about the underlying patterns of activity sequences needed in predicting subsequent actions. In the proposed method, the temporal accumulated motion energy of body parts in an activity is utilised in the segmentation process to obtain key actions from unlabelled activity sequences since body parts show changes in acceleration and deceleration during an activity. Based on the segments obtained, the temporal sequence of transitions across activity segments are learned by employing a Long Short-Term Memory Recurrent Neural Network. This ASSL technique has been evaluated using both an experimental human activity dataset and a public activity dataset, and achieved a better performance when compared with other techniques including an Auto-regressive Integrated Moving Average, Support Vector Regression and Gaussian Mixture Regression Models in learning to predict patterns of activity sequences.

Item Type:

Journal article

Publication Title:

Expert Systems with Applications

Creators:

Adama, D.A., Lotfi, A. and Ranson, R.

Publisher:

Elsevier

Date:

9 August 2020

ISSN:

0957-4174