PMH6 deriving clinical outcome measures from expression and speech: results of a proof of concept study

Tools

Siena, F., Vernon, M., Watts, P., Crundall, D., Byrom, B., Mowlem, F. and Breedon, P. ORCID: 0000-0002-1006-0942, 2020. PMH6 deriving clinical outcome measures from expression and speech: results of a proof of concept study. Value in Health, 23 (Supp 1), S200-S201. ISSN 1098-3015

Full text not available from this repository.

Official URL: https://doi.org/10.1016/j.jval.2020.04.627

Abstract

OBJECTIVES: We have developed a mobile application and cloud analytics software solution to extract possible health outcome measures based on expression, voice acoustics and speech sentiment analysis from video diary footage collected using an Android app. This study aimed to assess the validity of expression detection.

METHODS: Healthy volunteers were requested to view 21 validated images from the International Affective Picture System (IAPS) database through a mobile app which simultaneously captured video footage of their face using the selfie camera. Images were selected to generate the following emotional responses: anger, disgust, sadness, contempt, fear, surprise and happiness. Expression analysis was performed using Microsoft Azure Cognitive services and the association with IAPS data was assessed for each image.

RESULTS: Forty participants (ages 21-57 years; Sex: M (20), F (19), Not stated (1)) provided informed consent and participated in this proof-of-concept study. Emotion was summarised into valence (unpleasant to pleasant) and arousal (calm to exciting) values. Both valence and arousal scores estimated by the average emotion estimated by the analysis of video footage were adequate predictors of the IAPS image scores (p < 0.001 and p=0.04 respectively), based on linear mixed-effects multi-level modelling using a chi-squared test to assess the significance of video analysis estimates of emotional response.

CONCLUSIONS: This proof-of-concept study provides early encouraging findings that facial expression derived from video footage may provide appropriate measures of expression. In combination with voice acoustical measures and speech sentiment analysis, this may lead to novel measures of health status in patients using a video diary in indications including depression, schizophrenia, autism spectrum disorder and PTSD amongst other conditions.

Item Type:

Journal article

Publication Title:

Value in Health

Creators:

Siena, F., Vernon, M., Watts, P., Crundall, D., Byrom, B., Mowlem, F. and Breedon, P.

Publisher:

Elsevier BV

Date:

1 May 2020

Volume:

Number: