DigitalExposome: a multimodal sensor fusion approach to study the impact of the environment on momentary mental wellbeing

Johnson, TW, 2023. DigitalExposome: a multimodal sensor fusion approach to study the impact of the environment on momentary mental wellbeing. PhD, Nottingham Trent University.

[thumbnail of Thomas Johnson 2023.pdf]
Preview
Text
Thomas Johnson 2023.pdf - Published version

Download (36MB) | Preview

Abstract

In recent years, the field of environmental sciences has gained considerable attention, driven by increases in global population and rapid urbanisation. The issues have been widely recognised, as has the need for solutions to address it. Previous work to explore the impact relationship has shown poor air quality is harmful not only for health, mental health, and wellbeing but also in recent years as serious as death with the first landmark case of ‘air pollution’ as a cause of death.

DigitalExposome, a novel conceptual framework is introduced to quantify the impact of environment and mental wellbeing. The investigation uses real-time air quality with the approach of making inferences based on an individual’s personal characteristics, behaviour and momentary wellbeing within urban spaces. Using a multimodal sensorfusion approach in this work with the purpose of utilising miniaturised sensing and smartphone technologies aims to acquire environmental, human on-body physiological and mental wellbeing data, specifically labelled at the point of collection. This has entailed the creation of an affordable, sensor-based environmental monitoring station incorporating Internet of Things (IoT) technologies.

To address this, a practical approach is explored of three stages to unravel and understand the impact of the environment on wellbeing. Firstly, to observe a more human-based personalised approach, the use of trajectories were studied alongside the addition of semantics to collect environmental air quality and on-body physiological data. As a result, semantic-enriched trajectories combined with episodes supports the limitation to quantifying the impact at the point of exposure. Secondly, a study involving 40 participants in the real-world is conducted in a novel multimodal sensor fusion approach involving real-time data collection using self-labelled wellbeing, air quality characteristics and on-body physiological data. The study extends previous literature by quantifying multiple sensors and self-labelled wellbeing using a more digital approach through low-cost, affordable sensors and mobile technology. The aggregated approach supported a higher accuracy level and produces a more comprehensive relationship impact between the environment, human physiology, behaviour and wellbeing.

Thirdly, this work explores data analysis used to quantify the impact between air quality factors and wellbeing. To observe variable importance, statistical approaches such as Principle Component Analysis and Multiple Variant Regression, results in Particulate Matter and Nitrogen Dioxide having considerable negative impact to human wellbeing. Various models such as Dynamic Time Warping (DTW), Deep Belief Network (DBN) and Convolutional Neural Networks (CNN) have created new opportunities for real-world inference of mental wellbeing using environmental and on-body physiological sensor data. A personalised approach using DTW is proposed as a way to observe changes in wellbeing at a personal human-interaction level which in this work demonstrates a high level of accuracy achieving an F1- Score of 0.88 using a DTW network classifying on a 5-point wellbeing scale. To leverage the concept in quantifying an individual’s exposure to the environment using technology combined with artificial intelligence (AI) detailed in this thesis gains a deeper understanding into the negative impact air quality exposures can have towards mental wellbeing.

This thesis offers the first attempt towards assessing the relationship of air quality and mental wellbeing incorporating innovative methods of digital technology and artificial intelligence for the first time. This work has the potential to shed light on how individuals breathe, feel and interact with their environment in different surroundings

Item Type: Thesis
Creators: Johnson, T.W.
Contributors:
Name
Role
NTU ID
ORCID
Kanjo, E.
Thesis supervisor
CMP3KANJOE
Kaiwartya, O.
Thesis supervisor
CMP3KAIWAO
Sanal, S.
Thesis supervisor
CMP3SANEIS
UNSPECIFIED
Date: 2023
Rights: The copyright in this work is held by the author. You may copy up to 5% of this work for private study, or personal, non-commercial research. Any re-use of the information contained within this document should be fully referenced, quoting the author, title, university, degree level and pagination. Queries or requests for any other use, or if a more substantial copy is required, should be directed to the author.
Divisions: Schools > School of Science and Technology
Record created by: Jeremy Silvester
Date Added: 05 Feb 2025 09:47
Last Modified: 05 Feb 2025 09:47
URI: https://irep.ntu.ac.uk/id/eprint/52974

Actions (login required)

Edit View Edit View

Statistics

Views

Views per month over past year

Downloads

Downloads per month over past year