ORCID Identifier(s)

0000-0002-6613-4955

Graduation Semester and Year

2023

Language

English

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Computer Science

Department

Computer Science and Engineering

First Advisor

Fillia Makedon

Abstract

Cognition is the mental process of acquiring knowledge and understanding through thought, experience, and senses. Fatigue is a loss in cognitive or physical performance due to physiological factors such as insufficient sleep, long work hours, stress, and physical exertion. It adversely affects the human body and can slow reaction times, reduce attention, and limit short-term memory. Hence, there is a need to monitor a person's state to avoid extreme fatigue conditions that can result in physiological complications. However, tools to understand and assess fatigue are minimal. This thesis primarily focuses on building an experimental setup that induces cognitive fatigue (CF) and physical fatigue (PF) through multiple cognitive and physical tasks while simultaneously recording physiological data and visual cues from a person's face. First, we build a prototype sensor shirt embedded with various physiological sensors for easy use during cognitively and physically demanding tasks. Second, participants' self-reported visual analog scores (VAS) are reported after each task to confirm fatigue induction. Finally, an evaluation system is built that utilizes machine learning (ML) models to detect states of CF and PF from multi-modal sensor data, thus providing an objective measure. This effort is the first step towards building a robust cognitive assessment tool that can collect multi-modal data and be used for industrial applications to monitor a person's mental state. For instance, it enables safe human-robot cooperation (HRC) in industrial environments to avoid physical harm when a person's mental state is not good. Another example can be a personalized assistive robot for individuals with motor impairments to perform a task such as preparing lunch with real-time interventions based on the help required from the user.

Keywords

Multimodal, Physiological signals, Cognition, Machine learning

Disciplines

Computer Sciences | Physical Sciences and Mathematics

Comments

Degree granted by The University of Texas at Arlington

Download

Share

COinS