ORCID Identifier(s)

0000-0002-1504-7536

Graduation Semester and Year

2023

Language

English

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Computer Science

Department

Computer Science and Engineering

First Advisor

Fillia Makedon

Abstract

Cognition is the mental process of acquiring knowledge and understanding through thought, experience and senses. Based on Embodied Cognition theory, physical activities are an important manifestation of cognitive functions. As a result, they can be employed to both assess and train cognitive skills. In order to assess various cognitive measures, the ATEC system has been proposed. It consists of physical exercises with different variations and difficulty levels, designed to provide assessment of executive and motor functions. This thesis focuses on obtaining human activity representation from recorded videos of ATEC tasks in order to automatically assess embodied cognition performance. Representation learning is a collection of methods that allows a model to be fed with raw data and to automatically encode the representations needed for downstream task like activity recognition. Both supervised and self- supervised approaches are employed in this work, But the emphasis is on the latter which can exploit a small set of annotated data to obtain an effective representation. The performance of different self-supervised approaches are investigated for automated cognitive assessment of children performing ATEC tasks. This effort is the first step toward building a comprehensive digital phenotyping framework that can collect multi-modal data from variety of sensors such as cameras, wearables, etc., for monitoring human behaviour. Digital phenotyping is the moment by moment, quantification of the individual-level human phenotype using data from personal digital devices. Digital phenotyping will close the loop between detecting clinical phenomena and taking action by using data to trigger and deliver personalized digital treatment or prevention interventions.

Keywords

Cognitive assessment, Embodied cognition, Machine learning, Computer vision, Self-supervised learning

Disciplines

Computer Sciences | Physical Sciences and Mathematics

Comments

Degree granted by The University of Texas at Arlington

Download

Share

COinS