Graduation Semester and Year
Fall 2024
Language
English
Document Type
Dissertation
Degree Name
Doctor of Philosophy in Computer Science
Department
Computer Science and Engineering
First Advisor
Fillia Makedon
Second Advisor
Nicholas Gans
Third Advisor
Ming Li
Fourth Advisor
Leonidas Fegaras
Fifth Advisor
William Beksi
Abstract
People who are blind and vision impaired often require assistance in performing various tasks. With new technologies emerging in the recent years, vision impaired people either require assistance in accessing those technologies or in using those technologies to perform different tasks in real life. Previous works have focused on assisting vision impaired people in different scenarios such as navigation, accessing smartphone interfaces etc. With the recent developments in robotics, a new research has emerged where new systems can be developed for vision impaired people to interact with robots to perform various human robot interactive tasks. But with developing new and individual technologies for assisting vision impaired people in each and every task will either become a burden for them to use it or become infeasible for them to acquire those technologies. Also, many of the previous works have developed systems that might place a hardware load or computational load on the user while using them to perform tasks. There is a need for developing an unified system that can assist vision impaired people in performing different tasks by accessing the system.
This dissertation research focuses on developing an unified cross-modal interactive system that can be used by people who are vision impaired for assistance in both human navigation tasks and human robot interactive tasks in indoor environments. In terms of human navigation, this work focuses on assisting vision impaired people in avoiding collision with moving people during navigation in indoor environments. In terms of human robot interaction, this work focuses on two tasks:
• Identifying objects and their locations in indoor environment based on user requests
• Robotic collision avoidance even when vision impaired people are moving around in the environment
The user can use different modalities such as speech, and human pose hand signs to send request or data to the system, for assistance in human navigation tasks and human robot interaction tasks. The system will be implemented such that certain components can be interchangeably used for both human navigation tasks and human robot interactive tasks thus making it an integrated system.
Keywords
Human Robot Interaction, Vision Impaired Navigation, Collision Avoidance, Pose Classification, Depth Estimation, Robotics, SLAM
Disciplines
Artificial Intelligence and Robotics | Graphics and Human Computer Interfaces
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Recommended Citation
Nambiappan, Harish Ram, "A UNIFIED CROSS-MODAL INTERACTIVE SYSTEM FOR ASSISTING VISION IMPAIRED IN HUMAN NAVIGATION AND INDOOR BASED HUMAN ROBOT INTERACTION" (2024). Computer Science and Engineering Dissertations. 403.
https://mavmatrix.uta.edu/cse_dissertations/403
Included in
Artificial Intelligence and Robotics Commons, Graphics and Human Computer Interfaces Commons