Graduation Semester and Year
Fall 2024
Language
English
Document Type
Dissertation
Degree Name
Doctor of Philosophy in Computer Science
Department
Computer Science and Engineering
First Advisor
Fillia Makedon
Second Advisor
Nicholas Gans
Third Advisor
Vassilis Athitsos
Fourth Advisor
Junzhou Huang
Fifth Advisor
Dajiang Zhu
Abstract
Traversability in autonomous robotic navigation refers to the ability of an autonomous agent to safely navigate over a given terrain. It plays a critical role in enabling robots to navigate over unseen or unknown terrains. Traversability segmentation aims to find an arbitrary-shaped mask covering the traversable regions (termed free-space). Current research for traversability segmentation can be divided into two scenarios: outdoor and indoor environments. Compared to the outdoor environments mainly consists of paved roads, indoor environments present unique challenges for traversability segmentation, because of diverse lighting conditions, glass doors, various floor colors and textures, arbitrary shaped appliances and furniture, presence of humans, and moving objects. Recent advances in deep learning have greatly improved the performance of semantic segmentation tasks. However, the large-scaled deep learning models are also vulnerable to adversarial attacks. Adversarial attack aims to generate negligible perturbations on the input image to fool the model to produce wrong predictions and this could greatly impact the security of deep learning model applications. Therefore, we propose an additional loss function on the hidden layer to enhance the adversarial robustness to those negligible perturbations. Training a large-scaled deep learning model typically requires a large amount of fully-annotated data, but acquring high-quality annotated data is generally labor-intensive. However, directly applying existing few-shot learning methods on the traversability segmentation task does not yield ideal masks. This poses another challenge in developing a reliable traversability segmentation model that is able to generalize to new unseen cases. Our analysis find that the pretrained segmentation model is prone to overfit/bias towards the given training examples with similar textures but fails to generalize to new cases when there is a large domain gap of free-spaces between the training examples and the query. To enhance the model's generalization ability, some few-shot learning segmentation methods suggest to learn class-agnostic high-level representations for the target class. They emphasize on identifying the most similar features from the query to match training samples, but fails to mine useful information from base classes. We hypothesis that leveraging knowledge from base classes can help better find the target class's segment. Specifically, our proposed method first extract prototypes for both target and base classes from the training samples, and then infers the target class by discriminating the difference between the target and base classes. In this dissertation, we focus on studying another important issue that pure vision-based models have the difficulty in distinguishing objects with similar textures and colors. For example, a whiteboard standing on white ceramic tiles often confuses the segmentation model to find the correct free-space. Therefore, we propose to incorporate depth information to help find free-spaces accurately. We are going to propose a multi-modality framework that can cross-attend between RGB images and their corresponding depth data. Thus, our framework can better exclude false positives derived from the vision model by the depth information.
Keywords
Robotic navigation, computer vision, segmentation
Disciplines
Computer and Systems Architecture
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Recommended Citation
An, Qiyuan, "A FEW-SHOT MULTI-MODALITY TRAVERSABILITY SEGMENTATION FRAMEWORK FOR INDOOR ROBOTIC NAVIGATION" (2024). Computer Science and Engineering Dissertations. 404.
https://mavmatrix.uta.edu/cse_dissertations/404