Graduation Semester and Year
Summer 2026
Language
English
Document Type
Dissertation
Degree Name
Doctor of Philosophy in Computer Science
Department
Computer Science and Engineering
First Advisor
William J. Beksi
Second Advisor
Junzhou Huang
Third Advisor
Manfred Huber
Fourth Advisor
Farhad Kamangar
Abstract
Modern learning systems deployed in open-world environments must make reliable decisions despite predictive uncertainty, previously unseen classes, limited annotations, and distribution shifts. This dissertation develops methods for reliable and label-efficient learning in visual perception and robot control.
First, this work studies uncertainty in object detection by representing semantic and spatial predictions probabilistically. A deep-ensemble framework aggregates detections into class-probability distributions and probabilistic bounding boxes, while a subsequent extension combines deep ensembles with Monte Carlo dropout to further investigate predictive uncertainty. Second, this dissertation addresses open-set recognition, where classes absent during training may appear at inference time. An empirical study shows that calibration methods effective under closed-set conditions remain substantially less effective in open-set settings. Building on this observation, MetaMax uses extreme value theory and Weibull calibration to model non-match activations, enabling standard classifiers to identify unknown inputs without architectural modification or auxiliary training data.
Third, this dissertation develops label-efficient learning methods that make better use of unlabeled data. SS-VAAL combines agreement-based pseudo labeling, learning-to-rank loss prediction, and adversarial latent-space modeling for active learning. This direction is extended to open-set unlabeled pools through EB-OSAL, a dual-stage energy-based framework that first filters likely unknown samples and then ranks the informativeness of likely known samples. Experiments demonstrate improved annotation efficiency across 2D image and 3D point-cloud classification.
Finally, SentryVLA extends uncertainty-aware decision-making to vision-language-action robotic control. Conformalized quantile regression guides lower-risk action selection from stochastic candidates, while Mahalanobis-distance monitoring of latent execution states identifies abnormal runtime contexts. Simulation and physical-robot experiments demonstrate improved action selection and stronger success-failure discrimination without retraining the underlying policy.
Collectively, these contributions develop learning systems that quantify uncertainty, recognize unknowns, acquire useful supervision efficiently, and use reliability signals to support safer open-world decision-making.
Keywords
Artificial intelligence, Robotics, Computer vision, Uncertainty quantification, Open-set recognition, Active learning, Probabilistic object detection, Robot learning, Vision-language-action models, Open-world learning
Disciplines
Artificial Intelligence and Robotics
License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Recommended Citation
Lyu, Zongyao, "Reliable and Label-Efficient Learning for Open-World Visual Perception and Robot Learning Under Uncertainty" (2026). Computer Science and Engineering Dissertations. 21.
https://mavmatrix.uta.edu/cse_dissertations2/21