Soumik Mohian

ORCID Identifier(s)

0000-0003-4818-1210

Graduation Semester and Year

2023

Language

English

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Computer Science

Department

Computer Science and Engineering

First Advisor

Christoph Csallner

Abstract

Mobile application development often starts with creating low-fidelity sketches of user interfaces. Integrating these sketches into the software development process can reduce repetition, narrow the gap between user perception and final implementation, and improve app resilience. In this study, we introduce the DoodleUINet dataset, which comprises over 10K sketches of UI elements. Our Doodle2App tool converts low-fidelity sketches into a single-page, compilable Android app. At the same time, our PSDoodle provides an interactive, partial sketch-based search engine with a top-10 screen retrieval accuracy comparable to the state-of-the-art SWIRE line of work but with a 50% reduction in the average required time. Our TpD tool leverages natural language and sketching to retrieve and display the target screen in its top-10 search results with a success rate of 90%, achieving the top-10 accuracy of the state-of-the-art approach and consistently showing the target screen in the top-30. We have also developed D2S2, a drag-and-drop-based mobile screen search tool that incorporates the backend of TpD. Overall, our tools provide valuable resources for novice software engineers and facilitate the integration of low-fidelity sketches into the software development process.

Keywords

Sketching, Deep learning, Software engineering, User interface design

Disciplines

Computer Sciences | Physical Sciences and Mathematics

Comments

Degree granted by The University of Texas at Arlington

Download

Share

COinS