Graduation Semester and Year
2019
Language
English
Document Type
Thesis
Degree Name
Doctor of Philosophy in Computer Science
Department
Computer Science and Engineering
First Advisor
Farhad Kamangar
Second Advisor
David Levine
Abstract
Verifying that a web browser rendering engine correctly renders all valid web pages is challenging due to the size of the input space (valid web pages), the difficulty of determining correct rendering for any given web page (the test oracle problem), and the degree to which normal variation in browser rendering behavior can obscure other differences (fonts, bor- ders, input controls, etc). These challenges lead to manual human involvement during the testing process. We propose a new Property-Based Testing (PBT) approach that addresses these challenges in order to enable automated web browser render testing. Our approach is composed of the following modules: a system for translating HyperText Markup Lan- guage (HTML) and Cascading Style Sheets (CSS) specification data into grammar-based generators; a grammar weighting system that controls test case generation along with mul- tiple methods for automatically adjusting those weights; and a consensus oracle of multiple rendering engines to identify failing test cases. Our approach increases the practicality of real-world testing. It has the ability to re- produce existing test cases from external sources, is able to shrink test cases to assist with root-cause analysis, and supports generic test specifications. We validated and characterized the effectiveness of our approach with a constrained markup grammar developed for this purpose. Applying our approach while testing Mozilla Firefox and Google Chrome revealed 1695 unique test cases that yield rendering differences. Testing of Mozilla’s new Servo web browser revealed multiple bugs including eleven crashes and resource leaks. We reported these issues to Mozilla developers and are working together to develop and verify solutions.
Keywords
Software, Testing, Property-based testing, Automated testing, HTML, CSS, Web browser, Differential testing, Rendering
Disciplines
Computer Sciences | Physical Sciences and Mathematics
License
This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License.
Recommended Citation
Martin, Joel David, "Using Property-Based Testing, Weighted Grammar-Based Generators, and a Consensus Oracle to Test Browser Rendering Engines and to Reproduce Minimized Versions of Existing Test Cases" (2019). Computer Science and Engineering Theses. 470.
https://mavmatrix.uta.edu/cse_theses/470
Comments
Degree granted by The University of Texas at Arlington