Graduation Semester and Year
2019
Language
English
Document Type
Dissertation
Degree Name
Doctor of Philosophy in Computer Science
Department
Computer Science and Engineering
First Advisor
Christoph Csallner
Abstract
Rigorous validation of commercial cyber-physical system (CPS) tool chains (e.g., MATLAB/Simulink) through automated testing is of utmost importance since tool-chain generated artifacts are often deployed in safety-critical embedded hardware. Although automated differential testing through random program generation and equivalence modulo input (EMI)-based mutation has been well studied for procedural compiler testing, applying these techniques for Simulink, the widely used commercial CPS development tool pose unique challenges, which we explore in this series of work for the very first time. To better understand real-world CPS modeling and to automatically generate Simulink models similar to those crafted by engineers and researchers, we present the largest study of Simulink models to date. Using insights from this corpus we have built the very first publicly known random Simulink model generator and differential testing framework, which has found previously unknown compiler bugs in Simulink. To further improve the automated compiler testing framework we have then explored novel EMI-based mutation techniques for Simulink models, which deal with CPS language features that are not found in procedural programs, including an explicit notion of time and zombie code which combines the properties of both dead and live code. Our resulting open source tools have discovered 21 unique Simulink bugs in various production versions to date confirmed by MathWorks Support proving bug finding capabilities of these tools. 16 of these bugs were unknown to MathWorks Support.
Keywords
Cyber-physical systems, Randomized differential testing, Equivalent Modulo Input, Simulink, Compiler testing, Automated testing
Disciplines
Computer Sciences | Physical Sciences and Mathematics
License
This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License.
Recommended Citation
Chowdhury, Shafiul Azam, "Automated Testing of a Commercial Cyber-Physical System Development Tool Chain" (2019). Computer Science and Engineering Dissertations. 303.
https://mavmatrix.uta.edu/cse_dissertations/303
Comments
Degree granted by The University of Texas at Arlington