Graduation Semester and Year
2017
Language
English
Document Type
Thesis
Degree Name
Master of Science in Computer Science
Department
Computer Science and Engineering
First Advisor
David Levine
Abstract
Logging is the process of storing information for future reference and audit purposes. In software applications, logging plays a very critical role as a development utility and ensures code quality. It acts as an enabler for developers and support professionals by providing them capability to see application’s functionality and understand any issues with it. Data logging has a widespread use in scientific experiments and analytical systems. Major systems which heavily uses data logging are weather reporting services, digital advertisement, search engines, space exploration systems to name a few. Although, data logging increases the productivity and efficiency of a software system, the logging process itself needs to be an efficient one. A logging system should be highly reliable, should support easy scalability and must maintain high availability. Logging infrastructure should also be completely decoupled with the parent system to ensure non-blocking operation. Finally, it should be secure enough to meet the needs of businesses and government as required. In the age of big data, logging systems themselves are a huge challenge for companies, so much so that few corporations have teams dedicated to providing data services such as data storage, analytics and security. They use logging frameworks of varying capabilities and as per their need. However, most of the logging utilities tend to be partially efficient or they face critical challenges like scalability, high throughput and performance. At present, we have few logging frameworks providing analytical capabilities along with traditional functionalities of data formatting and storage. As part of this thesis work, we come up with a logging framework which seeks to solve both functional challenges and problems related to its efficiency and performance. The system demonstrated here combines best features of multiple utilities such as messaging brokers like Kafka, event publishing through SQS and data management and analytics using ELK stack. The system implementation also utilizes efficient design patterns to tackle nonfunctional challenges such as scalability, performance and throughput.
Keywords
Visual logging, Logging, Big data, Messaging queues, AWS S3 appender
Disciplines
Computer Sciences | Physical Sciences and Mathematics
License
This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License.
Recommended Citation
Nishant, Ravi, "VISUAL LOGGING FRAMEWORK USING ELK STACK" (2017). Computer Science and Engineering Theses. 501.
https://mavmatrix.uta.edu/cse_theses/501
Comments
Degree granted by The University of Texas at Arlington