Graduation Semester and Year
2020
Language
English
Document Type
Thesis
Degree Name
Master of Science in Computer Science
Department
Computer Science and Engineering
First Advisor
Shirin Nilizadeh
Abstract
There is no doubt that recruitment process plays an important role for both employers and applicants. Based on huge number of job candidates and open vacancies, recruitment process is expensive, time consuming and stressful for both applicants and companies. In today’s world so many recruitment processes are based on machine learning techniques. Therefore, it is very important to ensure security of these algorithms. Adversarial examples are proposed to examine vulnerability of machine leaning algorithms. Many research studies have been done on evaluating the resistance of artificial intelligence-based systems, in computer vision and text classification, against adversarial examples. However, to the best of our knowledge, there is no other work evaluating the robustness of NLP-based ranking algorithms that are used in recruitment process. In this study, we proposed an attack model for generating adversarial texts and evaluate its success rate on a set of real-world recruitment applications. We carried out our study into two settings: white-box and black-box. In white-box setting, we proposed a new approach for keyword extraction, and we applied our technique to change the target resume into an adversarial example. Through extensive experiments, we examined our approach for different recruitment algorithms, and we found that on average adversarial examples have significant rank improvements. In black-box setting, we assumed that the adversary has no knowledge about recruitment process and matching algorithms. We proposed a neural network architecture to determine the proper keywords to be added to the adversarial resumes. The keywords that were predicted by our proposed neural network were tested in two different settings: (1) simple setting where recruitment is a classification task for accepting and rejecting the resumes, and (2) more complex setting where recruitment algorithm is a ranking algorithm that ranks the resumes. We found that in setting (1) number of accepted resumes increased significantly after adding predicted keywords and in setting, over 95% present of resume got accepted (2) most of the resumes experienced great rank improvement after predicted keywords were applied for example over 50% of resumes them got over 150 number rank improvement. This study shows that ranking algorithms that use very popular embedding algorithms, such as TF-IDF, and USE are vulnerable to adversarial examples
Keywords
Natural language processing, Machine learning, Neural network
Disciplines
Computer Sciences | Physical Sciences and Mathematics
License
This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License.
Recommended Citation
Samadi, Anahita, "GENERATING ADVERSARIAL EXAMPLES FOR RECRUITMENT RANKING ALGORITHMS" (2020). Computer Science and Engineering Theses. 368.
https://mavmatrix.uta.edu/cse_theses/368
Comments
Degree granted by The University of Texas at Arlington