An Exploratory Study on Machine Learning to Combine Security Vulnerability Alerts from Static Analysis Tools
Abstract: Due to time-to-market needs and cost of manual validation techniques, software systems are often deployed with vulnerabilities that may be exploited to gain illegitimate access/control, ultimately resulting in non-negligible consequences. Static Analysis Tools (SATs) are widely used for vulnerability detection, where the source code is analyzed without executing it. However, the performance of SATs varies considerably and a high detection rate usually comes with significant false alarms. Recent studies considered combining various SATs to improve the overall detection ability, but they do not allow exploring different performance trade-offs, as basic and rigid rules are normally followed. Machine Learning (ML) algorithms have shown promising results in several complex problems, due to their ability to fit specific needs. This paper presents an exploratory study on the combination of the output of SATs through ML algorithms to improve vulnerability detection while trying to reduce false alarms. The dataset consists of SQL Injection (SQLi) and Cross-Site Scripting (XSS) vulnerabilities detected by five different SATs in a large set of WordPress plugins developed in PHP. Results show that, for the case of SQLi, a false alarm reduction is possible without compromising the vulnerabilities detected, and that using ML allows trade-offs (e.g. reduction in false alarms at the expense of a few vulnerabilities) that are not possible with existing techniques. The paper also proposes a regression-based approach for ranking source code files considering estimates of vulnerabilities computed using the output of SATs. Results show that the approach allows creating a ranking of the source code files that largely overlaps the real ranking (based on real known vulnerabilities).
Detailed Results about the Confusion Matrices - Section III.C.3
Detailed results about all Machine Learning Algorithms configuration. Both the configuration and the metrics are presented in the spreadsheet. Tab "Report" contains all the results, and tab "Details" contains the hyperparameters of each one of the configurations.
Venn Diagrams - Section III.C.3
Venn diagrams of the False Positives of the Prediction for the Machine Learning Algorithm Configuration that presented the best resutls.
SQL Injection (SQLi)
|
Cross-Site Scripting (XSS)
|
Detailed Results about the Regression - Section IV.C
List of files with the actual number of vulnerabilities, and the prediction by each one of the methods. The number of alert per Static Analysis Tool is also presented.