EDCC 2016: Software Metrics and Security Vulnerabilities: Dataset and Exploratory StudyAbstract: Code with certain characteristics is more prone to have security vulnerabilities. In fact, studies show that code not following best practices is harder to verify and maintain, and consequently is more probable to have vulnerabilities left unnoticed or inadvertently introduced. In this experience report, we study whether software metrics can reflect such characteristics, thus having some correlation with the existence of vulnerabilities. The analysis is based on 2875 security patches, used to build a dataset with metrics and vulnerabilities for all the functions, classes and files of 5750 versions of five widely used projects that are exposed to attacks: Linux Kernel, Mozilla, Xen Hypervisor, httpd and glibc. We calculated software metrics from their sources and used correlation algorithm and statistical tests on these metrics in order to identify relations between them and the existing vulnerabilities. Results show that software metrics are able to discriminate vulnerable and non vulnerable functions, but it is not possible to find strong correlations between these metrics and the number of vulnerabilities existing in the analyzed functions. Finally, the results indicate that vulnerable functions are probable to have other vulnerabilities in the future. |
|
|
LADC 2016: Experimenting Machine Learning Techniques to Predict VulnerabilitiesAbstract: Software metrics can be used as a indicator of the presence of software vulnerabilities. These metrics have been used with machine learning to predict source code prone to contain vulnerabilities. Although it is not possible to find the exact location of the flaws, the models can show which components require more attention during inspections and testing. Each new technique uses his own evaluation dataset, which many times has limited size and representativeness. In this experience report, we use a large and representative dataset to evaluate several state of the art vulnerability prediction techniques. This dataset was built with information of 2186 vulnerabilities from five widely used open source projects. Results show that the dataset can be used to distinguish which are the best techniques. It is also shown that some of the techniques can predict nearly all of the vulnerabilities present in the dataset, although with very low precisions. Finally, accuracy, precision and recall are not the most effective to characterize the effectiveness of this tools. |