Predicting Rankings of Software Verification Competitions

M. Czech, E. Hüllermeier, M.-C. Jakobs, H. Wehrheim, Predicting Rankings of Software Verification Competitions, 2017.

Download
Restricted Predicting Rankings of Soware Verification Competitions.pdf 869.98 KB
Report | English
Author
Czech, Mike; Hüllermeier, EykeLibreCat; Jakobs, Marie-Christine; Wehrheim, HeikeLibreCat
Abstract
Software verification competitions, such as the annual SV-COMP, evaluate software verification tools with respect to their effectivity and efficiency. Typically, the outcome of a competition is a (possibly category-specific) ranking of the tools. For many applications, such as building portfolio solvers, it would be desirable to have an idea of the (relative) performance of verification tools on a given verification task beforehand, i.e., prior to actually running all tools on the task.In this paper, we present a machine learning approach to predicting rankings of tools on verification tasks. The method builds upon so-called label ranking algorithms, which we complement with appropriate kernels providing a similarity measure for verification tasks. Our kernels employ a graph representation for software source code that mixes elements of control flow and program dependence graphs with abstract syntax trees. Using data sets from SV-COMP, we demonstrate our rank prediction technique to generalize well and achieve a rather high predictive accuracy. In particular, our method outperforms a recently proposed feature-based approach of Demyanova et al. (when applied to rank predictions).
Publishing Year
LibreCat-ID
72

Cite this

Czech M, Hüllermeier E, Jakobs M-C, Wehrheim H. Predicting Rankings of Software Verification Competitions.; 2017.
Czech, M., Hüllermeier, E., Jakobs, M.-C., & Wehrheim, H. (2017). Predicting Rankings of Software Verification Competitions.
@book{Czech_Hüllermeier_Jakobs_Wehrheim_2017, title={Predicting Rankings of Software Verification Competitions}, author={Czech, Mike and Hüllermeier, Eyke and Jakobs, Marie-Christine and Wehrheim, Heike}, year={2017} }
Czech, Mike, Eyke Hüllermeier, Marie-Christine Jakobs, and Heike Wehrheim. Predicting Rankings of Software Verification Competitions, 2017.
M. Czech, E. Hüllermeier, M.-C. Jakobs, and H. Wehrheim, Predicting Rankings of Software Verification Competitions. 2017.
Czech, Mike, et al. Predicting Rankings of Software Verification Competitions. 2017.
Main File(s)
File Name
Predicting Rankings of Soware Verification Competitions.pdf 869.98 KB
Access Level
Restricted Closed Access
Last Uploaded
2018-11-21T10:50:11Z


Export

Marked Publications

Open Data LibreCat

Search this title in

Google Scholar