TaintBench: Automatic real-world malware benchmarking of Android taint analyses

L. Luo, F. Pauck, G. Piskachev, M. Benz, I. Pashchenko, M. Mory, E. Bodden, B. Hermann, F. Massacci, Empirical Software Engineering (2021).

Journal Article | Published | English
Author
; ; ; ; ; ; ; ;
Abstract
Due to the lack of established real-world benchmark suites for static taint analyses of Android applications, evaluations of these analyses are often restricted and hard to compare. Even in evaluations that do use real-world apps, details about the ground truth in those apps are rarely documented, which makes it difficult to compare and reproduce the results. To push Android taint analysis research forward, this paper thus recommends criteria for constructing real-world benchmark suites for this specific domain, and presents TaintBench, the first real-world malware benchmark suite with documented taint flows. TaintBench benchmark apps include taint flows with complex structures, and addresses static challenges that are commonly agreed on by the community. Together with the TaintBench suite, we introduce the TaintBench framework, whose goal is to simplify real-world benchmarking of Android taint analyses. First, a usability test shows that the framework improves experts’ performance and perceived usability when documenting and inspecting taint flows. Second, experiments using TaintBench reveal new insights for the taint analysis tools Amandroid and FlowDroid: (i) They are less effective on real-world malware apps than on synthetic benchmark apps. (ii) Predefined lists of sources and sinks heavily impact the tools’ accuracy. (iii) Surprisingly, up-to-date versions of both tools are less accurate than their predecessors.
Publishing Year
Journal Title
Empirical Software Engineering
LibreCat-ID

Cite this

Luo L, Pauck F, Piskachev G, et al. TaintBench: Automatic real-world malware benchmarking of Android taint analyses. Empirical Software Engineering. Published online 2021. doi:10.1007/s10664-021-10013-5
Luo, L., Pauck, F., Piskachev, G., Benz, M., Pashchenko, I., Mory, M., Bodden, E., Hermann, B., & Massacci, F. (2021). TaintBench: Automatic real-world malware benchmarking of Android taint analyses. Empirical Software Engineering. https://doi.org/10.1007/s10664-021-10013-5
@article{Luo_Pauck_Piskachev_Benz_Pashchenko_Mory_Bodden_Hermann_Massacci_2021, title={TaintBench: Automatic real-world malware benchmarking of Android taint analyses}, DOI={10.1007/s10664-021-10013-5}, journal={Empirical Software Engineering}, author={Luo, Linghui and Pauck, Felix and Piskachev, Goran and Benz, Manuel and Pashchenko, Ivan and Mory, Martin and Bodden, Eric and Hermann, Ben and Massacci, Fabio}, year={2021} }
Luo, Linghui, Felix Pauck, Goran Piskachev, Manuel Benz, Ivan Pashchenko, Martin Mory, Eric Bodden, Ben Hermann, and Fabio Massacci. “TaintBench: Automatic Real-World Malware Benchmarking of Android Taint Analyses.” Empirical Software Engineering, 2021. https://doi.org/10.1007/s10664-021-10013-5.
L. Luo et al., “TaintBench: Automatic real-world malware benchmarking of Android taint analyses,” Empirical Software Engineering, 2021, doi: 10.1007/s10664-021-10013-5.
Luo, Linghui, et al. “TaintBench: Automatic Real-World Malware Benchmarking of Android Taint Analyses.” Empirical Software Engineering, 2021, doi:10.1007/s10664-021-10013-5.

Link(s) to Main File(s)
Access Level
Restricted Closed Access

Export

Marked Publications

Open Data LibreCat

Search this title in

Google Scholar