---
_id: '61108'
abstract:
- lang: eng
text: "Greybox fuzzing is used extensively in research and practice. There
are umpteen publications that improve greybox fuzzing. However, to what extent
do these improvements affect the internal components or internals of a given fuzzer
is not yet understood as the improvements are mostly evaluated using code coverage
and bug finding capability. Such an evaluation is insufficient to understand the
effect of improvements on the fuzzer internals. Some of the literature visualizes
the outcomes of fuzzing to enhance the understanding. However, they only focus
on high-level information and no previous research on visualization has been dedicated
to understanding fuzzing internals.\r\n To close this
gap, we propose the first step towards development of a fuzzing-specific visualization
framework: a taxonomy of visualization analysis tasks that fuzzing experts desire
to help them understand the fuzzing internals. Our approach involves conducting
interviews with fuzzing experts and using qualitative data analysis to systematically
extract the task taxonomy from the interview data. We also evaluate the support
of existing fuzzing visualization tools through the lens of our taxonomy. In our
study, we have conducted 33 interviews with fuzzing practitioners and extracted
a taxonomy of 120 visualization analysis tasks. Our evaluation shows that the
existing fuzzing visualization tools only provide aids to support 10 of them."
article_number: '3718346'
author:
- first_name: Sriteja
full_name: Kummita, Sriteja
id: '72582'
last_name: Kummita
- first_name: Miao
full_name: Miao, Miao
last_name: Miao
- first_name: Eric
full_name: Bodden, Eric
id: '59256'
last_name: Bodden
orcid: 0000-0003-3470-3647
- first_name: Shiyi
full_name: Wei, Shiyi
last_name: Wei
citation:
ama: Kummita S, Miao M, Bodden E, Wei S. Visualization Task Taxonomy to Understand
the Fuzzing Internals. ACM Transactions on Software Engineering and Methodology.
Published online 2025. doi:10.1145/3718346
apa: Kummita, S., Miao, M., Bodden, E., & Wei, S. (2025). Visualization Task
Taxonomy to Understand the Fuzzing Internals. ACM Transactions on Software
Engineering and Methodology, Article 3718346. https://doi.org/10.1145/3718346
bibtex: '@article{Kummita_Miao_Bodden_Wei_2025, title={Visualization Task Taxonomy
to Understand the Fuzzing Internals}, DOI={10.1145/3718346},
number={3718346}, journal={ACM Transactions on Software Engineering and Methodology},
publisher={Association for Computing Machinery (ACM)}, author={Kummita, Sriteja
and Miao, Miao and Bodden, Eric and Wei, Shiyi}, year={2025} }'
chicago: Kummita, Sriteja, Miao Miao, Eric Bodden, and Shiyi Wei. “Visualization
Task Taxonomy to Understand the Fuzzing Internals.” ACM Transactions on Software
Engineering and Methodology, 2025. https://doi.org/10.1145/3718346.
ieee: 'S. Kummita, M. Miao, E. Bodden, and S. Wei, “Visualization Task Taxonomy
to Understand the Fuzzing Internals,” ACM Transactions on Software Engineering
and Methodology, Art. no. 3718346, 2025, doi: 10.1145/3718346.'
mla: Kummita, Sriteja, et al. “Visualization Task Taxonomy to Understand the Fuzzing
Internals.” ACM Transactions on Software Engineering and Methodology, 3718346,
Association for Computing Machinery (ACM), 2025, doi:10.1145/3718346.
short: S. Kummita, M. Miao, E. Bodden, S. Wei, ACM Transactions on Software Engineering
and Methodology (2025).
date_created: 2025-09-01T10:15:26Z
date_updated: 2025-09-01T10:16:03Z
department:
- _id: '76'
doi: 10.1145/3718346
language:
- iso: eng
publication: ACM Transactions on Software Engineering and Methodology
publication_identifier:
issn:
- 1049-331X
- 1557-7392
publication_status: published
publisher: Association for Computing Machinery (ACM)
status: public
title: Visualization Task Taxonomy to Understand the Fuzzing Internals
type: journal_article
user_id: '15249'
year: '2025'
...
---
_id: '61546'
abstract:
- lang: eng
text: Fuzzing is a powerful software testing technique renowned for its
effectiveness in identifying software vulnerabilities. Traditional fuzzing evaluations
typically focus on overall fuzzer performance across a set of target programs,
yet few benchmarks consider how fine-grained program features influence fuzzing
effectiveness. To bridge this gap, we introduce FeatureBench, a novel benchmark
designed to generate programs with configurable, fine-grained program features
to enhance fuzzing evaluations. We reviewed 25 recent grey-box fuzzing studies,
extracting 7 program features related to control-flow and data-flow that can impact
fuzzer performance. Using these features, we generated a benchmark consisting
of 153 programs controlled by 10 fine-grained configurable parameters. We evaluated
11 fuzzers using this benchmark, with each fuzzer representing either distinct
claimed improvements or serving as a widely used baseline in fuzzing evaluations.
The results indicate that fuzzer performance varies significantly based on the
program features and their strengths, highlighting the importance of incorporating
program characteristics into fuzzing evaluations.
author:
- first_name: Miao
full_name: Miao, Miao
last_name: Miao
- first_name: Sriteja
full_name: Kummita, Sriteja
id: '72582'
last_name: Kummita
- first_name: Eric
full_name: Bodden, Eric
id: '59256'
last_name: Bodden
orcid: 0000-0003-3470-3647
- first_name: Shiyi
full_name: Wei, Shiyi
last_name: Wei
citation:
ama: Miao M, Kummita S, Bodden E, Wei S. Program Feature-Based Benchmarking for
Fuzz Testing. Proceedings of the ACM on Software Engineering. 2025;2(ISSTA):527-549.
doi:10.1145/3728899
apa: Miao, M., Kummita, S., Bodden, E., & Wei, S. (2025). Program Feature-Based
Benchmarking for Fuzz Testing. Proceedings of the ACM on Software Engineering,
2(ISSTA), 527–549. https://doi.org/10.1145/3728899
bibtex: '@article{Miao_Kummita_Bodden_Wei_2025, title={Program Feature-Based Benchmarking
for Fuzz Testing}, volume={2}, DOI={10.1145/3728899},
number={ISSTA}, journal={Proceedings of the ACM on Software Engineering}, publisher={Association
for Computing Machinery (ACM)}, author={Miao, Miao and Kummita, Sriteja and Bodden,
Eric and Wei, Shiyi}, year={2025}, pages={527–549} }'
chicago: 'Miao, Miao, Sriteja Kummita, Eric Bodden, and Shiyi Wei. “Program Feature-Based
Benchmarking for Fuzz Testing.” Proceedings of the ACM on Software Engineering
2, no. ISSTA (2025): 527–49. https://doi.org/10.1145/3728899.'
ieee: 'M. Miao, S. Kummita, E. Bodden, and S. Wei, “Program Feature-Based Benchmarking
for Fuzz Testing,” Proceedings of the ACM on Software Engineering, vol.
2, no. ISSTA, pp. 527–549, 2025, doi: 10.1145/3728899.'
mla: Miao, Miao, et al. “Program Feature-Based Benchmarking for Fuzz Testing.” Proceedings
of the ACM on Software Engineering, vol. 2, no. ISSTA, Association for Computing
Machinery (ACM), 2025, pp. 527–49, doi:10.1145/3728899.
short: M. Miao, S. Kummita, E. Bodden, S. Wei, Proceedings of the ACM on Software
Engineering 2 (2025) 527–549.
date_created: 2025-10-08T08:29:39Z
date_updated: 2025-10-08T08:32:57Z
department:
- _id: '76'
- _id: '662'
doi: 10.1145/3728899
intvolume: ' 2'
issue: ISSTA
language:
- iso: eng
page: 527-549
publication: Proceedings of the ACM on Software Engineering
publication_identifier:
issn:
- 2994-970X
publication_status: published
publisher: Association for Computing Machinery (ACM)
status: public
title: Program Feature-Based Benchmarking for Fuzz Testing
type: journal_article
user_id: '15249'
volume: 2
year: '2025'
...
---
_id: '23388'
abstract:
- lang: eng
text: As one of the most popular programming languages, PYTHON has become a relevant
target language for static analysis tools. The primary data structure for performing
an inter-procedural static analysis is call-graph (CG), which links call sites
to potential call targets in a program. There exists multiple algorithms for constructing
callgraphs, tailored to specific languages. However, comparatively few implementations
target PYTHON. Moreover, there is still lack of empirical evidence as to how these
few algorithms perform in terms of precision and recall. This paper thus presents
EVAL_CG, an extensible framework for comparative analysis of Python call-graphs.
We conducted two experiments which run the CG algorithms on different Python programming
constructs and real-world applications. In both experiments, we evaluate three
CG generation frameworks namely, Code2flow, Pyan, and Wala. We record precision,
recall, and running time, and identify sources of unsoundness of each framework.
Our evaluation shows that none of the current CG construction frameworks produce
a sound CG. Moreover, the static CGs contain many spurious edges. Code2flow is
also comparatively slow. Hence, further research is needed to support CG generation
for Python programs.
author:
- first_name: Sriteja
full_name: Kummita, Sriteja
id: '72582'
last_name: Kummita
- first_name: Goran
full_name: Piskachev, Goran
id: '41936'
last_name: Piskachev
orcid: 0000-0003-4424-5838
- first_name: Johannes
full_name: Spaeth, Johannes
last_name: Spaeth
- first_name: Eric
full_name: Bodden, Eric
id: '59256'
last_name: Bodden
orcid: 0000-0003-3470-3647
citation:
ama: 'Kummita S, Piskachev G, Spaeth J, Bodden E. Qualitative and Quantitative Analysis
of Callgraph Algorithms for PYTHON. In: Proceedings of the 2021 International
Conference on Code Quality (ICCQ). ; 2021. doi:10.1109/ICCQ51190.2021.9392986'
apa: Kummita, S., Piskachev, G., Spaeth, J., & Bodden, E. (2021). Qualitative
and Quantitative Analysis of Callgraph Algorithms for PYTHON. In Proceedings
of the 2021 International Conference on Code Quality (ICCQ). Virtual. https://doi.org/10.1109/ICCQ51190.2021.9392986
bibtex: '@inproceedings{Kummita_Piskachev_Spaeth_Bodden_2021, title={Qualitative
and Quantitative Analysis of Callgraph Algorithms for PYTHON}, DOI={10.1109/ICCQ51190.2021.9392986},
booktitle={Proceedings of the 2021 International Conference on Code Quality (ICCQ)},
author={Kummita, Sriteja and Piskachev, Goran and Spaeth, Johannes and Bodden,
Eric}, year={2021} }'
chicago: Kummita, Sriteja, Goran Piskachev, Johannes Spaeth, and Eric Bodden. “Qualitative
and Quantitative Analysis of Callgraph Algorithms for PYTHON.” In Proceedings
of the 2021 International Conference on Code Quality (ICCQ), 2021. https://doi.org/10.1109/ICCQ51190.2021.9392986.
ieee: S. Kummita, G. Piskachev, J. Spaeth, and E. Bodden, “Qualitative and Quantitative
Analysis of Callgraph Algorithms for PYTHON,” in Proceedings of the 2021 International
Conference on Code Quality (ICCQ), Virtual, 2021.
mla: Kummita, Sriteja, et al. “Qualitative and Quantitative Analysis of Callgraph
Algorithms for PYTHON.” Proceedings of the 2021 International Conference on
Code Quality (ICCQ), 2021, doi:10.1109/ICCQ51190.2021.9392986.
short: 'S. Kummita, G. Piskachev, J. Spaeth, E. Bodden, in: Proceedings of the 2021
International Conference on Code Quality (ICCQ), 2021.'
conference:
location: Virtual
name: International Conference on Code Quality (ICCQ)
start_date: 2021-03-27
date_created: 2021-08-12T14:00:54Z
date_updated: 2022-01-06T06:55:52Z
doi: 10.1109/ICCQ51190.2021.9392986
keyword:
- Static Analysis
- Callgraph Analysis
- Python
- Qualitative Analysis
- Quantitative Analysis
- Empirical Evaluation
language:
- iso: eng
main_file_link:
- url: https://ieeexplore.ieee.org/document/9392986
publication: Proceedings of the 2021 International Conference on Code Quality (ICCQ)
publication_identifier:
isbn:
- 978-1-7281-8477-7
publication_status: published
status: public
title: Qualitative and Quantitative Analysis of Callgraph Algorithms for PYTHON
type: conference
user_id: '72582'
year: '2021'
...
---
_id: '23389'
abstract:
- lang: eng
text: "Background - Software companies increasingly rely on static analysis tools
to detect potential bugs and security vulnerabilities in their software products.
In the past decade, more and more commercial and open-source static analysis tools
have been developed and are maintained. Each tool comes with its own reporting
format, preventing an easy integration of multiple analysis tools in a single
interface, such as the Static Analysis Server Protocol (SASP). In 2017, a collaborative
effort in industry, including Microsoft and GrammaTech, has proposed the Static
Analysis Results Interchange Format (SARIF) to address this issue. SARIF is a
standardized format in which static analysis warnings can be encoded, to allow
the import and export of analysis reports between different tools.\r\nPurpose
- This paper explains the SARIF format through examples and presents a proof of
concept of the connector that allows the static analysis tool CogniCrypt to generate
and export its results in SARIF format.\r\nDesign/Approach - We conduct a cross-sectional
study between the SARIF format and CogniCrypt's output format before detailing
the implementation of the connector. The study aims to find the components of
interest in CogniCrypt that the SARIF export module can complete.\r\nOriginality/Value
- The integration of SARIF into CogniCrypt described in this paper can be reused
to integrate SARIF into other static analysis tools.\r\nConclusion - After detailing
the SARIF format, we present an initial implementation to integrate SARIF into
CogniCrypt. After taking advantage of all the features provided by SARIF, CogniCrypt
will be able to support SASP."
author:
- first_name: Sriteja
full_name: Kummita, Sriteja
id: '72582'
last_name: Kummita
- first_name: Goran
full_name: Piskachev, Goran
id: '41936'
last_name: Piskachev
orcid: 0000-0003-4424-5838
citation:
ama: Kummita S, Piskachev G. Integration of the Static Analysis Results Interchange
Format in CogniCrypt.; 2019.
apa: Kummita, S., & Piskachev, G. (2019). Integration of the Static Analysis
Results Interchange Format in CogniCrypt.
bibtex: '@book{Kummita_Piskachev_2019, title={Integration of the Static Analysis
Results Interchange Format in CogniCrypt}, author={Kummita, Sriteja and Piskachev,
Goran}, year={2019} }'
chicago: Kummita, Sriteja, and Goran Piskachev. Integration of the Static Analysis
Results Interchange Format in CogniCrypt, 2019.
ieee: S. Kummita and G. Piskachev, Integration of the Static Analysis Results
Interchange Format in CogniCrypt. 2019.
mla: Kummita, Sriteja, and Goran Piskachev. Integration of the Static Analysis
Results Interchange Format in CogniCrypt. 2019.
short: S. Kummita, G. Piskachev, Integration of the Static Analysis Results Interchange
Format in CogniCrypt, 2019.
date_created: 2021-08-12T14:04:46Z
date_updated: 2022-01-06T06:55:52Z
extern: '1'
keyword:
- Static Analysis
- Static Analysis Results Interchange Format
- SARIF
- Static Analysis Server Protocol
- SASP
language:
- iso: eng
main_file_link:
- url: https://arxiv.org/abs/1907.02558
status: public
title: Integration of the Static Analysis Results Interchange Format in CogniCrypt
type: report
user_id: '72582'
year: '2019'
...