{"year":"2023","type":"conference","page":"36–40","ddc":["000"],"_id":"49111","status":"public","quality_controlled":"1","title":"Post-Processing Independent Evaluation of Sound Event Detection Systems","place":"Tampere, Finland","abstract":[{"lang":"eng","text":"Due to the high variation in the application requirements of sound event detection (SED) systems, it is not sufficient to evaluate systems only in a single operating mode. Therefore, the community recently adopted the polyphonic sound detection score (PSDS) as an evaluation metric, which is the normalized area under the PSD receiver operating characteristic (PSD-ROC). It summarizes the system performance over a range of operating modes resulting from varying the decision threshold that is used to translate the system output scores into a binary detection output. Hence, it provides a more complete picture of the overall system behavior and is less biased by specific threshold tuning. However, besides the decision threshold there is also the post-processing that can be changed to enter another operating mode. In this paper we propose the post-processing independent PSDS (piPSDS) as a generalization of the PSDS. Here, the post-processing independent PSD-ROC includes operating points from varying post-processings with varying decision thresholds. Thus, it summarizes even more operating modes of an SED system and allows for system comparison without the need of implementing a post-processing and without a bias due to different post-processings. While piPSDS can in principle combine different types of post-processing, we here, as a first step, present median filter independent PSDS (miPSDS) results for this year’s DCASE Challenge Task4a systems. Source code is publicly available in our sed_scores_eval package (https://github.com/fgnt/sed_scores_eval)."}],"file_date_updated":"2023-11-22T08:25:08Z","date_updated":"2023-11-22T08:26:14Z","project":[{"_id":"52","name":"PC2: Computing Resources Provided by the Paderborn Center for Parallel Computing"}],"has_accepted_license":"1","file":[{"file_name":"dcase2023_ebbers.pdf","relation":"main_file","date_updated":"2023-11-22T08:25:08Z","content_type":"application/pdf","success":1,"creator":"ebbers","date_created":"2023-11-22T08:25:08Z","access_level":"closed","file_id":"49112","file_size":221875}],"date_created":"2023-11-22T08:20:26Z","language":[{"iso":"eng"}],"user_id":"34851","citation":{"ama":"Ebbers J, Haeb-Umbach R, Serizel R. Post-Processing Independent Evaluation of Sound Event Detection Systems. In: <i>Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023)</i>. ; 2023:36–40.","bibtex":"@inproceedings{Ebbers_Haeb-Umbach_Serizel_2023, place={Tampere, Finland}, title={Post-Processing Independent Evaluation of Sound Event Detection Systems}, booktitle={Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023)}, author={Ebbers, Janek and Haeb-Umbach, Reinhold and Serizel, Romain}, year={2023}, pages={36–40} }","ieee":"J. Ebbers, R. Haeb-Umbach, and R. Serizel, “Post-Processing Independent Evaluation of Sound Event Detection Systems,” in <i>Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023)</i>, 2023, pp. 36–40.","chicago":"Ebbers, Janek, Reinhold Haeb-Umbach, and Romain Serizel. “Post-Processing Independent Evaluation of Sound Event Detection Systems.” In <i>Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023)</i>, 36–40. Tampere, Finland, 2023.","short":"J. Ebbers, R. Haeb-Umbach, R. Serizel, in: Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023), Tampere, Finland, 2023, pp. 36–40.","apa":"Ebbers, J., Haeb-Umbach, R., &#38; Serizel, R. (2023). Post-Processing Independent Evaluation of Sound Event Detection Systems. <i>Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023)</i>, 36–40.","mla":"Ebbers, Janek, et al. “Post-Processing Independent Evaluation of Sound Event Detection Systems.” <i>Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023)</i>, 2023, pp. 36–40."},"author":[{"first_name":"Janek","last_name":"Ebbers","full_name":"Ebbers, Janek"},{"last_name":"Haeb-Umbach","full_name":"Haeb-Umbach, Reinhold","first_name":"Reinhold"},{"last_name":"Serizel","full_name":"Serizel, Romain","first_name":"Romain"}],"publication":"Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023)","department":[{"_id":"54"}]}