--- _id: '48777' abstract: - lang: eng text: AbstractExplainable artificial intelligence has mainly focused on static learning scenarios so far. We are interested in dynamic scenarios where data is sampled progressively, and learning is done in an incremental rather than a batch mode. We seek efficient incremental algorithms for computing feature importance (FI). Permutation feature importance (PFI) is a well-established model-agnostic measure to obtain global FI based on feature marginalization of absent features. We propose an efficient, model-agnostic algorithm called iPFI to estimate this measure incrementally and under dynamic modeling conditions including concept drift. We prove theoretical guarantees on the approximation quality in terms of expectation and variance. To validate our theoretical findings and the efficacy of our approaches in incremental scenarios dealing with streaming data rather than traditional batch settings, we conduct multiple experimental studies on benchmark data with and without concept drift. author: - first_name: Fabian full_name: Fumagalli, Fabian last_name: Fumagalli - first_name: Maximilian full_name: Muschalik, Maximilian last_name: Muschalik - first_name: Eyke full_name: Hüllermeier, Eyke last_name: Hüllermeier - first_name: Barbara full_name: Hammer, Barbara last_name: Hammer citation: ama: 'Fumagalli F, Muschalik M, Hüllermeier E, Hammer B. Incremental permutation feature importance (iPFI): towards online explanations on data streams. Machine Learning. Published online 2023. doi:10.1007/s10994-023-06385-y' apa: 'Fumagalli, F., Muschalik, M., Hüllermeier, E., & Hammer, B. (2023). Incremental permutation feature importance (iPFI): towards online explanations on data streams. Machine Learning. https://doi.org/10.1007/s10994-023-06385-y' bibtex: '@article{Fumagalli_Muschalik_Hüllermeier_Hammer_2023, title={Incremental permutation feature importance (iPFI): towards online explanations on data streams}, DOI={10.1007/s10994-023-06385-y}, journal={Machine Learning}, publisher={Springer Science and Business Media LLC}, author={Fumagalli, Fabian and Muschalik, Maximilian and Hüllermeier, Eyke and Hammer, Barbara}, year={2023} }' chicago: 'Fumagalli, Fabian, Maximilian Muschalik, Eyke Hüllermeier, and Barbara Hammer. “Incremental Permutation Feature Importance (IPFI): Towards Online Explanations on Data Streams.” Machine Learning, 2023. https://doi.org/10.1007/s10994-023-06385-y.' ieee: 'F. Fumagalli, M. Muschalik, E. Hüllermeier, and B. Hammer, “Incremental permutation feature importance (iPFI): towards online explanations on data streams,” Machine Learning, 2023, doi: 10.1007/s10994-023-06385-y.' mla: 'Fumagalli, Fabian, et al. “Incremental Permutation Feature Importance (IPFI): Towards Online Explanations on Data Streams.” Machine Learning, Springer Science and Business Media LLC, 2023, doi:10.1007/s10994-023-06385-y.' short: F. Fumagalli, M. Muschalik, E. Hüllermeier, B. Hammer, Machine Learning (2023). date_created: 2023-11-10T14:15:36Z date_updated: 2023-11-10T14:24:27Z department: - _id: '424' - _id: '660' doi: 10.1007/s10994-023-06385-y keyword: - Artificial Intelligence - Software language: - iso: eng publication: Machine Learning publication_identifier: issn: - 0885-6125 - 1573-0565 publication_status: published publisher: Springer Science and Business Media LLC status: public title: 'Incremental permutation feature importance (iPFI): towards online explanations on data streams' type: journal_article user_id: '55908' year: '2023' ... --- _id: '50262' abstract: - lang: eng text: AbstractExplainable artificial intelligence has mainly focused on static learning scenarios so far. We are interested in dynamic scenarios where data is sampled progressively, and learning is done in an incremental rather than a batch mode. We seek efficient incremental algorithms for computing feature importance (FI). Permutation feature importance (PFI) is a well-established model-agnostic measure to obtain global FI based on feature marginalization of absent features. We propose an efficient, model-agnostic algorithm called iPFI to estimate this measure incrementally and under dynamic modeling conditions including concept drift. We prove theoretical guarantees on the approximation quality in terms of expectation and variance. To validate our theoretical findings and the efficacy of our approaches in incremental scenarios dealing with streaming data rather than traditional batch settings, we conduct multiple experimental studies on benchmark data with and without concept drift. author: - first_name: Fabian full_name: Fumagalli, Fabian last_name: Fumagalli - first_name: Maximilian full_name: Muschalik, Maximilian last_name: Muschalik - first_name: Eyke full_name: Hüllermeier, Eyke last_name: Hüllermeier - first_name: Barbara full_name: Hammer, Barbara last_name: Hammer citation: ama: 'Fumagalli F, Muschalik M, Hüllermeier E, Hammer B. Incremental permutation feature importance (iPFI): towards online explanations on data streams. Machine Learning. 2023;112(12):4863-4903. doi:10.1007/s10994-023-06385-y' apa: 'Fumagalli, F., Muschalik, M., Hüllermeier, E., & Hammer, B. (2023). Incremental permutation feature importance (iPFI): towards online explanations on data streams. Machine Learning, 112(12), 4863–4903. https://doi.org/10.1007/s10994-023-06385-y' bibtex: '@article{Fumagalli_Muschalik_Hüllermeier_Hammer_2023, title={Incremental permutation feature importance (iPFI): towards online explanations on data streams}, volume={112}, DOI={10.1007/s10994-023-06385-y}, number={12}, journal={Machine Learning}, publisher={Springer Science and Business Media LLC}, author={Fumagalli, Fabian and Muschalik, Maximilian and Hüllermeier, Eyke and Hammer, Barbara}, year={2023}, pages={4863–4903} }' chicago: 'Fumagalli, Fabian, Maximilian Muschalik, Eyke Hüllermeier, and Barbara Hammer. “Incremental Permutation Feature Importance (IPFI): Towards Online Explanations on Data Streams.” Machine Learning 112, no. 12 (2023): 4863–4903. https://doi.org/10.1007/s10994-023-06385-y.' ieee: 'F. Fumagalli, M. Muschalik, E. Hüllermeier, and B. Hammer, “Incremental permutation feature importance (iPFI): towards online explanations on data streams,” Machine Learning, vol. 112, no. 12, pp. 4863–4903, 2023, doi: 10.1007/s10994-023-06385-y.' mla: 'Fumagalli, Fabian, et al. “Incremental Permutation Feature Importance (IPFI): Towards Online Explanations on Data Streams.” Machine Learning, vol. 112, no. 12, Springer Science and Business Media LLC, 2023, pp. 4863–903, doi:10.1007/s10994-023-06385-y.' short: F. Fumagalli, M. Muschalik, E. Hüllermeier, B. Hammer, Machine Learning 112 (2023) 4863–4903. date_created: 2024-01-05T21:52:28Z date_updated: 2025-01-16T16:20:12Z department: - _id: '660' doi: 10.1007/s10994-023-06385-y intvolume: ' 112' issue: '12' keyword: - Artificial Intelligence - Software language: - iso: eng page: 4863-4903 project: - _id: '126' name: 'TRR 318 - C3: TRR 318 - Subproject C3' - _id: '117' name: 'TRR 318 - C: TRR 318 - Project Area C' - _id: '109' grant_number: '438445824' name: 'TRR 318: TRR 318 - Erklärbarkeit konstruieren' publication: Machine Learning publication_identifier: issn: - 0885-6125 - 1573-0565 publication_status: published publisher: Springer Science and Business Media LLC status: public title: 'Incremental permutation feature importance (iPFI): towards online explanations on data streams' type: journal_article user_id: '93420' volume: 112 year: '2023' ... --- _id: '25035' abstract: - lang: eng text: 'AbstractThe efficiency of state-of-the-art algorithms for the dueling bandits problem is essentially due to a clever exploitation of (stochastic) transitivity properties of pairwise comparisons: If one arm is likely to beat a second one, which in turn is likely to beat a third one, then the first is also likely to beat the third one. By now, however, there is no way to test the validity of corresponding assumptions, although this would be a key prerequisite to guarantee the meaningfulness of the results produced by an algorithm. In this paper, we investigate the problem of testing different forms of stochastic transitivity in an online manner. We derive lower bounds on the expected sample complexity of any sequential hypothesis testing algorithm for various forms of stochastic transitivity, thereby providing additional motivation to focus on weak stochastic transitivity. To this end, we introduce an algorithmic framework for the dueling bandits problem, in which the statistical validity of weak stochastic transitivity can be tested, either actively or passively, based on a multiple binomial hypothesis test. Moreover, by exploiting a connection between weak stochastic transitivity and graph theory, we suggest an enhancement to further improve the efficiency of the testing algorithm. In the active setting, both variants achieve an expected sample complexity that is optimal up to a logarithmic factor.' author: - first_name: Björn full_name: Haddenhorst, Björn last_name: Haddenhorst - first_name: Viktor full_name: Bengs, Viktor last_name: Bengs - first_name: Eyke full_name: Hüllermeier, Eyke last_name: Hüllermeier citation: ama: Haddenhorst B, Bengs V, Hüllermeier E. On testing transitivity in online preference learning. Machine Learning. Published online 2021:2063-2084. doi:10.1007/s10994-021-06026-2 apa: Haddenhorst, B., Bengs, V., & Hüllermeier, E. (2021). On testing transitivity in online preference learning. Machine Learning, 2063–2084. https://doi.org/10.1007/s10994-021-06026-2 bibtex: '@article{Haddenhorst_Bengs_Hüllermeier_2021, title={On testing transitivity in online preference learning}, DOI={10.1007/s10994-021-06026-2}, journal={Machine Learning}, author={Haddenhorst, Björn and Bengs, Viktor and Hüllermeier, Eyke}, year={2021}, pages={2063–2084} }' chicago: Haddenhorst, Björn, Viktor Bengs, and Eyke Hüllermeier. “On Testing Transitivity in Online Preference Learning.” Machine Learning, 2021, 2063–84. https://doi.org/10.1007/s10994-021-06026-2. ieee: 'B. Haddenhorst, V. Bengs, and E. Hüllermeier, “On testing transitivity in online preference learning,” Machine Learning, pp. 2063–2084, 2021, doi: 10.1007/s10994-021-06026-2.' mla: Haddenhorst, Björn, et al. “On Testing Transitivity in Online Preference Learning.” Machine Learning, 2021, pp. 2063–84, doi:10.1007/s10994-021-06026-2. short: B. Haddenhorst, V. Bengs, E. Hüllermeier, Machine Learning (2021) 2063–2084. date_created: 2021-09-24T11:07:54Z date_updated: 2022-01-06T06:56:44Z doi: 10.1007/s10994-021-06026-2 language: - iso: eng page: 2063-2084 publication: Machine Learning publication_identifier: issn: - 0885-6125 - 1573-0565 publication_status: published status: public title: On testing transitivity in online preference learning type: journal_article user_id: '38261' year: '2021' ... --- _id: '3510' abstract: - lang: eng text: Automated machine learning (AutoML) seeks to automatically select, compose, and parametrize machine learning algorithms, so as to achieve optimal performance on a given task (dataset). Although current approaches to AutoML have already produced impressive results, the field is still far from mature, and new techniques are still being developed. In this paper, we present ML-Plan, a new approach to AutoML based on hierarchical planning. To highlight the potential of this approach, we compare ML-Plan to the state-of-the-art frameworks Auto-WEKA, auto-sklearn, and TPOT. In an extensive series of experiments, we show that ML-Plan is highly competitive and often outperforms existing approaches. article_type: original author: - first_name: Felix full_name: Mohr, Felix last_name: Mohr - first_name: Marcel Dominik full_name: Wever, Marcel Dominik id: '33176' last_name: Wever orcid: ' https://orcid.org/0000-0001-9782-6818' - first_name: Eyke full_name: Hüllermeier, Eyke id: '48129' last_name: Hüllermeier citation: ama: 'Mohr F, Wever MD, Hüllermeier E. ML-Plan: Automated Machine Learning via Hierarchical Planning. Machine Learning. Published online 2018:1495-1515. doi:10.1007/s10994-018-5735-z' apa: 'Mohr, F., Wever, M. D., & Hüllermeier, E. (2018). ML-Plan: Automated Machine Learning via Hierarchical Planning. Machine Learning, 1495–1515. https://doi.org/10.1007/s10994-018-5735-z' bibtex: '@article{Mohr_Wever_Hüllermeier_2018, title={ML-Plan: Automated Machine Learning via Hierarchical Planning}, DOI={10.1007/s10994-018-5735-z}, journal={Machine Learning}, publisher={Springer}, author={Mohr, Felix and Wever, Marcel Dominik and Hüllermeier, Eyke}, year={2018}, pages={1495–1515} }' chicago: 'Mohr, Felix, Marcel Dominik Wever, and Eyke Hüllermeier. “ML-Plan: Automated Machine Learning via Hierarchical Planning.” Machine Learning, 2018, 1495–1515. https://doi.org/10.1007/s10994-018-5735-z.' ieee: 'F. Mohr, M. D. Wever, and E. Hüllermeier, “ML-Plan: Automated Machine Learning via Hierarchical Planning,” Machine Learning, pp. 1495–1515, 2018, doi: 10.1007/s10994-018-5735-z.' mla: 'Mohr, Felix, et al. “ML-Plan: Automated Machine Learning via Hierarchical Planning.” Machine Learning, Springer, 2018, pp. 1495–515, doi:10.1007/s10994-018-5735-z.' short: F. Mohr, M.D. Wever, E. Hüllermeier, Machine Learning (2018) 1495–1515. conference: end_date: 2018-09-14 location: Dublin, Ireland name: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases start_date: 2018-09-10 date_created: 2018-07-08T14:06:14Z date_updated: 2022-01-06T06:59:21Z ddc: - '000' department: - _id: '355' - _id: '34' - _id: '7' - _id: '26' doi: 10.1007/s10994-018-5735-z file: - access_level: closed content_type: application/pdf creator: ups date_created: 2018-11-02T15:32:16Z date_updated: 2018-11-02T15:32:16Z file_id: '5306' file_name: ML-PlanAutomatedMachineLearnin.pdf file_size: 1070937 relation: main_file success: 1 file_date_updated: 2018-11-02T15:32:16Z has_accepted_license: '1' keyword: - AutoML - Hierarchical Planning - HTN planning - ML-Plan language: - iso: eng main_file_link: - open_access: '1' url: https://rdcu.be/3Nc2 oa: '1' page: 1495-1515 project: - _id: '1' name: SFB 901 - _id: '3' name: SFB 901 - Project Area B - _id: '10' name: SFB 901 - Subproject B2 - _id: '52' name: Computing Resources Provided by the Paderborn Center for Parallel Computing publication: Machine Learning publication_identifier: eissn: - 1573-0565 issn: - 0885-6125 publication_status: epub_ahead publisher: Springer status: public title: 'ML-Plan: Automated Machine Learning via Hierarchical Planning' type: journal_article user_id: '5786' year: '2018' ...