{"date_updated":"2023-10-16T12:51:27Z","language":[{"iso":"eng"}],"place":"Cham","page":"575–589","publication":"Parallel Problem Solving from Nature — PPSN XVII","abstract":[{"lang":"eng","text":"Hyperparameter optimization (HPO) is a key component of machine learning models for achieving peak predictive performance. While numerous methods and algorithms for HPO have been proposed over the last years, little progress has been made in illuminating and examining the actual structure of these black-box optimization problems. Exploratory landscape analysis (ELA) subsumes a set of techniques that can be used to gain knowledge about properties of unknown optimization problems. In this paper, we evaluate the performance of five different black-box optimizers on 30 HPO problems, which consist of two-, three- and five-dimensional continuous search spaces of the XGBoost learner trained on 10 different data sets. This is contrasted with the performance of the same optimizers evaluated on 360 problem instances from the black-box optimization benchmark (BBOB). We then compute ELA features on the HPO and BBOB problems and examine similarities and differences. A cluster analysis of the HPO and BBOB problems in ELA feature space allows us to identify how the HPO problems compare to the BBOB problems on a structural meta-level. We identify a subset of BBOB problems that are close to the HPO problems in ELA feature space and show that optimizer performance is comparably similar on these two sets of benchmark problems. We highlight open challenges of ELA for HPO and discuss potential directions of future research and applications."}],"title":"HPO x ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis","editor":[{"full_name":"Rudolph, Günter","last_name":"Rudolph","first_name":"Günter"},{"last_name":"Kononova","full_name":"Kononova, Anna V.","first_name":"Anna V."},{"last_name":"Aguirre","full_name":"Aguirre, Hernán","first_name":"Hernán"},{"first_name":"Pascal","full_name":"Kerschke, Pascal","last_name":"Kerschke"},{"first_name":"Gabriela","full_name":"Ochoa, Gabriela","last_name":"Ochoa"},{"first_name":"Tea","full_name":"Tušar, Tea","last_name":"Tušar"}],"author":[{"full_name":"Schneider, Lennart","last_name":"Schneider","first_name":"Lennart"},{"last_name":"Schäpermeier","full_name":"Schäpermeier, Lennart","first_name":"Lennart"},{"first_name":"Raphael Patrick","full_name":"Prager, Raphael Patrick","last_name":"Prager"},{"first_name":"Bernd","full_name":"Bischl, Bernd","last_name":"Bischl"},{"orcid":"0000-0002-9788-8282","last_name":"Trautmann","full_name":"Trautmann, Heike","id":"100740","first_name":"Heike"},{"first_name":"Pascal","full_name":"Kerschke, Pascal","last_name":"Kerschke"}],"year":"2022","user_id":"15504","publication_identifier":{"isbn":["978-3-031-14714-2"]},"doi":"10.1007/978-3-031-14714-2_40","date_created":"2023-08-04T07:15:16Z","_id":"46306","status":"public","department":[{"_id":"34"},{"_id":"819"}],"publisher":"Springer International Publishing","type":"conference","citation":{"chicago":"Schneider, Lennart, Lennart Schäpermeier, Raphael Patrick Prager, Bernd Bischl, Heike Trautmann, and Pascal Kerschke. “HPO x ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis.” In <i>Parallel Problem Solving from Nature — PPSN XVII</i>, edited by Günter Rudolph, Anna V. Kononova, Hernán Aguirre, Pascal Kerschke, Gabriela Ochoa, and Tea Tušar, 575–589. Cham: Springer International Publishing, 2022. <a href=\"https://doi.org/10.1007/978-3-031-14714-2_40\">https://doi.org/10.1007/978-3-031-14714-2_40</a>.","mla":"Schneider, Lennart, et al. “HPO x ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis.” <i>Parallel Problem Solving from Nature — PPSN XVII</i>, edited by Günter Rudolph et al., Springer International Publishing, 2022, pp. 575–589, doi:<a href=\"https://doi.org/10.1007/978-3-031-14714-2_40\">10.1007/978-3-031-14714-2_40</a>.","apa":"Schneider, L., Schäpermeier, L., Prager, R. P., Bischl, B., Trautmann, H., &#38; Kerschke, P. (2022). HPO x ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis. In G. Rudolph, A. V. Kononova, H. Aguirre, P. Kerschke, G. Ochoa, &#38; T. Tušar (Eds.), <i>Parallel Problem Solving from Nature — PPSN XVII</i> (pp. 575–589). Springer International Publishing. <a href=\"https://doi.org/10.1007/978-3-031-14714-2_40\">https://doi.org/10.1007/978-3-031-14714-2_40</a>","short":"L. Schneider, L. Schäpermeier, R.P. Prager, B. Bischl, H. Trautmann, P. Kerschke, in: G. Rudolph, A.V. Kononova, H. Aguirre, P. Kerschke, G. Ochoa, T. Tušar (Eds.), Parallel Problem Solving from Nature — PPSN XVII, Springer International Publishing, Cham, 2022, pp. 575–589.","ama":"Schneider L, Schäpermeier L, Prager RP, Bischl B, Trautmann H, Kerschke P. HPO x ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis. In: Rudolph G, Kononova AV, Aguirre H, Kerschke P, Ochoa G, Tušar T, eds. <i>Parallel Problem Solving from Nature — PPSN XVII</i>. Springer International Publishing; 2022:575–589. doi:<a href=\"https://doi.org/10.1007/978-3-031-14714-2_40\">10.1007/978-3-031-14714-2_40</a>","ieee":"L. Schneider, L. Schäpermeier, R. P. Prager, B. Bischl, H. Trautmann, and P. Kerschke, “HPO x ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis,” in <i>Parallel Problem Solving from Nature — PPSN XVII</i>, 2022, pp. 575–589, doi: <a href=\"https://doi.org/10.1007/978-3-031-14714-2_40\">10.1007/978-3-031-14714-2_40</a>.","bibtex":"@inproceedings{Schneider_Schäpermeier_Prager_Bischl_Trautmann_Kerschke_2022, place={Cham}, title={HPO x ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis}, DOI={<a href=\"https://doi.org/10.1007/978-3-031-14714-2_40\">10.1007/978-3-031-14714-2_40</a>}, booktitle={Parallel Problem Solving from Nature — PPSN XVII}, publisher={Springer International Publishing}, author={Schneider, Lennart and Schäpermeier, Lennart and Prager, Raphael Patrick and Bischl, Bernd and Trautmann, Heike and Kerschke, Pascal}, editor={Rudolph, Günter and Kononova, Anna V. and Aguirre, Hernán and Kerschke, Pascal and Ochoa, Gabriela and Tušar, Tea}, year={2022}, pages={575–589} }"}}