--- _id: '47522' abstract: - lang: eng text: Artificial benchmark functions are commonly used in optimization research because of their ability to rapidly evaluate potential solutions, making them a preferred substitute for real-world problems. However, these benchmark functions have faced criticism for their limited resemblance to real-world problems. In response, recent research has focused on automatically generating new benchmark functions for areas where established test suites are inadequate. These approaches have limitations, such as the difficulty of generating new benchmark functions that exhibit exploratory landscape analysis (ELA) features beyond those of existing benchmarks.The objective of this work is to develop a method for generating benchmark functions for single-objective continuous optimization with user-specified structural properties. Specifically, we aim to demonstrate a proof of concept for a method that uses an ELA feature vector to specify these properties in advance. To achieve this, we begin by generating a random sample of decision space variables and objective values. We then adjust the objective values using CMA-ES until the corresponding features of our new problem match the predefined ELA features within a specified threshold. By iteratively transforming the landscape in this way, we ensure that the resulting function exhibits the desired properties. To create the final function, we use the resulting point cloud as training data for a simple neural network that produces a function exhibiting the target ELA features. We demonstrate the effectiveness of this approach by replicating the existing functions of the well-known BBOB suite and creating new functions with ELA feature values that are not present in BBOB. author: - first_name: Raphael Patrick full_name: Prager, Raphael Patrick last_name: Prager - first_name: Konstantin full_name: Dietrich, Konstantin last_name: Dietrich - first_name: Lennart full_name: Schneider, Lennart last_name: Schneider - first_name: Lennart full_name: Schäpermeier, Lennart last_name: Schäpermeier - first_name: Bernd full_name: Bischl, Bernd last_name: Bischl - first_name: Pascal full_name: Kerschke, Pascal last_name: Kerschke - first_name: Heike full_name: Trautmann, Heike id: '100740' last_name: Trautmann orcid: 0000-0002-9788-8282 - first_name: Olaf full_name: Mersmann, Olaf last_name: Mersmann citation: ama: 'Prager RP, Dietrich K, Schneider L, et al. Neural Networks as Black-Box Benchmark Functions Optimized for Exploratory Landscape Features. In: Proceedings of the 17th ACM/SIGEVO Conference on Foundations of Genetic Algorithms. FOGA ’23. Association for Computing Machinery; 2023:129–139. doi:10.1145/3594805.3607136' apa: Prager, R. P., Dietrich, K., Schneider, L., Schäpermeier, L., Bischl, B., Kerschke, P., Trautmann, H., & Mersmann, O. (2023). Neural Networks as Black-Box Benchmark Functions Optimized for Exploratory Landscape Features. Proceedings of the 17th ACM/SIGEVO Conference on Foundations of Genetic Algorithms, 129–139. https://doi.org/10.1145/3594805.3607136 bibtex: '@inproceedings{Prager_Dietrich_Schneider_Schäpermeier_Bischl_Kerschke_Trautmann_Mersmann_2023, place={New York, NY, USA}, series={FOGA ’23}, title={Neural Networks as Black-Box Benchmark Functions Optimized for Exploratory Landscape Features}, DOI={10.1145/3594805.3607136}, booktitle={Proceedings of the 17th ACM/SIGEVO Conference on Foundations of Genetic Algorithms}, publisher={Association for Computing Machinery}, author={Prager, Raphael Patrick and Dietrich, Konstantin and Schneider, Lennart and Schäpermeier, Lennart and Bischl, Bernd and Kerschke, Pascal and Trautmann, Heike and Mersmann, Olaf}, year={2023}, pages={129–139}, collection={FOGA ’23} }' chicago: 'Prager, Raphael Patrick, Konstantin Dietrich, Lennart Schneider, Lennart Schäpermeier, Bernd Bischl, Pascal Kerschke, Heike Trautmann, and Olaf Mersmann. “Neural Networks as Black-Box Benchmark Functions Optimized for Exploratory Landscape Features.” In Proceedings of the 17th ACM/SIGEVO Conference on Foundations of Genetic Algorithms, 129–139. FOGA ’23. New York, NY, USA: Association for Computing Machinery, 2023. https://doi.org/10.1145/3594805.3607136.' ieee: 'R. P. Prager et al., “Neural Networks as Black-Box Benchmark Functions Optimized for Exploratory Landscape Features,” in Proceedings of the 17th ACM/SIGEVO Conference on Foundations of Genetic Algorithms, 2023, pp. 129–139, doi: 10.1145/3594805.3607136.' mla: Prager, Raphael Patrick, et al. “Neural Networks as Black-Box Benchmark Functions Optimized for Exploratory Landscape Features.” Proceedings of the 17th ACM/SIGEVO Conference on Foundations of Genetic Algorithms, Association for Computing Machinery, 2023, pp. 129–139, doi:10.1145/3594805.3607136. short: 'R.P. Prager, K. Dietrich, L. Schneider, L. Schäpermeier, B. Bischl, P. Kerschke, H. Trautmann, O. Mersmann, in: Proceedings of the 17th ACM/SIGEVO Conference on Foundations of Genetic Algorithms, Association for Computing Machinery, New York, NY, USA, 2023, pp. 129–139.' date_created: 2023-09-27T15:43:17Z date_updated: 2023-10-16T12:33:02Z department: - _id: '34' - _id: '819' doi: 10.1145/3594805.3607136 keyword: - Benchmarking - Instance Generator - Black-Box Continuous Optimization - Exploratory Landscape Analysis - Neural Networks language: - iso: eng page: 129–139 place: New York, NY, USA publication: Proceedings of the 17th ACM/SIGEVO Conference on Foundations of Genetic Algorithms publication_identifier: isbn: - '9798400702020' publisher: Association for Computing Machinery series_title: FOGA ’23 status: public title: Neural Networks as Black-Box Benchmark Functions Optimized for Exploratory Landscape Features type: conference user_id: '15504' year: '2023' ... --- _id: '48882' abstract: - lang: eng text: In multimodal multi-objective optimization (MMMOO), the focus is not solely on convergence in objective space, but rather also on explicitly ensuring diversity in decision space. We illustrate why commonly used diversity measures are not entirely appropriate for this task and propose a sophisticated basin-based evaluation (BBE) method. Also, BBE variants are developed, capturing the anytime behavior of algorithms. The set of BBE measures is tested by means of an algorithm configuration study. We show that these new measures also transfer properties of the well-established hypervolume (HV) indicator to the domain of MMMOO, thus also accounting for objective space convergence. Moreover, we advance MMMOO research by providing insights into the multimodal performance of the considered algorithms. Specifically, algorithms exploiting local structures are shown to outperform classical evolutionary multi-objective optimizers regarding the BBE variants and respective trade-off with HV. author: - first_name: Jonathan full_name: Heins, Jonathan last_name: Heins - first_name: Jeroen full_name: Rook, Jeroen last_name: Rook - first_name: Lennart full_name: Schäpermeier, Lennart last_name: Schäpermeier - first_name: Pascal full_name: Kerschke, Pascal last_name: Kerschke - first_name: Jakob full_name: Bossek, Jakob id: '102979' last_name: Bossek orcid: 0000-0002-4121-4668 - first_name: Heike full_name: Trautmann, Heike last_name: Trautmann citation: ama: 'Heins J, Rook J, Schäpermeier L, Kerschke P, Bossek J, Trautmann H. BBE: Basin-Based Evaluation of Multimodal Multi-objective Optimization Problems. In: Rudolph G, Kononova AV, Aguirre H, Kerschke P, Ochoa G, Tusar T, eds. Parallel Problem Solving from Nature (PPSN XVII). Lecture Notes in Computer Science. Springer International Publishing; 2022:192–206. doi:10.1007/978-3-031-14714-2_14' apa: 'Heins, J., Rook, J., Schäpermeier, L., Kerschke, P., Bossek, J., & Trautmann, H. (2022). BBE: Basin-Based Evaluation of Multimodal Multi-objective Optimization Problems. In G. Rudolph, A. V. Kononova, H. Aguirre, P. Kerschke, G. Ochoa, & T. Tusar (Eds.), Parallel Problem Solving from Nature (PPSN XVII) (pp. 192–206). Springer International Publishing. https://doi.org/10.1007/978-3-031-14714-2_14' bibtex: '@inproceedings{Heins_Rook_Schäpermeier_Kerschke_Bossek_Trautmann_2022, place={Cham}, series={Lecture Notes in Computer Science}, title={BBE: Basin-Based Evaluation of Multimodal Multi-objective Optimization Problems}, DOI={10.1007/978-3-031-14714-2_14}, booktitle={Parallel Problem Solving from Nature (PPSN XVII)}, publisher={Springer International Publishing}, author={Heins, Jonathan and Rook, Jeroen and Schäpermeier, Lennart and Kerschke, Pascal and Bossek, Jakob and Trautmann, Heike}, editor={Rudolph, Günter and Kononova, Anna V. and Aguirre, Hernán and Kerschke, Pascal and Ochoa, Gabriela and Tusar, Tea}, year={2022}, pages={192–206}, collection={Lecture Notes in Computer Science} }' chicago: 'Heins, Jonathan, Jeroen Rook, Lennart Schäpermeier, Pascal Kerschke, Jakob Bossek, and Heike Trautmann. “BBE: Basin-Based Evaluation of Multimodal Multi-Objective Optimization Problems.” In Parallel Problem Solving from Nature (PPSN XVII), edited by Günter Rudolph, Anna V. Kononova, Hernán Aguirre, Pascal Kerschke, Gabriela Ochoa, and Tea Tusar, 192–206. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2022. https://doi.org/10.1007/978-3-031-14714-2_14.' ieee: 'J. Heins, J. Rook, L. Schäpermeier, P. Kerschke, J. Bossek, and H. Trautmann, “BBE: Basin-Based Evaluation of Multimodal Multi-objective Optimization Problems,” in Parallel Problem Solving from Nature (PPSN XVII), 2022, pp. 192–206, doi: 10.1007/978-3-031-14714-2_14.' mla: 'Heins, Jonathan, et al. “BBE: Basin-Based Evaluation of Multimodal Multi-Objective Optimization Problems.” Parallel Problem Solving from Nature (PPSN XVII), edited by Günter Rudolph et al., Springer International Publishing, 2022, pp. 192–206, doi:10.1007/978-3-031-14714-2_14.' short: 'J. Heins, J. Rook, L. Schäpermeier, P. Kerschke, J. Bossek, H. Trautmann, in: G. Rudolph, A.V. Kononova, H. Aguirre, P. Kerschke, G. Ochoa, T. Tusar (Eds.), Parallel Problem Solving from Nature (PPSN XVII), Springer International Publishing, Cham, 2022, pp. 192–206.' date_created: 2023-11-14T15:58:58Z date_updated: 2023-12-13T10:47:50Z department: - _id: '819' doi: 10.1007/978-3-031-14714-2_14 editor: - first_name: Günter full_name: Rudolph, Günter last_name: Rudolph - first_name: Anna V. full_name: Kononova, Anna V. last_name: Kononova - first_name: Hernán full_name: Aguirre, Hernán last_name: Aguirre - first_name: Pascal full_name: Kerschke, Pascal last_name: Kerschke - first_name: Gabriela full_name: Ochoa, Gabriela last_name: Ochoa - first_name: Tea full_name: Tusar, Tea last_name: Tusar extern: '1' keyword: - Anytime behavior - Benchmarking - Continuous optimization - Multi-objective optimization - Multimodality - Performance metric language: - iso: eng page: 192–206 place: Cham publication: Parallel Problem Solving from Nature (PPSN XVII) publication_identifier: isbn: - 978-3-031-14714-2 publisher: Springer International Publishing series_title: Lecture Notes in Computer Science status: public title: 'BBE: Basin-Based Evaluation of Multimodal Multi-objective Optimization Problems' type: conference user_id: '102979' year: '2022' ... --- _id: '46318' abstract: - lang: eng text: 'Multi-objective (MO) optimization, i.e., the simultaneous optimization of multiple conflicting objectives, is gaining more and more attention in various research areas, such as evolutionary computation, machine learning (e.g., (hyper-)parameter optimization), or logistics (e.g., vehicle routing). Many works in this domain mention the structural problem property of multimodality as a challenge from two classical perspectives: (1) finding all globally optimal solution sets, and (2) avoiding to get trapped in local optima. Interestingly, these streams seem to transfer many traditional concepts of single-objective (SO) optimization into claims, assumptions, or even terminology regarding the MO domain, but mostly neglect the understanding of the structural properties as well as the algorithmic search behavior on a problem’s landscape. However, some recent works counteract this trend, by investigating the fundamentals and characteristics of MO problems using new visualization techniques and gaining surprising insights. Using these visual insights, this work proposes a step towards a unified terminology to capture multimodality and locality in a broader way than it is usually done. This enables us to investigate current research activities in multimodal continuous MO optimization and to highlight new implications and promising research directions for the design of benchmark suites, the discovery of MO landscape features, the development of new MO (or even SO) optimization algorithms, and performance indicators. For all these topics, we provide a review of ideas and methods but also an outlook on future challenges, research potential and perspectives that result from recent developments.' author: - first_name: Christian full_name: Grimme, Christian last_name: Grimme - first_name: Pascal full_name: Kerschke, Pascal last_name: Kerschke - first_name: Pelin full_name: Aspar, Pelin last_name: Aspar - first_name: Heike full_name: Trautmann, Heike id: '100740' last_name: Trautmann orcid: 0000-0002-9788-8282 - first_name: Mike full_name: Preuss, Mike last_name: Preuss - first_name: André H. full_name: Deutz, André H. last_name: Deutz - first_name: Hao full_name: Wang, Hao last_name: Wang - first_name: Michael full_name: Emmerich, Michael last_name: Emmerich citation: ama: 'Grimme C, Kerschke P, Aspar P, et al. Peeking beyond peaks: Challenges and research potentials of continuous multimodal multi-objective optimization. Computers & Operations Research. 2021;136:105489. doi:https://doi.org/10.1016/j.cor.2021.105489' apa: 'Grimme, C., Kerschke, P., Aspar, P., Trautmann, H., Preuss, M., Deutz, A. H., Wang, H., & Emmerich, M. (2021). Peeking beyond peaks: Challenges and research potentials of continuous multimodal multi-objective optimization. Computers & Operations Research, 136, 105489. https://doi.org/10.1016/j.cor.2021.105489' bibtex: '@article{Grimme_Kerschke_Aspar_Trautmann_Preuss_Deutz_Wang_Emmerich_2021, title={Peeking beyond peaks: Challenges and research potentials of continuous multimodal multi-objective optimization}, volume={136}, DOI={https://doi.org/10.1016/j.cor.2021.105489}, journal={Computers & Operations Research}, author={Grimme, Christian and Kerschke, Pascal and Aspar, Pelin and Trautmann, Heike and Preuss, Mike and Deutz, André H. and Wang, Hao and Emmerich, Michael}, year={2021}, pages={105489} }' chicago: 'Grimme, Christian, Pascal Kerschke, Pelin Aspar, Heike Trautmann, Mike Preuss, André H. Deutz, Hao Wang, and Michael Emmerich. “Peeking beyond Peaks: Challenges and Research Potentials of Continuous Multimodal Multi-Objective Optimization.” Computers & Operations Research 136 (2021): 105489. https://doi.org/10.1016/j.cor.2021.105489.' ieee: 'C. Grimme et al., “Peeking beyond peaks: Challenges and research potentials of continuous multimodal multi-objective optimization,” Computers & Operations Research, vol. 136, p. 105489, 2021, doi: https://doi.org/10.1016/j.cor.2021.105489.' mla: 'Grimme, Christian, et al. “Peeking beyond Peaks: Challenges and Research Potentials of Continuous Multimodal Multi-Objective Optimization.” Computers & Operations Research, vol. 136, 2021, p. 105489, doi:https://doi.org/10.1016/j.cor.2021.105489.' short: C. Grimme, P. Kerschke, P. Aspar, H. Trautmann, M. Preuss, A.H. Deutz, H. Wang, M. Emmerich, Computers & Operations Research 136 (2021) 105489. date_created: 2023-08-04T07:28:34Z date_updated: 2023-10-16T12:58:42Z department: - _id: '34' - _id: '819' doi: https://doi.org/10.1016/j.cor.2021.105489 intvolume: ' 136' keyword: - Multimodal optimization - Multi-objective continuous optimization - Landscape analysis - Visualization - Benchmarking - Theory - Algorithms language: - iso: eng page: '105489' publication: Computers & Operations Research publication_identifier: issn: - 0305-0548 status: public title: 'Peeking beyond peaks: Challenges and research potentials of continuous multimodal multi-objective optimization' type: journal_article user_id: '15504' volume: 136 year: '2021' ... --- _id: '48849' abstract: - lang: eng text: One-shot optimization tasks require to determine the set of solution candidates prior to their evaluation, i.e., without possibility for adaptive sampling. We consider two variants, classic one-shot optimization (where our aim is to find at least one solution of high quality) and one-shot regression (where the goal is to fit a model that resembles the true problem as well as possible). For both tasks it seems intuitive that well-distributed samples should perform better than uniform or grid-based samples, since they show a better coverage of the decision space. In practice, quasi-random designs such as Latin Hypercube Samples and low-discrepancy point sets are indeed very commonly used designs for one-shot optimization tasks. We study in this work how well low star discrepancy correlates with performance in one-shot optimization. Our results confirm an advantage of low-discrepancy designs, but also indicate the correlation between discrepancy values and overall performance is rather weak. We then demonstrate that commonly used designs may be far from optimal. More precisely, we evolve 24 very specific designs that each achieve good performance on one of our benchmark problems. Interestingly, we find that these specifically designed samples yield surprisingly good performance across the whole benchmark set. Our results therefore give strong indication that significant performance gains over state-of-the-art one-shot sampling techniques are possible, and that evolutionary algorithms can be an efficient means to evolve these. author: - first_name: Jakob full_name: Bossek, Jakob id: '102979' last_name: Bossek orcid: 0000-0002-4121-4668 - first_name: Carola full_name: Doerr, Carola last_name: Doerr - first_name: Pascal full_name: Kerschke, Pascal last_name: Kerschke - first_name: Aneta full_name: Neumann, Aneta last_name: Neumann - first_name: Frank full_name: Neumann, Frank last_name: Neumann citation: ama: 'Bossek J, Doerr C, Kerschke P, Neumann A, Neumann F. Evolving Sampling Strategies for One-Shot Optimization Tasks. In: Parallel Problem Solving from Nature (PPSN XVI). Springer-Verlag; 2020:111–124. doi:10.1007/978-3-030-58112-1_8' apa: Bossek, J., Doerr, C., Kerschke, P., Neumann, A., & Neumann, F. (2020). Evolving Sampling Strategies for One-Shot Optimization Tasks. Parallel Problem Solving from Nature (PPSN XVI), 111–124. https://doi.org/10.1007/978-3-030-58112-1_8 bibtex: '@inproceedings{Bossek_Doerr_Kerschke_Neumann_Neumann_2020, place={Berlin, Heidelberg}, title={Evolving Sampling Strategies for One-Shot Optimization Tasks}, DOI={10.1007/978-3-030-58112-1_8}, booktitle={Parallel Problem Solving from Nature (PPSN XVI)}, publisher={Springer-Verlag}, author={Bossek, Jakob and Doerr, Carola and Kerschke, Pascal and Neumann, Aneta and Neumann, Frank}, year={2020}, pages={111–124} }' chicago: 'Bossek, Jakob, Carola Doerr, Pascal Kerschke, Aneta Neumann, and Frank Neumann. “Evolving Sampling Strategies for One-Shot Optimization Tasks.” In Parallel Problem Solving from Nature (PPSN XVI), 111–124. Berlin, Heidelberg: Springer-Verlag, 2020. https://doi.org/10.1007/978-3-030-58112-1_8.' ieee: 'J. Bossek, C. Doerr, P. Kerschke, A. Neumann, and F. Neumann, “Evolving Sampling Strategies for One-Shot Optimization Tasks,” in Parallel Problem Solving from Nature (PPSN XVI), 2020, pp. 111–124, doi: 10.1007/978-3-030-58112-1_8.' mla: Bossek, Jakob, et al. “Evolving Sampling Strategies for One-Shot Optimization Tasks.” Parallel Problem Solving from Nature (PPSN XVI), Springer-Verlag, 2020, pp. 111–124, doi:10.1007/978-3-030-58112-1_8. short: 'J. Bossek, C. Doerr, P. Kerschke, A. Neumann, F. Neumann, in: Parallel Problem Solving from Nature (PPSN XVI), Springer-Verlag, Berlin, Heidelberg, 2020, pp. 111–124.' date_created: 2023-11-14T15:58:53Z date_updated: 2023-12-13T10:43:53Z department: - _id: '819' doi: 10.1007/978-3-030-58112-1_8 extern: '1' keyword: - Continuous optimization - Fully parallel search - One-shot optimization - Regression - Surrogate-assisted optimization language: - iso: eng page: 111–124 place: Berlin, Heidelberg publication: Parallel Problem Solving from Nature (PPSN XVI) publication_identifier: isbn: - 978-3-030-58111-4 publication_status: published publisher: Springer-Verlag status: public title: Evolving Sampling Strategies for One-Shot Optimization Tasks type: conference user_id: '102979' year: '2020' ...