---
_id: '48850'
abstract:
- lang: eng
text: Sequential model-based optimization (SMBO) approaches are algorithms for solving
problems that require computationally or otherwise expensive function evaluations.
The key design principle of SMBO is a substitution of the true objective function
by a surrogate, which is used to propose the point(s) to be evaluated next. SMBO
algorithms are intrinsically modular, leaving the user with many important design
choices. Significant research efforts go into understanding which settings perform
best for which type of problems. Most works, however, focus on the choice of the
model, the acquisition function, and the strategy used to optimize the latter.
The choice of the initial sampling strategy, however, receives much less attention.
Not surprisingly, quite diverging recommendations can be found in the literature.
We analyze in this work how the size and the distribution of the initial sample
influences the overall quality of the efficient global optimization (EGO) algorithm,
a well-known SMBO approach. While, overall, small initial budgets using Halton
sampling seem preferable, we also observe that the performance landscape is rather
unstructured. We furthermore identify several situations in which EGO performs
unfavorably against random sampling. Both observations indicate that an adaptive
SMBO design could be beneficial, making SMBO an interesting test-bed for automated
algorithm design.
author:
- first_name: Jakob
full_name: Bossek, Jakob
id: '102979'
last_name: Bossek
orcid: 0000-0002-4121-4668
- first_name: Carola
full_name: Doerr, Carola
last_name: Doerr
- first_name: Pascal
full_name: Kerschke, Pascal
last_name: Kerschke
citation:
ama: 'Bossek J, Doerr C, Kerschke P. Initial Design Strategies and Their Effects
on Sequential Model-Based Optimization: An Exploratory Case Study Based on BBOB.
In: Proceedings of the Genetic and Evolutionary Computation Conference.
GECCO ’20. Association for Computing Machinery; 2020:778–786. doi:10.1145/3377930.3390155'
apa: 'Bossek, J., Doerr, C., & Kerschke, P. (2020). Initial Design Strategies
and Their Effects on Sequential Model-Based Optimization: An Exploratory Case
Study Based on BBOB. Proceedings of the Genetic and Evolutionary Computation
Conference, 778–786. https://doi.org/10.1145/3377930.3390155'
bibtex: '@inproceedings{Bossek_Doerr_Kerschke_2020, place={New York, NY, USA}, series={GECCO
’20}, title={Initial Design Strategies and Their Effects on Sequential Model-Based
Optimization: An Exploratory Case Study Based on BBOB}, DOI={10.1145/3377930.3390155},
booktitle={Proceedings of the Genetic and Evolutionary Computation Conference},
publisher={Association for Computing Machinery}, author={Bossek, Jakob and Doerr,
Carola and Kerschke, Pascal}, year={2020}, pages={778–786}, collection={GECCO
’20} }'
chicago: 'Bossek, Jakob, Carola Doerr, and Pascal Kerschke. “Initial Design Strategies
and Their Effects on Sequential Model-Based Optimization: An Exploratory Case
Study Based on BBOB.” In Proceedings of the Genetic and Evolutionary Computation
Conference, 778–786. GECCO ’20. New York, NY, USA: Association for Computing
Machinery, 2020. https://doi.org/10.1145/3377930.3390155.'
ieee: 'J. Bossek, C. Doerr, and P. Kerschke, “Initial Design Strategies and Their
Effects on Sequential Model-Based Optimization: An Exploratory Case Study Based
on BBOB,” in Proceedings of the Genetic and Evolutionary Computation Conference,
2020, pp. 778–786, doi: 10.1145/3377930.3390155.'
mla: 'Bossek, Jakob, et al. “Initial Design Strategies and Their Effects on Sequential
Model-Based Optimization: An Exploratory Case Study Based on BBOB.” Proceedings
of the Genetic and Evolutionary Computation Conference, Association for Computing
Machinery, 2020, pp. 778–786, doi:10.1145/3377930.3390155.'
short: 'J. Bossek, C. Doerr, P. Kerschke, in: Proceedings of the Genetic and Evolutionary
Computation Conference, Association for Computing Machinery, New York, NY, USA,
2020, pp. 778–786.'
date_created: 2023-11-14T15:58:53Z
date_updated: 2023-12-13T10:44:01Z
department:
- _id: '819'
doi: 10.1145/3377930.3390155
extern: '1'
keyword:
- continuous black-box optimization
- design of experiments
- initial design
- sequential model-based optimization
language:
- iso: eng
page: 778–786
place: New York, NY, USA
publication: Proceedings of the Genetic and Evolutionary Computation Conference
publication_identifier:
isbn:
- 978-1-4503-7128-5
publication_status: published
publisher: Association for Computing Machinery
series_title: GECCO ’20
status: public
title: 'Initial Design Strategies and Their Effects on Sequential Model-Based Optimization:
An Exploratory Case Study Based on BBOB'
type: conference
user_id: '102979'
year: '2020'
...
---
_id: '48838'
abstract:
- lang: eng
text: 'The majority of algorithms can be controlled or adjusted by parameters. Their
values can substantially affect the algorithms’ performance. Since the manual
exploration of the parameter space is tedious – even for few parameters – several
automatic procedures for parameter tuning have been proposed. Recent approaches
also take into account some characteristic properties of the problem instances,
frequently termed instance features. Our contribution is the proposal of a novel
concept for feature-based algorithm parameter tuning, which applies an approximating
surrogate model for learning the continuous feature-parameter mapping. To accomplish
this, we learn a joint model of the algorithm performance based on both the algorithm
parameters and the instance features. The required data is gathered using a recently
proposed acquisition function for model refinement in surrogate-based optimization:
the profile expected improvement. This function provides an avenue for maximizing
the information required for the feature-parameter mapping, i.e., the mapping
from instance features to the corresponding optimal algorithm parameters. The
approach is validated by applying the tuner to exemplary evolutionary algorithms
and problems, for which theoretically grounded or heuristically determined feature-parameter
mappings are available.'
author:
- first_name: Jakob
full_name: Bossek, Jakob
id: '102979'
last_name: Bossek
orcid: 0000-0002-4121-4668
- first_name: Bernd
full_name: Bischl, Bernd
last_name: Bischl
- first_name: Tobias
full_name: Wagner, Tobias
last_name: Wagner
- first_name: Günter
full_name: Rudolph, Günter
last_name: Rudolph
citation:
ama: 'Bossek J, Bischl B, Wagner T, Rudolph G. Learning Feature-Parameter Mappings
for Parameter Tuning via the Profile Expected Improvement. In: Proceedings
of the Genetic and Evolutionary Computation Conference. GECCO ’15. Association
for Computing Machinery; 2015:1319–1326. doi:10.1145/2739480.2754673'
apa: Bossek, J., Bischl, B., Wagner, T., & Rudolph, G. (2015). Learning Feature-Parameter
Mappings for Parameter Tuning via the Profile Expected Improvement. Proceedings
of the Genetic and Evolutionary Computation Conference, 1319–1326. https://doi.org/10.1145/2739480.2754673
bibtex: '@inproceedings{Bossek_Bischl_Wagner_Rudolph_2015, place={New York, NY,
USA}, series={GECCO ’15}, title={Learning Feature-Parameter Mappings for Parameter
Tuning via the Profile Expected Improvement}, DOI={10.1145/2739480.2754673},
booktitle={Proceedings of the Genetic and Evolutionary Computation Conference},
publisher={Association for Computing Machinery}, author={Bossek, Jakob and Bischl,
Bernd and Wagner, Tobias and Rudolph, Günter}, year={2015}, pages={1319–1326},
collection={GECCO ’15} }'
chicago: 'Bossek, Jakob, Bernd Bischl, Tobias Wagner, and Günter Rudolph. “Learning
Feature-Parameter Mappings for Parameter Tuning via the Profile Expected Improvement.”
In Proceedings of the Genetic and Evolutionary Computation Conference,
1319–1326. GECCO ’15. New York, NY, USA: Association for Computing Machinery,
2015. https://doi.org/10.1145/2739480.2754673.'
ieee: 'J. Bossek, B. Bischl, T. Wagner, and G. Rudolph, “Learning Feature-Parameter
Mappings for Parameter Tuning via the Profile Expected Improvement,” in Proceedings
of the Genetic and Evolutionary Computation Conference, 2015, pp. 1319–1326,
doi: 10.1145/2739480.2754673.'
mla: Bossek, Jakob, et al. “Learning Feature-Parameter Mappings for Parameter Tuning
via the Profile Expected Improvement.” Proceedings of the Genetic and Evolutionary
Computation Conference, Association for Computing Machinery, 2015, pp. 1319–1326,
doi:10.1145/2739480.2754673.
short: 'J. Bossek, B. Bischl, T. Wagner, G. Rudolph, in: Proceedings of the Genetic
and Evolutionary Computation Conference, Association for Computing Machinery,
New York, NY, USA, 2015, pp. 1319–1326.'
date_created: 2023-11-14T15:58:51Z
date_updated: 2023-12-13T10:40:30Z
department:
- _id: '819'
doi: 10.1145/2739480.2754673
extern: '1'
keyword:
- evolutionary algorithms
- model-based optimization
- parameter tuning
language:
- iso: eng
page: 1319–1326
place: New York, NY, USA
publication: Proceedings of the Genetic and Evolutionary Computation Conference
publication_identifier:
isbn:
- 978-1-4503-3472-3
publication_status: published
publisher: Association for Computing Machinery
series_title: GECCO ’15
status: public
title: Learning Feature-Parameter Mappings for Parameter Tuning via the Profile Expected
Improvement
type: conference
user_id: '102979'
year: '2015'
...