---
_id: '30868'
abstract:
- lang: eng
  text: "Algorithm configuration (AC) is concerned with the automated search of the\r\nmost
    suitable parameter configuration of a parametrized algorithm. There is\r\ncurrently
    a wide variety of AC problem variants and methods proposed in the\r\nliterature.
    Existing reviews do not take into account all derivatives of the AC\r\nproblem,
    nor do they offer a complete classification scheme. To this end, we\r\nintroduce
    taxonomies to describe the AC problem and features of configuration\r\nmethods,
    respectively. We review existing AC literature within the lens of our\r\ntaxonomies,
    outline relevant design choices of configuration approaches,\r\ncontrast methods
    and problem variants against each other, and describe the\r\nstate of AC in industry.
    Finally, our review provides researchers and\r\npractitioners with a look at future
    research directions in the field of AC."
author:
- first_name: Elias
  full_name: Schede, Elias
  last_name: Schede
- first_name: Jasmin
  full_name: Brandt, Jasmin
  last_name: Brandt
- first_name: Alexander
  full_name: Tornede, Alexander
  id: '38209'
  last_name: Tornede
- first_name: Marcel Dominik
  full_name: Wever, Marcel Dominik
  id: '33176'
  last_name: Wever
  orcid: ' https://orcid.org/0000-0001-9782-6818'
- first_name: Viktor
  full_name: Bengs, Viktor
  id: '76599'
  last_name: Bengs
- first_name: Eyke
  full_name: Hüllermeier, Eyke
  id: '48129'
  last_name: Hüllermeier
- first_name: Kevin
  full_name: Tierney, Kevin
  last_name: Tierney
citation:
  ama: Schede E, Brandt J, Tornede A, et al. A Survey of Methods for Automated Algorithm
    Configuration. <i>arXiv:220201651</i>. Published online 2022.
  apa: Schede, E., Brandt, J., Tornede, A., Wever, M. D., Bengs, V., Hüllermeier,
    E., &#38; Tierney, K. (2022). A Survey of Methods for Automated Algorithm Configuration.
    In <i>arXiv:2202.01651</i>.
  bibtex: '@article{Schede_Brandt_Tornede_Wever_Bengs_Hüllermeier_Tierney_2022, title={A
    Survey of Methods for Automated Algorithm Configuration}, journal={arXiv:2202.01651},
    author={Schede, Elias and Brandt, Jasmin and Tornede, Alexander and Wever, Marcel
    Dominik and Bengs, Viktor and Hüllermeier, Eyke and Tierney, Kevin}, year={2022}
    }'
  chicago: Schede, Elias, Jasmin Brandt, Alexander Tornede, Marcel Dominik Wever,
    Viktor Bengs, Eyke Hüllermeier, and Kevin Tierney. “A Survey of Methods for Automated
    Algorithm Configuration.” <i>ArXiv:2202.01651</i>, 2022.
  ieee: E. Schede <i>et al.</i>, “A Survey of Methods for Automated Algorithm Configuration,”
    <i>arXiv:2202.01651</i>. 2022.
  mla: Schede, Elias, et al. “A Survey of Methods for Automated Algorithm Configuration.”
    <i>ArXiv:2202.01651</i>, 2022.
  short: E. Schede, J. Brandt, A. Tornede, M.D. Wever, V. Bengs, E. Hüllermeier, K.
    Tierney, ArXiv:2202.01651 (2022).
date_created: 2022-04-12T12:00:08Z
date_updated: 2022-04-12T12:01:15Z
department:
- _id: '34'
- _id: '7'
- _id: '26'
external_id:
  arxiv:
  - '2202.01651'
language:
- iso: eng
project:
- _id: '1'
  name: 'SFB 901: SFB 901'
- _id: '3'
  name: 'SFB 901 - B: SFB 901 - Project Area B'
- _id: '10'
  name: 'SFB 901 - B2: SFB 901 - Subproject B2'
publication: arXiv:2202.01651
status: public
title: A Survey of Methods for Automated Algorithm Configuration
type: preprint
user_id: '38209'
year: '2022'
...
---
_id: '30867'
abstract:
- lang: eng
  text: "In online algorithm selection (OAS), instances of an algorithmic problem\r\nclass
    are presented to an agent one after another, and the agent has to quickly\r\nselect
    a presumably best algorithm from a fixed set of candidate algorithms.\r\nFor decision
    problems such as satisfiability (SAT), quality typically refers to\r\nthe algorithm's
    runtime. As the latter is known to exhibit a heavy-tail\r\ndistribution, an algorithm
    is normally stopped when exceeding a predefined\r\nupper time limit. As a consequence,
    machine learning methods used to optimize\r\nan algorithm selection strategy in
    a data-driven manner need to deal with\r\nright-censored samples, a problem that
    has received little attention in the\r\nliterature so far. In this work, we revisit
    multi-armed bandit algorithms for\r\nOAS and discuss their capability of dealing
    with the problem. Moreover, we\r\nadapt them towards runtime-oriented losses,
    allowing for partially censored\r\ndata while keeping a space- and time-complexity
    independent of the time\r\nhorizon. In an extensive experimental evaluation on
    an adapted version of the\r\nASlib benchmark, we demonstrate that theoretically
    well-founded methods based\r\non Thompson sampling perform specifically strong
    and improve in comparison to\r\nexisting methods."
author:
- first_name: Alexander
  full_name: Tornede, Alexander
  id: '38209'
  last_name: Tornede
- first_name: Viktor
  full_name: Bengs, Viktor
  id: '76599'
  last_name: Bengs
- first_name: Eyke
  full_name: Hüllermeier, Eyke
  id: '48129'
  last_name: Hüllermeier
citation:
  ama: Tornede A, Bengs V, Hüllermeier E. Machine Learning for Online Algorithm Selection
    under Censored Feedback. <i>Proceedings of the 36th AAAI Conference on Artificial
    Intelligence</i>. Published online 2022.
  apa: Tornede, A., Bengs, V., &#38; Hüllermeier, E. (2022). Machine Learning for
    Online Algorithm Selection under Censored Feedback. In <i>Proceedings of the 36th
    AAAI Conference on Artificial Intelligence</i>. AAAI.
  bibtex: '@article{Tornede_Bengs_Hüllermeier_2022, title={Machine Learning for Online
    Algorithm Selection under Censored Feedback}, journal={Proceedings of the 36th
    AAAI Conference on Artificial Intelligence}, publisher={AAAI}, author={Tornede,
    Alexander and Bengs, Viktor and Hüllermeier, Eyke}, year={2022} }'
  chicago: Tornede, Alexander, Viktor Bengs, and Eyke Hüllermeier. “Machine Learning
    for Online Algorithm Selection under Censored Feedback.” <i>Proceedings of the
    36th AAAI Conference on Artificial Intelligence</i>. AAAI, 2022.
  ieee: A. Tornede, V. Bengs, and E. Hüllermeier, “Machine Learning for Online Algorithm
    Selection under Censored Feedback,” <i>Proceedings of the 36th AAAI Conference
    on Artificial Intelligence</i>. AAAI, 2022.
  mla: Tornede, Alexander, et al. “Machine Learning for Online Algorithm Selection
    under Censored Feedback.” <i>Proceedings of the 36th AAAI Conference on Artificial
    Intelligence</i>, AAAI, 2022.
  short: A. Tornede, V. Bengs, E. Hüllermeier, Proceedings of the 36th AAAI Conference
    on Artificial Intelligence (2022).
date_created: 2022-04-12T11:58:56Z
date_updated: 2022-08-24T12:44:27Z
department:
- _id: '34'
- _id: '7'
- _id: '26'
external_id:
  arxiv:
  - '2109.06234'
language:
- iso: eng
project:
- _id: '1'
  name: 'SFB 901: SFB 901'
- _id: '3'
  name: 'SFB 901 - B: SFB 901 - Project Area B'
- _id: '10'
  name: 'SFB 901 - B2: SFB 901 - Subproject B2'
publication: Proceedings of the 36th AAAI Conference on Artificial Intelligence
publisher: AAAI
status: public
title: Machine Learning for Online Algorithm Selection under Censored Feedback
type: preprint
user_id: '38209'
year: '2022'
...
---
_id: '18014'
author:
- first_name: Adil
  full_name: El Mesaoudi-Paul, Adil
  last_name: El Mesaoudi-Paul
- first_name: Dimitri
  full_name: Weiß, Dimitri
  last_name: Weiß
- first_name: Viktor
  full_name: Bengs, Viktor
  id: '76599'
  last_name: Bengs
- first_name: Eyke
  full_name: Hüllermeier, Eyke
  id: '48129'
  last_name: Hüllermeier
- first_name: Kevin
  full_name: Tierney, Kevin
  last_name: Tierney
citation:
  ama: 'El Mesaoudi-Paul A, Weiß D, Bengs V, Hüllermeier E, Tierney K. Pool-Based
    Realtime Algorithm Configuration: A Preselection Bandit Approach. In: <i>Learning
    and Intelligent Optimization. LION 2020.</i> Vol 12096. Lecture Notes in Computer
    Science. Cham: Springer; 2020:216-232. doi:<a href="https://doi.org/10.1007/978-3-030-53552-0_22">10.1007/978-3-030-53552-0_22</a>'
  apa: 'El Mesaoudi-Paul, A., Weiß, D., Bengs, V., Hüllermeier, E., &#38; Tierney,
    K. (2020). Pool-Based Realtime Algorithm Configuration: A Preselection Bandit
    Approach. In <i>Learning and Intelligent Optimization. LION 2020.</i> (Vol. 12096,
    pp. 216–232). Cham: Springer. <a href="https://doi.org/10.1007/978-3-030-53552-0_22">https://doi.org/10.1007/978-3-030-53552-0_22</a>'
  bibtex: '@inbook{El Mesaoudi-Paul_Weiß_Bengs_Hüllermeier_Tierney_2020, place={Cham},
    series={Lecture Notes in Computer Science}, title={Pool-Based Realtime Algorithm
    Configuration: A Preselection Bandit Approach}, volume={12096}, DOI={<a href="https://doi.org/10.1007/978-3-030-53552-0_22">10.1007/978-3-030-53552-0_22</a>},
    booktitle={Learning and Intelligent Optimization. LION 2020.}, publisher={Springer},
    author={El Mesaoudi-Paul, Adil and Weiß, Dimitri and Bengs, Viktor and Hüllermeier,
    Eyke and Tierney, Kevin}, year={2020}, pages={216–232}, collection={Lecture Notes
    in Computer Science} }'
  chicago: 'El Mesaoudi-Paul, Adil, Dimitri Weiß, Viktor Bengs, Eyke Hüllermeier,
    and Kevin Tierney. “Pool-Based Realtime Algorithm Configuration: A Preselection
    Bandit Approach.” In <i>Learning and Intelligent Optimization. LION 2020.</i>,
    12096:216–32. Lecture Notes in Computer Science. Cham: Springer, 2020. <a href="https://doi.org/10.1007/978-3-030-53552-0_22">https://doi.org/10.1007/978-3-030-53552-0_22</a>.'
  ieee: 'A. El Mesaoudi-Paul, D. Weiß, V. Bengs, E. Hüllermeier, and K. Tierney, “Pool-Based
    Realtime Algorithm Configuration: A Preselection Bandit Approach,” in <i>Learning
    and Intelligent Optimization. LION 2020.</i>, vol. 12096, Cham: Springer, 2020,
    pp. 216–232.'
  mla: 'El Mesaoudi-Paul, Adil, et al. “Pool-Based Realtime Algorithm Configuration:
    A Preselection Bandit Approach.” <i>Learning and Intelligent Optimization. LION
    2020.</i>, vol. 12096, Springer, 2020, pp. 216–32, doi:<a href="https://doi.org/10.1007/978-3-030-53552-0_22">10.1007/978-3-030-53552-0_22</a>.'
  short: 'A. El Mesaoudi-Paul, D. Weiß, V. Bengs, E. Hüllermeier, K. Tierney, in:
    Learning and Intelligent Optimization. LION 2020., Springer, Cham, 2020, pp. 216–232.'
date_created: 2020-08-17T11:44:37Z
date_updated: 2022-01-06T06:53:25Z
department:
- _id: '34'
- _id: '7'
- _id: '355'
doi: 10.1007/978-3-030-53552-0_22
intvolume: '     12096'
language:
- iso: eng
page: 216 - 232
place: Cham
project:
- _id: '52'
  name: Computing Resources Provided by the Paderborn Center for Parallel Computing
publication: Learning and Intelligent Optimization. LION 2020.
publication_identifier:
  isbn:
  - '9783030535513'
  - '9783030535520'
  issn:
  - 0302-9743
  - 1611-3349
publication_status: published
publisher: Springer
series_title: Lecture Notes in Computer Science
status: public
title: 'Pool-Based Realtime Algorithm Configuration: A Preselection Bandit Approach'
type: book_chapter
user_id: '76599'
volume: 12096
year: '2020'
...
---
_id: '18017'
abstract:
- lang: eng
  text: "We consider an extension of the contextual multi-armed bandit problem, in\r\nwhich,
    instead of selecting a single alternative (arm), a learner is supposed\r\nto make
    a preselection in the form of a subset of alternatives. More\r\nspecifically,
    in each iteration, the learner is presented a set of arms and a\r\ncontext, both
    described in terms of feature vectors. The task of the learner is\r\nto preselect
    $k$ of these arms, among which a final choice is made in a second\r\nstep. In
    our setup, we assume that each arm has a latent (context-dependent)\r\nutility,
    and that feedback on a preselection is produced according to a\r\nPlackett-Luce
    model. We propose the CPPL algorithm, which is inspired by the\r\nwell-known UCB
    algorithm, and evaluate this algorithm on synthetic and real\r\ndata. In particular,
    we consider an online algorithm selection scenario, which\r\nserved as a main
    motivation of our problem setting. Here, an instance (which\r\ndefines the context)
    from a certain problem class (such as SAT) can be solved\r\nby different algorithms
    (the arms), but only $k$ of these algorithms can\r\nactually be run."
author:
- first_name: Adil
  full_name: El Mesaoudi-Paul, Adil
  last_name: El Mesaoudi-Paul
- first_name: Viktor
  full_name: Bengs, Viktor
  id: '76599'
  last_name: Bengs
- first_name: Eyke
  full_name: Hüllermeier, Eyke
  id: '48129'
  last_name: Hüllermeier
citation:
  ama: El Mesaoudi-Paul A, Bengs V, Hüllermeier E. Online Preselection with Context
    Information under the Plackett-Luce  Model. <i>arXiv:200204275</i>.
  apa: El Mesaoudi-Paul, A., Bengs, V., &#38; Hüllermeier, E. (n.d.). Online Preselection
    with Context Information under the Plackett-Luce  Model. <i>ArXiv:2002.04275</i>.
  bibtex: '@article{El Mesaoudi-Paul_Bengs_Hüllermeier, title={Online Preselection
    with Context Information under the Plackett-Luce  Model}, journal={arXiv:2002.04275},
    author={El Mesaoudi-Paul, Adil and Bengs, Viktor and Hüllermeier, Eyke} }'
  chicago: El Mesaoudi-Paul, Adil, Viktor Bengs, and Eyke Hüllermeier. “Online Preselection
    with Context Information under the Plackett-Luce  Model.” <i>ArXiv:2002.04275</i>,
    n.d.
  ieee: A. El Mesaoudi-Paul, V. Bengs, and E. Hüllermeier, “Online Preselection with
    Context Information under the Plackett-Luce  Model,” <i>arXiv:2002.04275</i>.
    .
  mla: El Mesaoudi-Paul, Adil, et al. “Online Preselection with Context Information
    under the Plackett-Luce  Model.” <i>ArXiv:2002.04275</i>.
  short: A. El Mesaoudi-Paul, V. Bengs, E. Hüllermeier, ArXiv:2002.04275 (n.d.).
date_created: 2020-08-17T11:49:40Z
date_updated: 2022-01-06T06:53:25Z
department:
- _id: '34'
- _id: '7'
- _id: '355'
language:
- iso: eng
project:
- _id: '52'
  name: Computing Resources Provided by the Paderborn Center for Parallel Computing
publication: arXiv:2002.04275
publication_status: draft
status: public
title: Online Preselection with Context Information under the Plackett-Luce  Model
type: preprint
user_id: '76599'
year: '2020'
...
---
_id: '14027'
author:
- first_name: Viktor
  full_name: Bengs, Viktor
  id: '76599'
  last_name: Bengs
- first_name: Matthias
  full_name: Eulert, Matthias
  last_name: Eulert
- first_name: Hajo
  full_name: Holzmann, Hajo
  last_name: Holzmann
citation:
  ama: Bengs V, Eulert M, Holzmann H. Asymptotic confidence sets for the jump curve
    in bivariate regression problems. <i>Journal of Multivariate Analysis</i>. 2019:291-312.
    doi:<a href="https://doi.org/10.1016/j.jmva.2019.02.017">10.1016/j.jmva.2019.02.017</a>
  apa: Bengs, V., Eulert, M., &#38; Holzmann, H. (2019). Asymptotic confidence sets
    for the jump curve in bivariate regression problems. <i>Journal of Multivariate
    Analysis</i>, 291–312. <a href="https://doi.org/10.1016/j.jmva.2019.02.017">https://doi.org/10.1016/j.jmva.2019.02.017</a>
  bibtex: '@article{Bengs_Eulert_Holzmann_2019, title={Asymptotic confidence sets
    for the jump curve in bivariate regression problems}, DOI={<a href="https://doi.org/10.1016/j.jmva.2019.02.017">10.1016/j.jmva.2019.02.017</a>},
    journal={Journal of Multivariate Analysis}, author={Bengs, Viktor and Eulert,
    Matthias and Holzmann, Hajo}, year={2019}, pages={291–312} }'
  chicago: Bengs, Viktor, Matthias Eulert, and Hajo Holzmann. “Asymptotic Confidence
    Sets for the Jump Curve in Bivariate Regression Problems.” <i>Journal of Multivariate
    Analysis</i>, 2019, 291–312. <a href="https://doi.org/10.1016/j.jmva.2019.02.017">https://doi.org/10.1016/j.jmva.2019.02.017</a>.
  ieee: V. Bengs, M. Eulert, and H. Holzmann, “Asymptotic confidence sets for the
    jump curve in bivariate regression problems,” <i>Journal of Multivariate Analysis</i>,
    pp. 291–312, 2019.
  mla: Bengs, Viktor, et al. “Asymptotic Confidence Sets for the Jump Curve in Bivariate
    Regression Problems.” <i>Journal of Multivariate Analysis</i>, 2019, pp. 291–312,
    doi:<a href="https://doi.org/10.1016/j.jmva.2019.02.017">10.1016/j.jmva.2019.02.017</a>.
  short: V. Bengs, M. Eulert, H. Holzmann, Journal of Multivariate Analysis (2019)
    291–312.
date_created: 2019-10-30T14:22:57Z
date_updated: 2022-01-06T06:51:52Z
department:
- _id: '34'
- _id: '355'
doi: 10.1016/j.jmva.2019.02.017
language:
- iso: eng
page: 291-312
publication: Journal of Multivariate Analysis
publication_identifier:
  issn:
  - 0047-259X
publication_status: published
status: public
title: Asymptotic confidence sets for the jump curve in bivariate regression problems
type: journal_article
user_id: '76599'
year: '2019'
...
---
_id: '14028'
author:
- first_name: Viktor
  full_name: Bengs, Viktor
  id: '76599'
  last_name: Bengs
- first_name: Hajo
  full_name: Holzmann, Hajo
  last_name: Holzmann
citation:
  ama: Bengs V, Holzmann H. Adaptive confidence sets for kink estimation. <i>Electronic
    Journal of Statistics</i>. 2019:1523-1579. doi:<a href="https://doi.org/10.1214/19-ejs1555">10.1214/19-ejs1555</a>
  apa: Bengs, V., &#38; Holzmann, H. (2019). Adaptive confidence sets for kink estimation.
    <i>Electronic Journal of Statistics</i>, 1523–1579. <a href="https://doi.org/10.1214/19-ejs1555">https://doi.org/10.1214/19-ejs1555</a>
  bibtex: '@article{Bengs_Holzmann_2019, title={Adaptive confidence sets for kink
    estimation}, DOI={<a href="https://doi.org/10.1214/19-ejs1555">10.1214/19-ejs1555</a>},
    journal={Electronic Journal of Statistics}, author={Bengs, Viktor and Holzmann,
    Hajo}, year={2019}, pages={1523–1579} }'
  chicago: Bengs, Viktor, and Hajo Holzmann. “Adaptive Confidence Sets for Kink Estimation.”
    <i>Electronic Journal of Statistics</i>, 2019, 1523–79. <a href="https://doi.org/10.1214/19-ejs1555">https://doi.org/10.1214/19-ejs1555</a>.
  ieee: V. Bengs and H. Holzmann, “Adaptive confidence sets for kink estimation,”
    <i>Electronic Journal of Statistics</i>, pp. 1523–1579, 2019.
  mla: Bengs, Viktor, and Hajo Holzmann. “Adaptive Confidence Sets for Kink Estimation.”
    <i>Electronic Journal of Statistics</i>, 2019, pp. 1523–79, doi:<a href="https://doi.org/10.1214/19-ejs1555">10.1214/19-ejs1555</a>.
  short: V. Bengs, H. Holzmann, Electronic Journal of Statistics (2019) 1523–1579.
date_created: 2019-10-30T14:25:16Z
date_updated: 2022-01-06T06:51:52Z
department:
- _id: '34'
- _id: '355'
doi: 10.1214/19-ejs1555
language:
- iso: eng
page: 1523-1579
publication: Electronic Journal of Statistics
publication_identifier:
  issn:
  - 1935-7524
publication_status: published
status: public
title: Adaptive confidence sets for kink estimation
type: journal_article
user_id: '76599'
year: '2019'
...
---
_id: '14031'
author:
- first_name: Viktor
  full_name: Bengs, Viktor
  id: '76599'
  last_name: Bengs
citation:
  ama: Bengs V. <i>Confidence Sets for Change-Point Problems in Nonparametric Regression
    </i>.; 2018. doi:<a href="https://doi.org/10.17192/z2018.0511">https://doi.org/10.17192/z2018.0511</a>
  apa: Bengs, V. (2018). <i>Confidence sets for change-point problems in nonparametric
    regression </i>. <a href="https://doi.org/10.17192/z2018.0511">https://doi.org/10.17192/z2018.0511</a>
  bibtex: '@book{Bengs_2018, title={Confidence sets for change-point problems in nonparametric
    regression }, DOI={<a href="https://doi.org/10.17192/z2018.0511">https://doi.org/10.17192/z2018.0511</a>},
    author={Bengs, Viktor}, year={2018} }'
  chicago: Bengs, Viktor. <i>Confidence Sets for Change-Point Problems in Nonparametric
    Regression </i>, 2018. <a href="https://doi.org/10.17192/z2018.0511">https://doi.org/10.17192/z2018.0511</a>.
  ieee: V. Bengs, <i>Confidence sets for change-point problems in nonparametric regression
    </i>. 2018.
  mla: Bengs, Viktor. <i>Confidence Sets for Change-Point Problems in Nonparametric
    Regression </i>. 2018, doi:<a href="https://doi.org/10.17192/z2018.0511">https://doi.org/10.17192/z2018.0511</a>.
  short: V. Bengs, Confidence Sets for Change-Point Problems in Nonparametric Regression
    , 2018.
date_created: 2019-10-30T14:27:45Z
date_updated: 2022-01-06T06:51:52Z
doi: https://doi.org/10.17192/z2018.0511
language:
- iso: eng
status: public
title: 'Confidence sets for change-point problems in nonparametric regression '
type: dissertation
user_id: '76599'
year: '2018'
...