---
_id: '46649'
abstract:
- lang: eng
  text: "Different conflicting optimization criteria arise naturally in various Deep\r\nLearning
    scenarios. These can address different main tasks (i.e., in the\r\nsetting of
    Multi-Task Learning), but also main and secondary tasks such as loss\r\nminimization
    versus sparsity. The usual approach is a simple weighting of the\r\ncriteria,
    which formally only works in the convex setting. In this paper, we\r\npresent
    a Multi-Objective Optimization algorithm using a modified Weighted\r\nChebyshev
    scalarization for training Deep Neural Networks (DNNs) with respect\r\nto several
    tasks. By employing this scalarization technique, the algorithm can\r\nidentify
    all optimal solutions of the original problem while reducing its\r\ncomplexity
    to a sequence of single-objective problems. The simplified problems\r\nare then
    solved using an Augmented Lagrangian method, enabling the use of\r\npopular optimization
    techniques such as Adam and Stochastic Gradient Descent,\r\nwhile efficaciously
    handling constraints. Our work aims to address the\r\n(economical and also ecological)
    sustainability issue of DNN models, with a\r\nparticular focus on Deep Multi-Task
    models, which are typically designed with a\r\nvery large number of weights to
    perform equally well on multiple tasks. Through\r\nexperiments conducted on two
    Machine Learning datasets, we demonstrate the\r\npossibility of adaptively sparsifying
    the model during training without\r\nsignificantly impacting its performance,
    if we are willing to apply\r\ntask-specific adaptations to the network weights.
    Code is available at\r\nhttps://github.com/salomonhotegni/MDMTN."
author:
- first_name: Sedjro Salomon
  full_name: Hotegni, Sedjro Salomon
  id: '97995'
  last_name: Hotegni
- first_name: Manuel Bastian
  full_name: Berkemeier, Manuel Bastian
  id: '51701'
  last_name: Berkemeier
- first_name: Sebastian
  full_name: Peitz, Sebastian
  id: '47427'
  last_name: Peitz
  orcid: 0000-0002-3389-793X
citation:
  ama: 'Hotegni SS, Berkemeier MB, Peitz S. Multi-Objective Optimization for Sparse
    Deep Multi-Task Learning. In: <i>2024 International Joint Conference on Neural
    Networks (IJCNN)</i>. IEEE; 2024:9. doi:<a href="https://doi.org/10.1109/IJCNN60899.2024.10650994">10.1109/IJCNN60899.2024.10650994</a>'
  apa: Hotegni, S. S., Berkemeier, M. B., &#38; Peitz, S. (2024). Multi-Objective
    Optimization for Sparse Deep Multi-Task Learning. <i>2024 International Joint
    Conference on Neural Networks (IJCNN)</i>, 9. <a href="https://doi.org/10.1109/IJCNN60899.2024.10650994">https://doi.org/10.1109/IJCNN60899.2024.10650994</a>
  bibtex: '@inproceedings{Hotegni_Berkemeier_Peitz_2024, place={Yokohama, Japan},
    title={Multi-Objective Optimization for Sparse Deep Multi-Task Learning}, DOI={<a
    href="https://doi.org/10.1109/IJCNN60899.2024.10650994">10.1109/IJCNN60899.2024.10650994</a>},
    booktitle={2024 International Joint Conference on Neural Networks (IJCNN)}, publisher={IEEE},
    author={Hotegni, Sedjro Salomon and Berkemeier, Manuel Bastian and Peitz, Sebastian},
    year={2024}, pages={9} }'
  chicago: 'Hotegni, Sedjro Salomon, Manuel Bastian Berkemeier, and Sebastian Peitz.
    “Multi-Objective Optimization for Sparse Deep Multi-Task Learning.” In <i>2024
    International Joint Conference on Neural Networks (IJCNN)</i>, 9. Yokohama, Japan:
    IEEE, 2024. <a href="https://doi.org/10.1109/IJCNN60899.2024.10650994">https://doi.org/10.1109/IJCNN60899.2024.10650994</a>.'
  ieee: 'S. S. Hotegni, M. B. Berkemeier, and S. Peitz, “Multi-Objective Optimization
    for Sparse Deep Multi-Task Learning,” in <i>2024 International Joint Conference
    on Neural Networks (IJCNN)</i>, Yokohama, Japan, 2024, p. 9, doi: <a href="https://doi.org/10.1109/IJCNN60899.2024.10650994">10.1109/IJCNN60899.2024.10650994</a>.'
  mla: Hotegni, Sedjro Salomon, et al. “Multi-Objective Optimization for Sparse Deep
    Multi-Task Learning.” <i>2024 International Joint Conference on Neural Networks
    (IJCNN)</i>, IEEE, 2024, p. 9, doi:<a href="https://doi.org/10.1109/IJCNN60899.2024.10650994">10.1109/IJCNN60899.2024.10650994</a>.
  short: 'S.S. Hotegni, M.B. Berkemeier, S. Peitz, in: 2024 International Joint Conference
    on Neural Networks (IJCNN), IEEE, Yokohama, Japan, 2024, p. 9.'
conference:
  end_date: 2024-07-05
  location: Yokohama, Japan
  name: 2024 International Joint Conference on Neural Networks (IJCNN)
  start_date: 2024-06-30
date_created: 2023-08-24T07:44:36Z
date_updated: 2024-09-27T10:24:22Z
department:
- _id: '655'
doi: 10.1109/IJCNN60899.2024.10650994
has_accepted_license: '1'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://ieeexplore.ieee.org/document/10650994
oa: '1'
page: '9'
place: Yokohama, Japan
publication: 2024 International Joint Conference on Neural Networks (IJCNN)
publication_identifier:
  eisbn:
  - 979-8-3503-5931-2
  eissn:
  - ' 2161-4407'
publication_status: published
publisher: IEEE
status: public
title: Multi-Objective Optimization for Sparse Deep Multi-Task Learning
type: conference
user_id: '97995'
year: '2024'
...
---
_id: '45695'
author:
- first_name: Sedjro Salomon
  full_name: Hotegni, Sedjro Salomon
  id: '97995'
  last_name: Hotegni
- first_name: Sepideh
  full_name: Mahabadi, Sepideh
  last_name: Mahabadi
- first_name: Ali
  full_name: Vakilian, Ali
  last_name: Vakilian
citation:
  ama: 'Hotegni SS, Mahabadi S, Vakilian A. Approximation Algorithms for Fair Range
    Clustering. In: <i>Proceedings of the 40th International Conference on Machine
    Learning, Honolulu, Hawaii, USA. PMLR 202, 2023.</i>'
  apa: Hotegni, S. S., Mahabadi, S., &#38; Vakilian, A. (n.d.). Approximation Algorithms
    for Fair Range Clustering. <i>Proceedings of the 40th International Conference
    on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 2023.</i> International
    Conference on Machine Learning, Honolulu, Hawaii, USA.
  bibtex: '@inproceedings{Hotegni_Mahabadi_Vakilian, title={Approximation Algorithms
    for Fair Range Clustering}, booktitle={Proceedings of the 40th International Conference
    on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 2023.}, author={Hotegni,
    Sedjro Salomon and Mahabadi, Sepideh and Vakilian, Ali} }'
  chicago: Hotegni, Sedjro Salomon, Sepideh Mahabadi, and Ali Vakilian. “Approximation
    Algorithms for Fair Range Clustering.” In <i>Proceedings of the 40th International
    Conference on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 2023.</i>, n.d.
  ieee: S. S. Hotegni, S. Mahabadi, and A. Vakilian, “Approximation Algorithms for
    Fair Range Clustering,” presented at the International Conference on Machine Learning,
    Honolulu, Hawaii, USA.
  mla: Hotegni, Sedjro Salomon, et al. “Approximation Algorithms for Fair Range Clustering.”
    <i>Proceedings of the 40th International Conference on Machine Learning, Honolulu,
    Hawaii, USA. PMLR 202, 2023.</i>
  short: 'S.S. Hotegni, S. Mahabadi, A. Vakilian, in: Proceedings of the 40th International
    Conference on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 2023., n.d.'
conference:
  end_date: 2023-07-29
  location: Honolulu, Hawaii, USA
  name: International Conference on Machine Learning
  start_date: 2023-07-23
date_created: 2023-06-20T22:29:33Z
date_updated: 2023-06-20T23:03:12Z
department:
- _id: '655'
keyword:
- Fair range clustering
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://openreview.net/forum?id=gBoKJT5JhM
oa: '1'
publication: Proceedings of the 40th International Conference on Machine Learning,
  Honolulu, Hawaii, USA. PMLR 202, 2023.
publication_status: accepted
status: public
title: Approximation Algorithms for Fair Range Clustering
type: conference
user_id: '97995'
year: '2023'
...