On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation

K. Bieker, B. Gebken, S. Peitz, IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).

Download
Restricted On_the_Treatment_of_Optimization_Problems_with_L1_Penalty_Terms_via_Multiobjective_Continuation.pdf 7.99 MB
Journal Article | Epub ahead of print | English
Abstract
We present a novel algorithm that allows us to gain detailed insight into the effects of sparsity in linear and nonlinear optimization, which is of great importance in many scientific areas such as image and signal processing, medical imaging, compressed sensing, and machine learning (e.g., for the training of neural networks). Sparsity is an important feature to ensure robustness against noisy data, but also to find models that are interpretable and easy to analyze due to the small number of relevant terms. It is common practice to enforce sparsity by adding the ℓ1-norm as a weighted penalty term. In order to gain a better understanding and to allow for an informed model selection, we directly solve the corresponding multiobjective optimization problem (MOP) that arises when we minimize the main objective and the ℓ1-norm simultaneously. As this MOP is in general non-convex for nonlinear objectives, the weighting method will fail to provide all optimal compromises. To avoid this issue, we present a continuation method which is specifically tailored to MOPs with two objective functions one of which is the ℓ1-norm. Our method can be seen as a generalization of well-known homotopy methods for linear regression problems to the nonlinear case. Several numerical examples - including neural network training - demonstrate our theoretical findings and the additional insight that can be gained by this multiobjective approach.
Publishing Year
Journal Title
IEEE Transactions on Pattern Analysis and Machine Intelligence
LibreCat-ID

Cite this

Bieker K, Gebken B, Peitz S. On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation. IEEE Transactions on Pattern Analysis and Machine Intelligence. Published online 2021. doi:10.1109/TPAMI.2021.3114962
Bieker, K., Gebken, B., & Peitz, S. (2021). On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2021.3114962
@article{Bieker_Gebken_Peitz_2021, title={On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation}, DOI={10.1109/TPAMI.2021.3114962}, journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, publisher={IEEE}, author={Bieker, Katharina and Gebken, Bennet and Peitz, Sebastian}, year={2021} }
Bieker, Katharina, Bennet Gebken, and Sebastian Peitz. “On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021. https://doi.org/10.1109/TPAMI.2021.3114962.
K. Bieker, B. Gebken, and S. Peitz, “On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, doi: 10.1109/TPAMI.2021.3114962.
Bieker, Katharina, et al. “On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation.” IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE, 2021, doi:10.1109/TPAMI.2021.3114962.
All files available under the following license(s):
Creative Commons Attribution 4.0 International Public License (CC-BY 4.0):
Main File(s)
File Name
On_the_Treatment_of_Optimization_Problems_with_L1_Penalty_Terms_via_Multiobjective_Continuation.pdf 7.99 MB
Access Level
Restricted Closed Access
Last Uploaded
2021-09-25T11:59:15Z


Link(s) to Main File(s)
Access Level
Restricted Closed Access

Export

Marked Publications

Open Data LibreCat

Search this title in

Google Scholar