Approximation Algorithms for Fair Range Clustering - Hotegni, Sedjro Salomon; Mahabadi, Sepideh; Vakilian, Ali (Proceedings of the 40th International Conference on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 2023.)
Keywords: Fair range clustering

Partial observations, coarse graining and equivariance in Koopman operator theory for large-scale dynamical systems - Peitz, Sebastian; Harder, Hans; Nüske, Feliks; Philipp, Friedrich; Schaller, Manuel; Worthmann, Karl (arXiv:2307.15325)
The Koopman operator has become an essential tool for data-driven analysis, prediction and control of complex systems, the main reason being the enormous potential of identifying linear function space representations of nonlinear dynamics from measurements. Until now, the situation where for large-scale systems, we (i) only have access to partial observations (i.e., measurements, as is very common for experimental data) or (ii) deliberately perform coarse graining (for efficiency reasons) has not been treated to its full extent. In this paper, we address the pitfall associated with this situation, that the classical EDMD algorithm does not automatically provide a Koopman operator approximation for the underlying system if we do not carefully select the number of observables. Moreover, we show that symmetries in the system dynamics can be carried over to the Koopman operator, which allows us to massively increase the model efficiency. We also briefly draw a connection to domain decomposition techniques for partial differential equations and present numerical evidence using the Kuramoto--Sivashinsky equation.

Finite-data error bounds for Koopman-based prediction and control - Nüske, Feliks; Peitz, Sebastian; Philipp, Friedrich; Schaller, Manuel; Worthmann, Karl (Journal of Nonlinear Science)
The Koopman operator has become an essential tool for data-driven approximation of dynamical (control) systems in recent years, e.g., via extended dynamic mode decomposition. Despite its popularity, convergence results and, in particular, error bounds are still quite scarce. In this paper, we derive probabilistic bounds for the approximation error and the prediction error depending on the number of training data points; for both ordinary and stochastic differential equations. Moreover, we extend our analysis to nonlinear control-affine systems using either ergodic trajectories or i.i.d. samples. Here, we exploit the linearity of the Koopman generator to obtain a bilinear system and, thus, circumvent the curse of dimensionality since we do not autonomize the system by augmenting the state by the control inputs. To the best of our knowledge, this is the first finite-data error analysis in the stochastic and/or control setting. Finally, we demonstrate the effectiveness of the proposed approach by comparing it with state-of-the-art techniques showing its superiority whenever state and control are coupled.

Efficient time stepping for numerical integration using reinforcement learning - Dellnitz, Michael; Hüllermeier, Eyke; Lücke, Marvin; Ober-Blöbaum, Sina; Offen, Christian; Peitz, Sebastian; Pfannschmidt, Karlson (SIAM Journal on Scientific Computing)
Many problems in science and engineering require an efficient numerical approximation of integrals or solutions to differential equations. For systems with rapidly changing dynamics, an equidistant discretization is often inadvisable as it results in prohibitively large errors or computational effort. To this end, adaptive schemes, such as solvers based on Runge–Kutta pairs, have been developed which adapt the step size based on local error estimations at each step. While the classical schemes apply very generally and are highly efficient on regular systems, they can behave suboptimally when an inefficient step rejection mechanism is triggered by structurally complex systems such as chaotic systems. To overcome these issues, we propose a method to tailor numerical schemes to the problem class at hand. This is achieved by combining simple, classical quadrature rules or ODE solvers with data-driven time-stepping controllers. Compared with learning solution operators to ODEs directly, it generalizes better to unseen initial data as our approach employs classical numerical schemes as base methods. At the same time it can make use of identified structures of a problem class and, therefore, outperforms state-of-the-art adaptive schemes. Several examples demonstrate superior efficiency. Source code is available at https://github.com/lueckem/quadrature-ML.

ElectricGrid.jl - A Julia-based modeling and simulationtool for power electronics-driven electric energy grids - Wallscheid, Oliver; Peitz, Sebastian; Stenner, Jan; Weber, Daniel; Boshoff, Septimus; Meyer, Marvin; Chidananda, Vikas; Schweins, Oliver (Journal of Open Source Software)
Keywords: General Earth and Planetary Sciences; General Environmental Science

Transferability of a discrepancy model for the dynamics of electromagnetic oscillating circuits - Wohlleben, Meike Claudia; Muth, Lars; Peitz, Sebastian; Sextro, Walter (Proceedings in Applied Mathematics and Mechanics)
Modelling of dynamic systems plays an important role in many engineering disciplines. Two different approaches are physical modelling and data‐driven modelling, both of which have their respective advantages and disadvantages. By combining these two approaches, hybrid models can be created in which the respective disadvantages are mitigated, with discrepancy models being a particular subclass. Here, the basic system behaviour is described physically, that is, in the form of differential equations. Inaccuracies resulting from insufficient modelling or numerics lead to a discrepancy between the measurements and the model, which can be compensated by a data‐driven error correction term. Since discrepancy methods still require a large amount of measurement data, this paper investigates the extent to which a single discrepancy model can be trained for a physical model with additional parameter dependencies without the need for retraining. As an example, a damped electromagnetic oscillating circuit is used. The physical model is realised by a differential equation describing the electric current, considering only inductance and capacitance; dissipation due to resistance is neglected. This creates a discrepancy between measurement and model, which is corrected by a data‐driven model. In the experiments, the inductance and the capacity are varied. It is found that the same data‐driven model can only be used if additional parametric dependencies in the data‐driven term are considered as well.
Keywords: Electrical and Electronic Engineering; Atomic and Molecular Physics; and Optics

ROM-Based Multiobjective Optimization of Elliptic PDEs via Numerical Continuation - Banholzer, Stefan; Gebken, Bennet; Dellnitz, Michael; Peitz, Sebastian; Volkwein, Stefan (Non-Smooth and Complementarity-Based Distributed Parameter Systems)
Multiobjective optimization plays an increasingly important role in modern applications, where several objectives are often of equal importance. The task in multiobjective optimization and multiobjective optimal control is therefore to compute the set of optimal compromises (the Pareto set) between the conflicting objectives. Since the Pareto set generally consists of an infinite number of solutions, the computational effort can quickly become challenging which is particularly problematic when the objectives are costly to evaluate as is the case for models governed by partial differential equations (PDEs). To decrease the numerical effort to an affordable amount, surrogate models can be used to replace the expensive PDE evaluations. Existing multiobjective optimization methods using model reduction are limited either to low parameter dimensions or to few (ideally two) objectives. In this article, we present a combination of the reduced basis model reduction method with a continuation approach using inexact gradients. The resulting approach can handle an arbitrary number of objectives while yielding a significant reduction in computing time.

Efficient Virtual Design and Testing of Autonomous Vehicles - Peitz, Sebastian; Dellnitz, Michael; Bannenberg, Sebastian (German Success Stories in Industrial Mathematics)
With the ever increasing capabilities of sensors and controllers, autonomous driving is quickly becoming a reality. This disruptive change in the automotive industry poses major challenges for manufacturers as well as suppliers as entirely new design and testing strategies have to be developed to remain competitive. Most importantly, the complexity of autonomously driving vehicles in a complex, uncertain, and safety-critical environment requires new testing procedures to cover the almost infinite range of potential scenarios.

Koopman analysis of quantum systems - Klus, Stefan; Nüske, Feliks; Peitz, Sebastian (Journal of Physics A: Mathematical and Theoretical)
Koopman operator theory has been successfully applied to problems from various research areas such as fluid dynamics, molecular dynamics, climate science, engineering, and biology. Applications include detecting metastable or coherent sets, coarse-graining, system identification, and control. There is an intricate connection between dynamical systems driven by stochastic differential equations and quantum mechanics. In this paper, we compare the ground-state transformation and Nelson's stochastic mechanics and demonstrate how data-driven methods developed for the approximation of the Koopman operator can be used to analyze quantum physics problems. Moreover, we exploit the relationship between Schrödinger operators and stochastic control problems to show that modern data-driven methods for stochastic control can be used to solve the stationary or imaginary-time Schrödinger equation. Our findings open up a new avenue towards solving Schrödinger's equation using recently developed tools from data science.

Multi-Objective Trust-Region Filter Method for Nonlinear Constraints using Inexact Gradients - Berkemeier, Manuel Bastian; Peitz, Sebastian (arXiv:2208.12094)
In this article, we build on previous work to present an optimization algorithm for nonlinearly constrained multi-objective optimization problems. The algorithm combines a surrogate-assisted derivative-free trust-region approach with the filter method known from single-objective optimization. Instead of the true objective and constraint functions, so-called fully linear models are employed and we show how to deal with the gradient inexactness in the composite step setting, adapted from single-objective optimization as well. Under standard assumptions, we prove convergence of a subset of iterates to a quasi-stationary point and if constraint qualifications hold, then the limit point is also a KKT-point of the multi-objective problem.

On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation - Bieker, Katharina; Gebken, Bennet; Peitz, Sebastian (IEEE Transactions on Pattern Analysis and Machine Intelligence)
We present a novel algorithm that allows us to gain detailed insight into the effects of sparsity in linear and nonlinear optimization, which is of great importance in many scientific areas such as image and signal processing, medical imaging, compressed sensing, and machine learning (e.g., for the training of neural networks). Sparsity is an important feature to ensure robustness against noisy data, but also to find models that are interpretable and easy to analyze due to the small number of relevant terms. It is common practice to enforce sparsity by adding the ℓ1-norm as a weighted penalty term. In order to gain a better understanding and to allow for an informed model selection, we directly solve the corresponding multiobjective optimization problem (MOP) that arises when we minimize the main objective and the ℓ1-norm simultaneously. As this MOP is in general non-convex for nonlinear objectives, the weighting method will fail to provide all optimal compromises. To avoid this issue, we present a continuation method which is specifically tailored to MOPs with two objective functions one of which is the ℓ1-norm. Our method can be seen as a generalization of well-known homotopy methods for linear regression problems to the nonlinear case. Several numerical examples - including neural network training - demonstrate our theoretical findings and the additional insight that can be gained by this multiobjective approach.

Development of a Hybrid Modeling Methodology for Oscillating Systems with Friction - Wohlleben, Meike Claudia; Bender, Amelie; Peitz, Sebastian; Sextro, Walter (Machine Learning, Optimization, and Data Science)

Derivative-Free Multiobjective Trust Region Descent Method Using Radial Basis Function Surrogate Models - Berkemeier, Manuel Bastian; Peitz, Sebastian (Mathematical and Computational Applications)
We present a flexible trust region descend algorithm for unconstrained and convexly constrained multiobjective optimization problems. It is targeted at heterogeneous and expensive problems, i.e., problems that have at least one objective function that is computationally expensive. The method is derivative-free in the sense that neither need derivative information be available for the expensive objectives nor are gradients approximated using repeated function evaluations as is the case in finite-difference methods. Instead, a multiobjective trust region approach is used that works similarly to its well-known scalar pendants. Local surrogate models constructed from evaluation data of the true objective functions are employed to compute possible descent directions. In contrast to existing multiobjective trust region algorithms, these surrogates are not polynomial but carefully constructed radial basis function networks. This has the important advantage that the number of data points scales linearly with the parameter space dimension. The local models qualify as fully linear and the corresponding general scalar framework is adapted for problems with multiple objectives. Convergence to Pareto critical points is proven and numerical examples illustrate our findings.