Using second-order information in gradient sampling methods for nonsmooth optimization
In this article, we show how second-order derivative information can be
incorporated into gradient sampling methods for nonsmooth optimization. The
second-order information we consider is essentially the set of coefficients of
all second-order Taylor expansions of the objective in a closed ball around a
given point. Based on this concept, we define a model of the objective as the
maximum of these Taylor expansions. Iteratively minimizing this model
(constrained to the closed ball) results in a simple descent method, for which
we prove convergence to minimal points in case the objective is convex. To
obtain an implementable method, we construct an approximation scheme for the
second-order information based on sampling objective values, gradients and
Hessian matrices at finitely many points. Using a set of test problems, we
compare the resulting method to five other available solvers. Considering the
number of function evaluations, the results suggest that the method we propose
is superior to the standard gradient sampling method, and competitive compared
to other methods.