Nonsmooth implicit differentiation for optimization

Title

Nonsmooth implicit differentiation for optimization

Speaker

Edouard Pauwels - Toulouse 3 Paul Sabatier

Abstract

Our motivating problem is that of bilevel optimization problems arising from hyperparameter tuning of nonsmooth penalized models, the typical example being lasso. This reduces to differentiation of optimality conditions expressed in the form of a Lipschitz fixed point equation, a problem which can be adressed with the implicit function theorem. The classical implicit function theorem has two parts; the first deals with uniqueness and regularity of the implicit function while the second provides an implicit differentiation calculus. We focus on the second part, illustrating the fact that nonsmooth implicit differentiation fails with Clarke Jacobians and proposing a solution using Conservative Jacobians. Conservative Jacobians have been introduced recently as tools for nonsmooth calculus, compatible with the compositional rules of differential calculus while preserving minimizing behavior of gradient type algorithms. We describe how these objects can be used to resolve the issues of nonsmooth implicit differentiation and how they extend the domain of validity of the rules of differential calculus in nonsmooth analysis. These results will be illustrated on the motivating problem of hyperparameter for penalized models.

Bio

Edouard Pauwels is assistant professor in Toulouse 3 Paul Sabatier university, working between the Informatics and Mathematics institutes. Edouard received his PhD in November 2013 at Center for computational biology, Mines ParisTech, under the supervision of professor Veronique Stoven. From January to September 2014, he was a postdoc in MAC team at LAAS-CNRS. Between October 2014 and July 2015, he did Postdoc at the Technion, Israel. His research is broadly centered on optimization and applications in machine learning, with a recent focus on nonsmooth analytic and algorithmic developments as well as nonsmooth differential calculus.

When

May 2nd 2022, 15:00

Where

Room 508, UniGe DIMA, Via Dodecaneso 35, Genova, Italy.

Streaming will be available at the link below.