16/02/2021 – 14h00-15h00 – Online
A mathematical model for nonsmooth algorithmic differentiation with applications to machine learning.
Abstract. We are interested in nonsmooth analysis of algorithmic differentiation, a central building block of the learning phase implemented in modern deep learning software librairies, such as Tensorflow or Pytorch.
First, I will illustrate how blind application of differential calculus to nonsmooth objects can be problematic, requiring a proper mathematical model. Then I will introduce a weak notion of generalized derivative, named conservativity, and illustrate how it complies with calculus and optimization for well structured objects. We provide stability results for empirical risk minimization similar as in the smooth setting for the combination of nonsmooth automatic differentiation, minibatch stochastic approximation and first order optimization.
This is joint work with Jérôme Bolte.
Biography. Edouard Pauwels is assistant professor in Toulouse 3 Paul Sabatier university, working between the Informatics and Mathematics institutes. Edouard received his PhD in November 2013 at Center for computational biology, Mines ParisTech, under the supervision of professor Veronique Stoven. From January to September 2014, he was a postdoc in MAC team at LAAS-CNRS. Between October 2014 and July 2015, he did Postdoc at the Technion, Israel.
Practical details. The seminar will take place online, using Microsoft Teams. If you want to attend this seminar (or any of the forthcoming online Automatiques seminars), and if you do not already have access to the AutoSeminar group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account). You will find the link to the seminar on the « General » AutoSeminar channel on Teams, approximately 15 minutes before the beginning.
The technical side of things: you can use Teams either directly from you web browser or using the « fat client », which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.