Three Mathematical Tales of Machine Learning
Title
Three Mathematical Tales of Machine Learning
Speaker
Massimo Fornasier - Technical University of Munich
Abstract
I will tell three mathematical tales of machine learning related to my most recent work: 1. identification of deep neural networks, 2. global optimization over manifolds, 3. Mean-field optimal control of NeurODE. Tale 1. is about the proof that, despite the NP-hardness of the problem, generic neural networks can be identified up to natural symmetries by a finite number of input-output samples scaling with the complexity of the network. Numerical validation of the result is presented. A crucial subproblem of the identification pipeline is the solution of a nonconvex optimization over the sphere. Tale 2. is in fact about solving global optimizations over spheres by means of a multi-agent dynamics, which combines a consensus mechanism and random exploration. The proof of global solution is based on showing that the large particle limit of the SDE system is distributed as the solution of the deterministic PDE, whose large time asymptotics converges to a global minimizer. I present numerical results in robust linear regression for computing eigenfaces. In the Tale 3. I introduce NeurODE, which are neural networks approximable by ODE. I show that their training can be formulated as a mean-field optimal control and I present the derivation of a mean-field Pontryagin maximum principle characterizing optimal parameters/controls and its well-posedness. Again a numerical experiment of a simple 2D classification problem validates the theoretical results.
Bio
Massimo Fornasier received his doctoral degree in computational mathematics in 2003 from the University of Padua, Italy. After spending from 2003 to 2006 as a postdoctoral research fellow at the University of Vienna and University of Rome La Sapienza, he joined the Johann Radon Institute for Computational and Applied Mathematics (RICAM) of the Austrian Academy of Sciences where he served as a senior research scientist until March 2011. He was an associate researcher from 2006 to 2007 for the Program in Applied and Computational Mathematics of Princeton University, USA. In 2011 Fornasier was appointed Chair of Applied Numerical Analysis at TUM. In 2021 he was awarded by a ERC Starting Grant. The research of Massimo Fornasier embraces a broad spectrum of problems in mathematical modeling, analysis and numerical analysis. He is particularly interested in the concept of compression as appearing in different forms in data analysis, image and signal processing, and in the adaptive numerical solutions of partial differential equations or high-dimensional optimization problems.
When
2021-10-04 at 3:30 pm
Where
Remote, @UniGE