Gaussian Approximation and Bayesian Posterior Distribution in Random Deep Neural Networks

Title

Gaussian Approximation and Bayesian Posterior Distribution in Random Deep Neural Networks

Speaker

Dario Trevisan - Unversity of Pisa

Abstract

We establish novel rates for the Gaussian approximation of randomly initialized deep neural networks with Gaussian parameters and Lipschitz activation functions, in the so-called wide limit, i.e., where the sizes of all hidden layers become large. Using the Wasserstein metric and related functional analytic tools, we demonstrate the distribution of the output of a network and the corresponding Gaussian approximation are at a distance that scales inversely with the width of the network, surpassing previously established rates.

Furthermore, we extend our findings to approximate the exact Bayesian posterior distribution of the network when the likelihood is a bounded Lipschitz function of the network output, on a finite training set. This includes common cases, such as the Gaussian likelihood, which is defined as the exponential of the negative mean squared error. Our inequalities thus shed light on the network's Gaussian behavior by quantitatively capturing the distributional convergence results in the wide limit.

The exposition will aim to be self-contained, by introducing all the basic concepts related to artificial neural networks and Bayesian statistics to a mathematical audience. Based on arXiv:2203.07379 (joint with A. Basteri) and arXiv:2312.11737.

Bio

Dario Trevisan is currently an Associate Professor at the University of Pisa, specializing in Probability and Mathematical Statistics. His research focuses on various areas, including Optimal Transport and its applications to Combinatorial Optimization, theory of Deep Learning, and Quantum Information theory. Prior to his current position, he obtained his PhD at the Scuola Normale Superiore and held research positions at the University of Pisa. He has an extensive publication record, with over 35 articles published in reputable journals, and he has been a speaker at international conferences and workshops. In recognition of his significant contributions, he was honored with the prestigious Guido Fubini 2021 Award for his exceptional work in the field of "Probability and its role in Analysis and Mathematical Physics".

When

Thursday March 14th, 14:00

Where

Room 704 @ DIBRIS/DIMA, Via Dodecaneso 35, Genoa