Over-parameterization in (two-layer) neural networks: double descent, function spaces, curse of dimensionality
Fanghui Liu - University of Warwick
The conventional wisdom of simple models in machine learning misses the bigger picture, especially over-parameterized neural networks (NNs), where the number of parameters are much larger than the number of training data. Our goal is to explore the mystery behind over-parameterized models from a theoretical side. In this talk, I will discuss the role of over-parameterization in machine learning from kernel methods to neural networks, to theoretically understand the separation from the perspective of function space. First, I talk about random features models from under-parameterized regime to over-parameterized regime in terms of double descent. Second, I will talk about the relationship between kernel methods and neural networks via random features in certain $F_p$ spaces, which provides a refined analysis of learning in Barron spaces beyond RKHS.
Fanghui Liu is currently an assistant professor at University of Warwick, UK. His research interests focus on kernel methods and learning theory to build the mathematical foundations of machine learning as well as theoretical-oriented applications. For his work on learning theory and cooperation, he was chosen for AAAI New Faculty Highlights 2024, Rising Star in AI (KAUST 2023) and presented two tutorials at ICASSP 2023 and CVPR 2023. Prior to his current position, Fanghui worked as a postdoc researcher at EPFL, Switzerland and KU Leuven, Belgium, respectively. He received his PhD from Shanghai Jiao Tong University, China at 2019.
Thursday February 15th, 3.00pm
Room 322 @ DIBRIS/DIMA, Via Dodecaneso 35, Genoa