Date: July 11, 2022
Time: 10:00 – 11:30 am ET
Location: Groseclose 402
Zoom link: https://gatech.zoom.us/j/2574295502
Minshuo Chen
Machine Learning PhD Student
School of Industrial and Systems Engineering
Georgia Institute of Technology
Committee
1 Dr. Tuo Zhao (Advisor, School of Industrial and Systems Engineering, Gatech)
2 Dr. Wenjing Liao (Co-advisor, School of Mathematics, Gatech)
3 Dr. Alexander Shapiro (School of Industrial and Systems Engineering, Gatech)
4 Dr. Yajun Mei (School of Industrial and Systems Engineering, Gatech)
5 Dr. Hongyuan Zha (School of Data Science, CUHKSZ)
Abstract
Significant success of deep learning has brought unprecedented challenges to conventional wisdom in statistics, optimization, and applied mathematics. In many high-dimensional applications, e.g., image data of hundreds of thousands of pixels, deep learning is remarkably scalable and mysteriously generalizes well. Although such appealing behavior stimulates wide applications, a fundamental theoretical challenge -- curse of data dimensionality -- naturally arises. Roughly put, the sample complexity in practical applications is significantly smaller than that predicted by theory. It is a common belief that deep neural networks are good at learning various geometric structures hidden in data sets. However, little theory has been established to explain such a power. This thesis aims to bridge the gap between theory and practice by studying function approximation and statistical theories of deep neural networks in exploitation of geometric structures in data.
Function Approximation Theories on Low-dimensional Manifolds using Deep Neural Networks
We first develop an efficient universal approximation theory functions on a low-dimensional Riemannian manifold. A feedforward network architecture is constructed for function approximation, where the size of the network grows depending on the manifold dimension. Furthermore, we prove efficient approximation theory for convolutional residual networks in approximating Besov functions. Lastly, we demonstrate the benefit of overparameterized neural networks in function approximation. Specifically, we show that large neural networks are capable of accurately approximating a target function, and the network itself enjoys Lipschitz continuity.
Statistical Theories on Low-dimensional Data using Deep Neural Networks
Efficient approximation theories of neural networks provide valuable guidelines to properly choose network architectures, when data exhibit geometric structures. In combination with statistical tools, we prove that neural networks can circumvent the curse of data dimensionality and enjoy fast statistical convergence in various learning problems, including nonparametric regression/classification, generative distribution estimation, and doubly-robust policy learning.
