Rahul Parhi – Deep Learning Meets Sparse Regularization

Abstract
Deep learning has been wildly successful in practice and most state-of-the-art artificial intelligence systems are based on neural networks. Lacking, however, is a rigorous mathematical theory that adequately explains the amazing performance of deep neural networks. In this talk, I present a new mathematical framework that provides the beginning of a deeper understanding of deep learning. This framework precisely characterizes the functional properties of trained neural networks. The key mathematical tools which support this framework include transform-domain sparse regularization, the Radon transform of computed tomography, and approximation theory. This framework explains the effect of weight decay regularization in neural network training, the importance of skip connections and low-rank weight matrices in network architectures, the role of sparsity in neural networks, and explains why neural networks can perform well in high-dimensional problems.

Bio
Rahul Parhi is currently a postdoctoral researcher with the Biomedical Imaging Group at the École Polytechnique Fédérale de Lausanne (EPFL). He completed his PhD in electrical engineering at the University of Wisconsin-Madison in 2022, where he was supported by an NSF graduate research fellowship. His research interests lie at the interface between harmonic analysis, signal processing, and data science, in particular, the mathematical aspects of neural networks and deep learning.

Details

• Date: Tuesday, March 5
• Time: 11:00 am - 12:00 pm
• Category: Special Seminar
• Location: Grier A 34-401A

Host

• Lizhong Zheng
• Email: chadcoll@mit.edu