Alex Damian – Foundations of Deep Learning: Optimization and Representation Learning

Abstract: 

Deep learning’s success stems from the ability of neural networks to  
automatically discover meaningful representations from raw data. In this  
talk, I will describe some recent insights into how optimization enables this  
learning process. First, I will explore how gradient descent enables neural  
networks to adapt to low-dimensional structure in the data, and how these  
ideas extend to  understanding in-context learning in transformers. I will  
then discuss my work toward a predictive theory of deep learning optimization  
that characterizes how different optimizers navigate deep learning loss  
landscapes and how these different behaviors affect training efficiency,  
stability, and generalization.
 
Bio: 

Alex Damian is a fifth-year Ph.D. student in the Program for Applied and  
Computational Mathematics (PACM) at Princeton University, advised by Jason  
Lee. His research is focused on deep learning theory with an emphasis on  
optimization and representation learning. His work has been supported by an  
NSF Graduate Research Fellowship and a Jane Street Graduate Research  
Fellowship.

Details

• Date: Thursday, March 20
• Time: 11:00 am - 12:00 pm
• Category: EECS Seminar
• Location: 34-401 Grier A

Host

• Piotr Indyk