Doctoral Thesis: Learning inside the prediction function

Ferran Alet

Abstract:

Many of the achievements in machine learning have followed different variations on a single recipe: we pick a supervised training dataset and assume there exists a function mapping inputs to outputs. We then leverage the expressivity of deep learning (together with few but carefully chosen inductive biases for each domain) and train a neural network to approximate this unknown function. In this thesis, we show that this single-function, single-neural-network approach can be too constraining and instead suggest spawning per-point models. This allows us to encode inductive biases in flexible ways and model expressive, structured generative models of the data distribution.

First, we present Tailoring: a novel way of encoding inductive biases by optimizing unsupervised objectives inside the prediction function. This ensures the structure is imposed both at training and test time. Furthermore, its generality allows applications in domains as diverse as physics time-series prediction, adversarial defenses, and contrastive representation learning. We also propose Noether Networks, which automatically discover these inductive biases, in the form of conservation laws. Finally, we propose Functional risk minimization(FRM), an alternative framework to the standard Empirical risk minimization(ERM) setting where loss functions act in function space rather than output space. We show how we can make learning in this new framework efficient and can lead to improved performance compared to the standard ML setting.

Details

• Date: Wednesday, August 3
• Time: 2:00 pm - 3:00 pm
• Category: Thesis Defense
• Location: 32-D463

Additional Location Details:

Thesis Committee: Leslie Pack Kaelbling, Tomas Lozano-Perez, Joshua B. Tenenbaum

Zoom (https://mit.zoom.us/j/98163372734)

Speaker Bio: Ferran Alet is a PhD candidate at MIT CSAIL advised by Leslie Kaelbling, Tomas Lozano-Perez, and Joshua Tenenbaum. His research is on machine learning and leverages techniques from meta-learning, learning to search, program synthesis, and insights from mathematics and the physical sciences. During his PhD, he created the MIT Embodied Intelligence Seminar, mentored 17 students, and won the MIT Outstanding Mentor award 2021. Ferran studied mathematics and physics in Barcelona thanks to CFIS, a program for doing two degrees, where he was the valedictorian of his promotion. In grad school, he earned a “La Caixa” fellowship and was responsible for the high-level planner of the MIT-Princeton team for the Amazon Robotics Challenge, which won the stowing task in 2017. You can find more information and papers at www.alet-et.al

For more information please contact: Ferran Alet, <a href=”mailto:alet@csail.mit.edu”>alet@csail.mit.edu</a>