Doctoral Thesis: Representation Learning Through the Lens of Science: Symmetry, Language and Symbolic Inductive Biases

Rumen Dangovski

Abstract:

In this thesis, we explore representation learning, a key technique in machine learning and artificial intelligence that has led to remarkable advancements in fields such as speech, vision, language perception and generation, as well as solving complex scientific problems like protein folding. Despite its success, the prevailing method of representation learning, end-to-end supervised learning, faces challenges including the need for large datasets, overconfident and non-interpretable classifications, and difficulties in transferring representations.

To address these limitations, we adopt a scientific perspective, focusing on machine learning tasks that are particularly affected by these issues, and developing benchmarks inspired by scientific principles. Our approach centers on the identification and development of novel inductive biases (assumptions made by the learning algorithm to improve generalization) based on symmetry, language, and symbolic properties. These inductive biases prove beneficial for both solving scientific problems using machine learning and enhancing representation learning methods.

We term this methodology “Representation Learning Through the Lens of Science” and demonstrate its effectiveness in various applications. Finally, we discuss the limitations of our approach and propose directions for future research.

Thesis Supervisor: Professor Marin Soljačić

Details

• Date: Wednesday, August 30
• Time: 11:00 am - 12:30 pm
• Category: Thesis Defense
• Location: 4-331 (Duboc Room)

Additional Location Details:

Zoom Link available upon request.