EECS Special Seminar: Gokul Swamy, “Efficient Interactive Learning: Learning More From Less”

Abstract
Even as we near the limits of what human-generated data we can scrape from the Internet, today’s decision-making agents — from robots to large language models (LLMs) — are still far from perfect. Thus, as we start to move past the era of simply scaling up training datasets, I believe the most pressing question in decision-making is /how we can learn more from less data/. 

In theory, agents collecting their own data and learning from this experience might allow us to transcend the limits of static datasets. However, there are two core challenges that make delivering on this promise of reinforcement learning (RL) practically challenging. The first is /exploration/: experiencing the right things. The second is /specification/: knowing if what you experienced was good or bad. My research focuses on algorithms that address both of these challenges in tandem.

This talk will cover both theoretical advancements and their practical implications. First, I will discuss how we can teach robots to provably recover from their own mistakes without extensive trial-and-error exploration. Second, I will describe provably robust algorithms for training language models from conflicting preferences. Third, I will explain how RL learns more from less without having to create data /ex nihilo/. To conclude, I will outline what I believe are the most promising directions to enable the next generation of agents to learn /even more/ from less.

Bio
Gokul is a final-year PhD student in the Robotics Institute at Carnegie Mellon University. He works on efficient interactive learning algorithms for training agents like robots and language models. More fundamentally, he is interested in techniques for learning to make good decisions efficiently, even when “good” is hard to specify. Gokul was named a Rising Star in Data Science and Robotics and received an Outstanding Teaching Award for his student mentorship and creation of a new reinforcement learning course. He has also spent summers at Microsoft Research and Google Research.

Details

• Date: Monday, March 30
• Time: 10:00 am - 11:00 am
• Location: Star 32-D463

Host

• Leslie P Kaelbling
• Email: fern@csail.mit.edu