Doctoral Thesis: From Visual Computing to Physical AI: Simulation, Optimization, and Learning for Physical World

Presenter: Yifei Li
Presenter’s Affiliation: CSAIL
Thesis Supervisor(s): Wojciech Matusik, William T. Freeman, Jonathan Ragan-Kelley

Details

• Date: Monday, May 11
• Time: 10:00 am - 12:00 pm
• Location: 32-G882

Additional Location Details:

Abstract:
In traditional graphics, simulation success is defined by visual plausibility. If a simulation looks correct to human eyes, the task is complete. However, as simulation moves from digital content creation to domains like robotics, physical design, and engineering, the goal shifts to physical consequence, where simulation errors lead to real-world failure. High-fidelity simulators exist for these domains, but they act as forward-only oracles that lack the gradients necessary for deep physical reasoning and optimization. Conversely, while pure machine learning models offer fast reasoning and data adaptability, they often lack the physical consistency required for high-stakes deployment.
In this talk, I will present a framework for Physical AI that bridges these two paradigms. I will first introduce differentiable simulation engines that transform physics into usable gradients for optimization, enabling the unified co-design of form and control. I will demonstrate this on diverse applications including the inverse design of fluidic devices and artificial heart, policy learning for assistive dressing robots, and the creation of simulation-ready digital twins from 3D scans. Finally, I will discuss how to move beyond idealized modeling equations using Neural Modular Learning, enabling systems to adapt to complex, real-world dynamics from measurements during operation.