Doctoral Thesis: Algorithms & Systems for Differentiable Graphics Programming

Talk Abstract:
Differentiable graphics representations are now a center-piece for learning-based approaches in inverse rendering, novel-view synthesis, data & compute efficient rendering, and even 3D generative models. We see great advances in performance & fidelity when mixing classical wisdom from existing primitives like meshes and textures, and novel primitives like tiny neural networks. This cross-pollination of ML & graphics is key to these advances, but is held back due to complications: existing frameworks like PyTorch are ill-suited to graphics programming both due to algorithmic problems, like discontinuities, and system-design problems that lead to poor performance & expressive power. My research attempts to find generalizable solutions to these road-blocks. I will first talk about our effort to tackle discontinuities by automating boundary sampling methods through a custom compiler (Teg). I will then discuss my work on an alternative, more light-weight approach to discontinuity-handling called the warped-area reparameterization method, and how it broadly enables several differentiable renderers. Then, I will talk about SLANG.D, a collaborative effort with NVIDIA to build a high-performance compiler for next-generation differentiable & neural graphics systems. I’ll show how our user-centric focus when building automatic differentiation into the Slang shading language enables users to write large-scale differentiable graphics pipelines without sacrificing performance. Finally, I will highlight the impact of my works on the broader research field, and close with some thoughts on the state-of-the-art.

Bio:
Sai Bangaru is a 5th-year PhD candidate at MIT CSAIL, advised by Prof. Frédo Durand. He specializes in algorithms & systems for differentiable programming, with applications to computer graphics & vision. Sai is supported by an NVIDIA Graduate Fellowship and an MIT Edgerton Fellowship