Saikat Dutta “Randomness-Aware Testing of Machine Learning-based Systems”

Abstract:
The goal of my research is to develop novel testing techniques and tools to make Machine Learning-based systems more reliable. Machine Learning is rapidly revolutionizing the way modern-day systems are developed. However, testing Machine Learning-based systems is challenging due to 1) the presence of non-determinism, both internal (e.g., stochastic algorithms) and external (e.g., execution environment), and 2) the absence of well-defined accuracy specifications. Most traditional software testing techniques widely used today cannot tackle these challenges because they often assume determinism and require a precise test oracle.

In this talk, I will present my work on automated testing of Machine Learning-based systems and on improving developer-written tests in such systems. To achieve these goals, I develop principled techniques that build on solid mathematical foundations from probability theory and statistics to reason about the underlying non-determinism and accuracy. I implement my techniques in practical and scalable tools that help developers to detect more bugs and to efficiently navigate trade-offs between test quality and efficiency. To date, my research has exposed more than 50 bugs and improved the quality of more than 200 tests in over 60 popular open-source ML libraries, many of which are widely used at companies like DeepMind, Google, Meta, Microsoft, and Uber as well as in many academic and scientific communities.

Biography:
Saikat Dutta is a PhD candidate in Computer Science at the University of Illinois Urbana-Champaign, advised by Prof. Sasa Misailovic. Saikat’s research interests lie at the intersection of Software Engineering and Machine Learning. Saikat’s current research focuses on improving the reliability of Machine-learning based systems by developing novel testing techniques and tools. Saikat has received the Facebook PhD Fellowship, the 3M Foundation Fellowship, and the Mavis Future Faculty Fellowship for his contributions.

Details

• Date: Thursday, March 23
• Time: 11:00 am - 12:00 pm
• Category: Special Seminar
• Location: 34-401A

Host

• Martin Rinard