Mobile-manipulation robots performing service tasks in human-centric indoor environments need to know about relevant aspects of their spatial surroundings. However, service robots rarely know the exact state of the world, unlike industrial robots in structured environments. Additionally, as the world is shared with humans, uncertainty in the complete state of the world is inevitable over time. Mobile-manipulation robots therefore need to continuously perform state estimation, using perceptual information to maintain a representation of the state, and its uncertainty, of task-relevant aspects of the world. Because indoor tasks frequently require interacting with objects, objects should be given critical emphasis in spatial representations for service robots. In my dissertation, I propose a world model based on objects, their semantic attributes (task-relevant properties such as type and pose), and their geometric realizations in the physical world.
Objects are challenging to keep track of because there is significant uncertainty in their states. Object detection and recognition using robotic vision is still error-prone. Objects can also be inherently ambiguous because they have similar attributes. Besides detection noise, other agents may change the state of the world. Compounded over multitudes of objects and long temporal horizons, the above sources of uncertainty give rise to a challenging estimation problem. Fortunately, most objects do not change quickly, and sensing is relatively cheap, so we can leverage information from multiple diverse snapshots of similar world states. However, putting the information together introduces a data association problem, which I tackle with constrained Bayesian nonparametric models. By carefully aggregating information across different viewpoints, times, and sensors, I show that robots can reduce their uncertainty in the state of the world and maintain more accurate object-based world models.
Supervisors: Leslie Pack Kaelbling, Tomás Lozano-Pérez