Doctoral Thesis: Compositional Robot Learning for Generalizable Interactions

Yen-Ling Kuo

Abstract:

To understand environments effectively and to interact safely with humans, robots must generalize their learned models to scenarios they have never been trained on before, such as new commands and new agents. Humans have shown a remarkable ability to compose concepts they have learned before in order to interpret and to act in a novel environment. In contrast, many deep-learning based methods fail at compositional generalization, i.e., an ability to generalize to novel combinations of concepts that have not been seen before in training.

In this talk, I will present several learning-based approaches that leverage compositionality to enable generalization in various reasoning skills. First, I will show how compositional linguistic structure can be incorporated into robotic models to enable robots to follow novel commands and act rationally in new scenarios. Then I will show how recursive reward estimation can enable robots to reason about sequences of actions and about novel social interactions. Finally, I will show how we can incorporate compositionality into trajectory prediction by using language as an intermediate representation.

Details

• Date: Friday, May 6
• Time: 3:00 pm
• Location: 32-882 (Hewlett Room)

Additional Location Details:

Thesis Supervisors: Drs. Boris Katz, Andrei Barbu