MIT Department of Electrical Engineering & Computer Science

Layered Learning in Multi-Agent Systems

Peter Stone
Computer Science Dept., Carnegie Mellon University

Monday, April 13, 1998
4:00 PM (refreshments 3:45)
Room NE43-518
EECS Special Seminar

Abstract

Multi-agent systems in complex, real-time domains require agents to act effectively both autonomously and as part of a team. Because of the inherent complexity of most multi-agent systems, Machine Learning is an interesting and promising area to merge with Multi-Agent Systems. In this talk, I present "layered learning," a new general multi-agent learning paradigm. Layered learning allows for a bottom-up definition of agent capabilities at different levels in a complete multi-agent domain. Individual and collaborative behaviors in the presence of adversaries are organized, learned, and combined in a layered fashion: learned, robust, individual behaviors facilitate team learning of collaborative behaviors, which in turn serve as the basis for strategic, adversarial learning.

Empirical results validate the layered learning approach. We apply layered learning in simulated robotic soccer, an exciting new domain that embodies many challenging complexities: multiple agents must act autonomously while working towards a team goal; they must act in real time; they must deal with noisy sensors and actuators; and they must handle adversarial situations. In this domain, our implemented autonomous learned behaviors generalize from sparse training data via action-dependent features that reduce the dimensionality of the state-space. Our flexible teamwork structure decomposes the task-space among multiple collaborative agents for team learning.

At a recent simulator competition, our team---CMUnited-97---made it to the semi-finals out of 29 teams. The teamwork structure was also implemented on our real robot team, which won the small robot league. Some video footage of our successful implementations are presented.