MIT Department of Electrical Engineering & Computer Science

A Geometric Viewpoint to Computer Vision and Vision Based Control

Dr. Yi Ma
University of California at Berkeley

Thursday, April 6, 2000
3:15 PM (refreshments 3:00)
36-156
EECS Special Seminar

Abstract

As an important feature of any autonomous mobile agent, such as the human or unmanned (ground and aerial) vehicles, there is usually a vision system embedded within the decision making loop. The role of the vision system, whether biological or artificial, is responsible for retrieving 3D information of the environment from 2D images. Such 3D information contributes to either low-level feedback control so as to safely navigate within and interact with the surroundings, or high-level decision making so as to reliably recognize, evade, pursue or manipulate 3D objects or coordinate with other agents.

Among all the cues available for computing 3D information, the motion cue (also called the stereo, parallax or structure from motion cues) provides the most unequivocal information about the camera motion, calibration and 3D structure. Thus the study of the motion cue has been the subject of intense research in the computer vision community. The majority of the results have been established primarily within a Projective Geometric framework which is not easily exploited by the control and robotics community.

In this talk, we show how to further use a blend of novel techniques in Differential Geometry, Estimation Theory, Optimization and Dynamic Systems to improve our understanding of the basic geometric laws which govern the visual perception. This new perspective has initiated a series of new developments in almost every classic problem associated to the motion cue. We demonstrate this through a coherent set of topics: 1. A continuous version of the standard 8 point linear two-view motion estimation algorithm which estimates camera velocity from optical flows. 2. Multiview motion estimation based on a nonlinear quadratic regulation (NLQR) of the reprojection error, which results in a statistical normalization of the well-known multilinear constraints. 3. A systematic study of the properties of Kruppa's equations that leads to a deeper understanding of both geometric and algorithmic aspects of camera self-calibration. Such a study also reveals difficulty with degeneracy in conventional calibration methods and shows how they can be salvaged.

The proposed common mathematical framework between computer vision and control theory and robotics enables a better formulation of vision based control. If time allows, I will also mention some basic approaches to a theory of vision based control, through a brief introduction to two vision based control projects: vision based navigation of an unmanned ground vehicle and vision based landing of an unmanned aerial vehicle.