MIT Electrical Engineering and Computer Science EECS Event

Inference for Vision

Bill Freeman
MERL (Mitsubishi Electric Research Labs)

Tuesday, April 24, 2001
4:15 PM (refreshments 4:00)
Room NE43-518
EECS Special Seminar

Abstract

I describe a learning-based method for low-level vision problems--estimating scenes from images. We generate a synthetic world of scenes and their corresponding rendered images, modeling their relationships with a Markov network. Given image data, we seek to infer the most probable scene explanation.

Belief propagation for inference is exact only for networks without loops. We have shown that for Gaussian processes, the estimates are exact, and that belief propagation can only converge to stationary points of a particular approximation to a free energy known as the Bethe approximation. Building on these theoretical results, we use the belief propagation update rules even for this Makov network with many loops, which allows us to efficiently find good estimates for the scene, given an image. We call this approach VISTA--Vision by Image/Scene TrAining.

We apply VISTA to the ``super-resolution'' problem (estimating high frequency details from a low-resolution image), showing state-of-the-art results. To illustrate the potential breadth of the technique, we also apply it for the motion estimation problem in a "blobs world". We show figure/ground discrimination, solution of the aperture problem, and filling-in arising from application of the same probabilistic machinery.