MIT Electrical Engineering and Computer Science EECS Event

Counting probability distributions: The geometry of parsimonious inference

Prof. Vijay Balasubramanian
University of Pennsylvania

Tuesday, October 24, 2000
4:00 PM (reception following)
Room 35-225
LIDS Colloquium

Abstract

William of Ockham's famous principle of parsimony states that explanations should be as simple as possible, given observed data. Most well-founded theories of statistical inference automatically embody Ockham's Razor, penalizing complex models until the data justifies their selection. I will discuss how such complexity penalties arise in both Bayesian and Minimum Description Length approaches to inference. In both cases, the complexity of a statistical model can be understood most intuitively by considering the geometry of the model seen as a manifold embedded in the space of probability distributions.