|
|
MIT Electrical Engineering and Computer Science
EECS Event |
Monday, April 9, 2001
3:00 PM (refreshments 2:45)
Grier Room, Room 34-401A
EECS Special Seminar
Abstract
"A bit is a bit is a bit" seems to have become the motto of our digital age. Though we often take it for granted, the existence of such a common currency for information is far from obvious. The philosophical justification for using bits in this way is the fundamental separation theorem of Information Theory that ties source and channel coding together. This theorem is the keystone of Shannon's program and it tells us that there is no essential loss in treating the problem of communicating information in terms of reliably dealing with bits.
In this talk, I apply Shannon's program to the issue of communicating delay sensitive information streams. To bring matters into sharp focus, I examine the specific problem of communicating the values of a simple scalar discrete time Markov process $X_{t+1} = A X_t + W_t$ through a memoryless noisy channel so as to have finite average squared error from end-to-end. The twist is that $A > 1$ and so the process is decidedly nonstationary as the signal grows with time. Before our work, such processes were considered pathological and mostly beyond the pale of Information Theory since none of the prior approaches could tame their persistent dependencies through time.
On the source side, I give a new coding theorem showing that we can approach the rate-distortion bound for such signals if we are willing to tolerate enough end-to-end delay and use variable length codes. The proof weaves together a lossless code on the integers with a lossy vector quantizer and is able to work for the infinite horizon problem which had remained open till now. On the channel side, I will introduce a new sense of reliable transmission (anytime capacity) that is more demanding than the traditional Shannon sense but is weaker than the sense underlying zero-error capacity. I will then state a new source/channel separation theorem that encompasses unstable processes and establishes that our stronger notion of anytime capacity is essential to be able to deal with such delay sensitive streams.
This work was motivated in part by control settings where noisy channels are encountered in a feedback link from sensing to actuation. It also sheds some light on the concept of "quality of service requirements" and the need for "differentiated service" in communication systems.