Machine learning has been tremendously successful in the past decade. In this presentation, we introduce guidance and insights from information theory to practical machine learning algorithms.
In particular, we study three application domains and demonstrate the algorithmic gain of integrating machine learning with information theory. In the first part of the talk, we show how to deploy the principle of network coding to propose a decomposition scheme for distributing a neural network over a physical communication network. We will show through experiments that our proposed scheme dramatically reduces the energy used compared to existing communication schemes under various channel statistics and network topologies. In the second part, we will present a novel coding design in bio-molecular profiling. We propose a learning-based coding scheme, by modifying error correction codes in an application specific manner. Our scheme significantly outperforms existing schemes in reducing the false negative rate of rare bio-molecular types. In the third part, we will exercise guesswork on the machine translation problem. We study machine translation using seq2seq model. We propose a framework based on guesswork to quantify the difficulty and uncertainty in such models and to shed light on practical designs.
Thesis Supervisor: Prof Muriel Medard
To attend this defense, please contact the contact the doctoral candidate at litianl at mit dot edu