Doctoral Thesis: Towards a Deeper Understanding of Neural Language Generation

Tianxing He

Abstract:

In recent years, the field of language modelling has witnessed exciting developments. Researchers are able to train language models which can generate realistic text. However, our understanding of these powerful language models remains shallow. This talk highlights our efforts towards a deeper understanding of neural language generation. In the first part, we will discuss “how to do generation”. We will review several popular sampling algorithms, and show their performance are on par with each other. Then, we will extract high-level shared properties from the definition of the algorithms, and design experiments to validate their importance. In the second part, we will discuss “what to generate”. We will review some known problems arised from neural language generation. We propose the negative training algorithm to deal with those bad behaviors.

Details

• Date: Wednesday, April 27
• Time: 2:00 pm
• Location: 32-G882 (Hewlett Room) with zoom option

Additional Location Details:

Thesis Supervisor: Dr. Jim Glass

Virtual Link: https://mit.zoom.us/j/98047395332