6.870 Multimodal User Interfaces (H)

L TR9:30-11, Room 5-134
Professors Randall Davis, Trevor Darrell
Prereq.: 6.034
3-0-9

This subject qualifies as an Aritifical Intelligence and Applications Engineering Concentration subject.

This course will cover the design, implementation, and evaluation of intelligent and multi-modal user interfaces. It will involve a combination of papers from the original literature and hands-on exercises to familiarize students with the basic technologies for handling speech, vision, and pen-based interaction, and enable them to explore architectures for combining modalities in a variety of ways.

Students will have the choice of implementing a demonstration system or writing a term paper.

Specific topics will be chosen by the instructors based on class interest and may include:

* Gaze tracking and gaze-aware interfaces;

* Gesture recognition;

* Mobile image-based interfaces

* Scene context recognition

* Multimodal conversational models

* Audiovisual speech recognition

* Pen-based interaction

* Multimodal fusion and adaptation algorithms

* Methods for combining channels and/or mutual disambiguation

* Case studies

* Evaluation methodologies for Multimodal Interfaces

6.170 will be useful but other programming experience is a satisfactory alternative This course is distinct in subject matter from 6.831 (User Interface Design and Implementation) and is an appropriate follow-on.