Doctoral Thesis: CoMo: a Whiteboard that Converses about Code


Event Speaker: 

Andrew Correa

Event Location: 

32-D463 (Star Room)

Event Date/Time: 

Wednesday, July 16, 2014 - 9:30am



Software engineers routinely solve problems by brainstorming at whiteboards. Among other modes, they communicate with speech and sketch. Unfortunately, the whiteboard plays the role of a passive medium. It serves only as a place to draw. But what if it could engage in the conversation, even to a limited degree? Ideally it would help guide the engineers to a solution by being an active participant in the conversation.


This thesis presents an early version of that vision: CoMo, a whiteboard that converses about code. CoMo is capable of engaging its user in a constrained mixed-initiative symmetric-multimodal conversation about a data structure manipulation.  When it understands the data structure, it uses a code synthesis system to generate functioning C code. It can successfully hold a limited conversation and synthesize code for 50 manipulations on 8 data structures. This thesis further presents findings from an observational user study that helped guide the interaction with CoMo. Finally, this thesis presents the mixed-initiative code-generation framework that CoMo implements to achieve its interaction, and the insights about having limited natural conversations about data structure manipulations that were gleaned while creating the framework.


Thesis Supervisor: Prof. Randall Davis