Davis & Ouyang make handwriting on the wall (tablet, that is) into computer lingo

February 19, 2010

Davis and EECS grad student Tom Ouyang develop tablet that can interpret sketches.
EECS Professor and currently a Research Director in the Computer Science and Artificial Intelligence Lab (CSAIL) Randy Davis and PhD student Tom Ouyang have been developing software that will allow humans to sketch data freehand into a tablet computer (or wallboard) and enable the computer to interpret the information and make it useful multiple ways.

As described by the MIT News Office the work of Davis and Ouyang goes beyond 'simple' sketch recognition.

"...while a drawing can be rich in information, it's information that's usually inaccessible to computers. If you draw a diagram on the screen of a tablet computer, like the new Apple iPad, the computer can of course store the drawing as an image. But it can't tell what the image means.

MIT researchers intend to change that, with a new system that can interpret sketches. If a chemist, for example, uses a stylus — an inkless plastic pen — to draw a molecule on a tablet computer, the software can identify different types of chemical bonds and element symbols and determine the structure of the molecule. Similarly, if an electrical engineer draws a circuit diagram, the software will identify the circuit's separate components — like resistors, capacitors, batteries, and simple wires — and display them in different colors.

Once a sketch has been interpreted by computer, it becomes much more useful. A chemical sketch, for instance, could be the basis for a literature search, to see whether there's any prior research on the same molecule; analysis software could determine whether the circuit depicted in a sketch will perform as intended. Or design software could simply clean up and standardize a sketch for display in a journal or PowerPoint presentation."

Read more:

MIT News February 19, 2010 article: Sketch-interpreting software, A new system that lets people enter data into a tablet computer simply by drawing diagrams on the screen could lead to interactive whiteboards. (Larry Hardesty, MIT News Office)