New glove lets you incorporate real-life objects into virtual worlds

June 3, 2020

Augmented reality (AR) and virtual reality (VR) companies have been working for years on developing systems that can incorporate real-life objects into their digitized worlds. With that in mind, a team from MIT’s Computer Science and Artificial Intelligence Lab has developed a smart glove that can detect your hand pose, as well as distinguish between more than 30 different household objects. 

Adam Conner Simons | MIT CSAIL

“Imagine a video game where you can grab an object off your desk and have it be seamlessly incorporated into gameplay,” says MIT graduate student Andrew Spielberg, one of the co-lead authors on a new paper about the glove. “That’s the sort of application that a system like this could help enable in the future.”

Other signals the glove can capture include heart rate and various metrics about an object being held, including temperature, pressure and conductibility. It can even tell which letter of the alphabet its user is writing based on their hand pose. (They’ve only tested it on A through J so far, achieving 85 percent accuracy.) 

Spielberg said that, while they didn’t explicitly test it in the paper, the glove could also potentially measure the temperature of the glove-wearer themself - which would make it a useful non-contact way for doctors to take vital signs of patients as part of a COVID-19 diagnosis.

A key aspect of the glove is its relatively simple design. Where many state-of-the-art gloves have a complex assembly of embedded electronics, the CSAIL team’s glove doesn’t have any in-hand electronics - its design is based purely on knitted and sewn fibers, interwoven tubes and other off-the-shelf components. The glove’s parts cost only about $60, most of which is spent on the pressure sensors.

One of the team’s original goals was to see precisely how few pressure sensors they could put on a glove for it to still be able to return meaningful data. They found that even with just six or seven sensors, they could develop a coarse pressure map of what the user is grasping.

“With some very basic sensing we can do some analog measurements on how firmly you’re grasping, the amount of force you’re applying, and the stiffness of the objects you’re grabbing,” says Spielberg.

The team named their system the “Mens et Manus Glove” after MIT’s motto (“MemGlove” for short). The other co-lead authors include CSAIL postdoc Josie Hughes, and MIT undergraduate Mark Chounlakone, alongside designer Gloria Chang and MIT professors Wojciech Matusik and Daniela Rus.

Original article published on the MIT CSAIL website on June 2, 2020.