Undergraduate Research: A Glove That Helps Teach Sign Language

Office of CommunicationsAcademics, College of Professional Schools, Faculty & Staff, News, Research & Innovation, Z2

Senior Maggie Cross was one of 28 Norwich University undergraduates awarded Summer Research Fellowships to explore diverse topics across the arts, sciences and professional fields. Developed by the university’s Office of Academic Research, the competitive, six- and ten-week fellowships are funded by university endowments dedicated to supporting student academic investigation.
Norwich University Office of Communications

August 31, 2015

Overlapping worlds often spark innovation. Engineering major Maggie Cross knew she wanted to pursue an NU Summer Research Fellowship. Her advisor Prof. Michael Prairie suggested she explore haptics, the use of tactile feedback to speed learning.

Cross proposed that she develop a glove that could help wearers learn the sign language alphabet more quickly. Her inspiration: a sign language interpreter she observed while volunteering at a Vermont state LEGO robotics competition last fall.

Ideally, the glove would analyze and respond to the hand movements of wearers, activating embedded buzzers when students bent individual fingers incorrectly.

Such haptic feedback has been used to cue surgeons when they slice the wrong tissue during surgeries with obstructed views. More recently, engineers at Georgia Tech developed a “piano glove” that helps wearers learn the first 45 notes of Beethoven’s “Ode to Joy” more quickly.

Cross emailed the PhD student involved in that project to float ideas about her own project.

In June Cross began work in the lab. Early experiments involved attaching bend resistors, accelerometers and gyroscopes to a white cotton glove to track hand motions.

She soon abandoned that approach after discovering the Leap motion controller, an off-the-shelf device that senses and plots natural hand motions in real time. Think Wii game station controller or the Matrix sci-fi film.

Cross then moved on to increasingly complex challenges of programming, wireless integration, database comparison, and sensor feedback.

For each sign letter attempt, her glove needed to capture the wearer’s gestures; compare them against a database of proper letter signs; and send, receive and activate the correct tactile feedback between the computer-based database and the glove.

Never a strong coder, Cross says the project forced her to dive deep into a number of programming languages, including C++, Objective C, JavaScript and Xcode, the coding language behind Apple iPhone aps.

The rising senior observes that there are nearly 70 million deaf people in the world and that sign language is important not only to the heard of hearing, but to their friends and family.

“Mastering sign language takes dedication and, above all, time. Using haptic feedback to learn sign language would reduce the amount of time required,” Cross notes.

“It could make bridging the gap between the hearing and hearing impaired communities more accessible.”

Cross says she was inspired to apply for her 10-week summer research fellowship by a friend, Ryan Fecteau ’15, who conducted social science research on binge drinking at military colleges as a Norwich University undergraduate summer research fellow last year.

“He said it was a good experience and he learned a lot—not just learned technically, but learned how to do research,” Cross says.

Cross has had a similar experience this summer. “I’ve worked all weekend before on a lab project. But at the end of the weekend, it’s always finished,” Cross says.

“Whereas with this, you’ll work five [or] six … seven or eight hour days and then get nothing. So you have to take a step back and realize that you’re learning how not to do it. So you are still learning, even though you don’t have anything to show for it.”

Cross had made significant progress on her glove over the course of her fellowship.

“In a way our engineers are working with the languages of machines—but many machines, many languages,” says Norwich University Professor of Electrical Engineering Ronald Lessard. “They’re trying to find out [how] to communicate with these machines so that they can get them, the machines, to work together to do what they need.”

Cross says she may continue her work as a senior thesis project, expanding the scope of her initial summer research fellowship. “Instead of just being the hands and the alphabet, you could make a sleeve out of it [to promote haptic learning of the] full sign language alphabet with all the gestures.”

Related Articles on Undergraduate Summer Research Fellows: