Froehlich, Findlater, Chellappa Awarded $1 Million DoD Grant to Assist Visually-Impaired Veterans

Mon Sep 29, 2014

Three UMIACS faculty members—Jon Froehlich, Leah Findlater and Rama Chellappa—have been awarded a $1 million grant from the U.S. Army Medical Research and Materiel Command via the Department of Defense to work on a project that will assist visually-impaired veterans.

The team will develop a new vision-augmented touch system called HandSight, which is intended to support daily activities by sensing and feeding back non-tactile information about the physical world as it is touched.

HandSight consists of tiny CMOS cameras and micro-haptic actuators integrated into one or more fingers and is supported by a smart watch for processing power and speech output. Computer-vision and machine learning algorithms are utilized to support fingertip-based sensing.

Their first goal is to allow visually-impaired users to touch printed text and receive speech output in real-time. Eventually, they hope users will be able to touch a piece of clothing and hear suggestions for coordinating an outfit or touch a piece of fruit and instantly know its color.

The grant will be awarded over the course of three years. Froehlich, an assistant professor of computer science, is the primary investigator on the grant while Findlater, an assistant professor in the iSchool, and Chellappa, chair of the Department of Electrical and Computer Engineering, serve as co-principal investigators. The team is being assisted by three computer science masters and doctoral students, and an undergraduate student from the School of Architecture.