Kapadia, Crandall earn prestigious Google Faculty Research Award

Luddy School of Informatics, Computing, and Engineering Associate Professor of Computer Science Apu Kapadia, Associate Professor of Informatics and Computing David Crandall, and Taslima Akter, a Ph.D. student in computer science who is part of Luddy’s Privacy Lab, have been awarded a Google Faculty Research Award for their research into automated mechanisms that can preserve privacy for the visually impaired who are using assistive technology.

The Google Faculty Research Awards are designed to support technical research in computer science, engineering, and related fields that allows Google researchers to partner with faculty at academic institutions around the world to develop new technologies. The proposal submitted by Kapadia and Crandall was one of just 150 selected by Google from more than 900 submissions.

“We are excited to have received this highly competitive grant from Google, which is a strong endorsement of Taslima’s research on assistive technologies for the visually impaired,” Kapadia said.

People with visual impairments often use services to help them identify everyday objects or answer questions about photographs taken by wearable or handheld cameras. Since computer vision is limited in its ability to identify objects with a high level of accuracy or with the appropriate context—think identifying a “food can” vs. a “Campbell’s soup can”—services typically use human volunteers to answer questions.

The use of human volunteers, however, creates a certain level of privacy and security risks for users since sensitive information or bystanders can be captured in the photographs submitted to the services. Crandall and Kapadia are developing automated mechanisms that can identify and characterize sensitive content in a way that maintains both the quality of assistance while protecting the privacy of users and those around them.

“Ultimately, we hope that assistive technologies relying on artificial intelligence can be more dependable, not only in conveying information more appropriately but also to preserve and strengthen social relationships,” Kapadia said. “Nobody wants assistive technologies that ‘lie’ and cause misunderstandings. Taslima’s research has raised interesting questions about the accuracy of AI-based systems and the implications of automatically conveying incorrect information about other people. This project will research ways to communicate inferred information about one’s surroundings in a way that is more nuanced.”

The proposal aims to demonstrate automated classification of objects and people as being in the foreground or background of photos, allowing the background of photos to be protected. The project also will build a classification pipeline that explains object and person classifications by combining and interpreting classification labels as well as confidence information.

“We’re always looking for opportunities to collaborate with leading companies in the tech industry, and we’re thrilled with this partnership with Google,” said Kay Connelly, the associate dean for research at the Luddy School. “Apu and David are leaders in the fields of privacy and computer vision, and this is the perfect partnership to develop tools that can be critically important to people with visual impairments.”