RIT/NTID joins forces with Google, Georgia Tech to create PopSign
A powerful collaboration among two universities and a tech giant is working to develop a mobile app that will enable parents of deaf children to more easily learn American Sign Language.
The Center on Access Technology at Rochester Institute of Technology’s National Technical Institute for the Deaf, in partnership with Google and Georgia Institute of Technology, is creating PopSign, which provides an extensive, interactive learning experience that parents can use anytime, anyplace. The app displays a video of a person signing to introduce new vocabulary, and then the user’s memory is tested. To advance to the next level, the user drags and shoots a ball to hit bubbles with written words that match the sign.
The game is available on Google Play and Android devices.
“We are thrilled to collaborate with Georgia Institute of Technology and Google on this project,” said Spencer Montan, associate director of NTID’s Center on Access Technology. “Our student employees are also actively involved in re-designing the app’s user interface and user experience and making it more ‘accessibility friendly.’ We aim to expand the list of signs and the number of developers and designers to make the game more fun and available for anybody who wants to learn ASL.”
In the United States, nearly 95 percent of deaf and hard-of-hearing children have hearing parents. Most hearing parents do not know American Sign Language, which can lead to struggles and frustrations in communicating with their children. According to a study by the National Institutes of Health, language deprivation may cause complications in a child’s mental health and development and, in this case, it is recommended that parents learn ASL. However, ASL classes are often inaccessible to parents, and watching ASL videos can feel distant and unengaging.
PopSign began as an extension of a doctoral dissertation by Kimberly Xu, a former Georgia Tech student, to help hearing parents of deaf infants learn the ASL vocabulary needed to communicate with their children.
“My students have been developing PopSign since 2016,” said Thad Starner, professor, founder, and director of the Contextual Computing Group at Georgia Tech’s College of Computing. “While we designed the game to help parents learn ASL, the game may be useful for children who want to be bilingual in ASL and English. That is a surprising angle that we are just starting to explore.
“We are working with RIT/NTID students to improve the playability, accessibility, and enjoyment of the current game. However, the next games in development, PopSign Generation and CopyCat Mobile, are being redesigned from scratch with the RIT/NTID students, at their suggestion. These games will use a smartphone’s selfie camera to recognize the player’s signing as part of the game.”
RIT/NTID CAT Lab students Ikemefuna Chukwunyerenwa, a web and mobile computing student from Houston; Jinlan Li, a professional studies student from China; and Loam Shin, a human-centered computing student from Lawrenceville, Ga., serve as user interface designers, user experience analysts, and software developers, while Georgia Tech students Colby Duke, Sahir Shahryar, and Prerna Ravi serve as project managers and data collectors, and develop the sign language recognizers that PopSign uses.
RIT/NTID alumnus Sam Sepah ’08 MS (human resource development), lead accessibility research project manager at Google, also has been involved in the project.
Established by the U.S. Congress in 1965, the National Technical Institute for the Deaf is the first and largest technological college in the world for deaf and hard-of-hearing students. NTID offers associate degree programs for deaf and hard-of-hearing students and provides support and access services for deaf and hard-of-hearing students who study in the other eight colleges of RIT. NTID also offers a certificate in healthcare interpretation, bachelor’s degree programs in sign language interpreting, and community development and inclusive leadership, and master’s degrees in healthcare interpretation and secondary education for individuals interested in teaching deaf and hard-of-hearing students. Deaf and hard-of-hearing students come from all over the United States and around the world to take advantage of the opportunities available to them at RIT/NTID. Go to www.rit.edu/ntid
Rochester Institute of Technology is home to leading creators, entrepreneurs, innovators and researchers. Founded in 1829, RIT enrolls 19,700 students in more than 200 career-oriented and professional programs, making it among the largest private universities in the U.S.
The university is internationally recognized and ranked for academic leadership in business, computing, engineering, imaging science, liberal arts, sustainability, and fine and applied arts. RIT also offers unparalleled support services for deaf and hard-of-hearing students. The cooperative education program is one of the oldest and largest in the nation. Global partnerships include campuses in China, Croatia, Dubai and Kosovo.
