Kinect Used to Convert Sign Language Into Text

Sep 6, 2013
Tsuneyuki Miyake, Tech-On!
[Click to enlarge image]

Mizuho Information & Research Institute Inc and Chiba University started to develop a system that recognizes a sign language and automatically coverts it into Japanese characters by using a commercially-available motion sensor such as Microsoft Corp's "Kinect."

The researchers from the institute and the university aim to help communications between hearing-impaired people and normal listeners. They plan to complete a prototype in October 2013 and put it into practical use within 2014.

The system (1) senses the movements of the signer's forearm (wrists, elbows, etc), (2) compares the movements with motion data for each word, (3) automatically estimates the meanings of the movements and (4) shows Japanese characters on a display in real time.

In the co-development project, Mizuho Information & Research Institute will develop the application of the system, and Chiba University will offer a technique to recognize sign languages and prepare motion data for each word.

They started to discuss the commercialization of sign language recognition technology in April 2013 and concluded that it is possible to develop a system with a high enough performance and accuracy by combining the university's sign language recognition technology and a commercially-available motion sensor.