A computer which automatically recognises sign language enables children who are deaf or hard of hearing to expand their sign language vocabulary more quickly. Jeroen Lichtenauer will receive his PhD on automatic sign language recognition at Delft University of Technology (The Netherlands) on Tuesday 13 October. His fellow researcher Jeroen Arendsen follows suit on Monday 19 October.
Perception
Three PhD candidates have spent the last few years at TU Delft simultaneously working on sign language. One of these is Jeroen Arendsen: 'Very little is known about the perception of sign language. The aim of this research was to expand our knowledge of human observation of signing. For instance, it turns out that sign language users only need to see a small part of a hand movement to know it is a sign and what it means.'
Automatic recognition
The insights obtained into human perception can subsequently be used for developing the technology for automatic sign language recognition by cameras and computers. In the TU Delft study, this led to a remarkable application of automatic sign language recognition: an Electronic Learning Environment (ELo) for children who are deaf or hard of hearing.
Pictures
In conjunction with the NSDSK (Dutch foundation for children who are deaf or hard of hearing), the TU Delft researchers developed a computer which can recognise sign language in real time and can therefore be used to teach children who are deaf and hard of hearing new signs more quickly.
When shown a picture, the children aged three to six were asked to make the correct sign. The computer then had to assess the sign. Arendsen: 'And that is quite difficult, because a computer is more easily confused than people by irrelevant fidgeting.'
Comprehensible
The task of fellow PhD candidate Jeroen Lichtenauer (EEMCS faculty) was to convert the signs into information which is comprehensible to a computer. As an Industrial Design Engineering student, Arendsen was more involved in the human aspects. Gineke Ten Holt was the third PhD candidate, who worked as a bridge between the two disciplines.
Sign language vocabulary
Further research showed that the Electronic Learning Environment really does work. The sign language vocabulary of those children who had practised with it increased compared to that of a control group. This only applied to the slightly older children however.
Whether the ELo will indeed be applied in practice is still uncertain, Arendsen says. 'It depends on aspects such as finance and organisational issues.'
Taken from www.tudelft.nl/live/.
Computer Helps Deaf Children Learn Sign Language
Share:
