VOI - Voice of Indonesia | RRI - Radio Republik Indonesia

A researcher and senior lecturer from the Faculty of Computer Science, University of Indonesia (Fasilkom UI), Dr. Erdefi Rakun developed a mobile application capable of recognizing, translating and transforming Indonesian Sign System -SIBI motion videos into Indonesian text. Currently, these applications can be accessed via smartphones with the Android platform. The translation system developed by Dr. Erdefi is able to recognize SIBI cues with an accuracy rate reaching 98%.

Dr. Erdefi explained that the purpose of developing this application is to facilitate sign language learning for those who need it such as those with special needs, teaching staff, parents, and the community. He is currently developing two applications related to SIBI, namely an application for translating text-to-gesture, and an application for translating gesture-to-text. However, now his side is focusing on dealing with the most difficult problem first, namely the gesture-to-text translation system. The translation system they have developed can accommodate gesture movements in SIBI, which follow the grammatical structure of Indonesian.

To build this translation system, Dr. Erdefi and the team use MobileNet to extract video features. Then, it also uses the Threshold Conditional Random Fields (T-CRF) model to identify which video frames contain gestures and which video frames contain non-gestures. To be able to translate SIBI gestures into Indonesian text, the application needs the ability to recognize every SIBI gesture. So, Dr. Erdefi collaborated with teachers and students of the Santi Rama Special School in recording the SIBI gesture movement.