Real-time Italian Sign Language Recognition with Deep Learning

Image recognition systems have evolved so much that they can actually be exploited to solve significant challenges today, such as facilitating communication for people with hearing impairments relying on sign languages. This project aims to apply deep learning and fine-tuning techniques to build an automatic recognition system for the Italian Sign Language (LIS). More specifically, our goal is a real-time image recognition system capable of accurately identifying the letters of the LIS alphabet provided by a user in a Human Computer Interaction (HCI) framework by means of Python’s Open Source Computer Vision (OpenCV) library and two models based on convolutional neural networks, namely CNN and VGG19, applied for large-scale image and video recognition. In addition to testing the performance of different architectures, our work constitutes a novel step towards the application of automatic image recognition techniques with the recently acknowledged LIS and a lately released open-source dataset, also representing the only source available for this type of research on single-handed isolated signs. This project may not only play a role in the interpretation and learning of the Italian Sign Language, encouraging its spread and study, but also in the inclusion of hearing-impaired individuals in the language research domain.

Publication year:2022

Accessibility:Open