Sign Language Recognition Module (2022)

Machine learning system for real-time sign language gesture recognition and translation.

Sign Language Recognition Module

About the Project

Machine learning system for real-time sign language gesture recognition and translation.

Technologies Used

Machine Learning Computer Vision Accessibility Technology

IntroductionIn France in 2020, there were approximately 4 and 5 million deaf orhard of hearing people who have difficulties or are simply unable tocommunicate through speech. Concerning the deaf speakers of FrenchSign Language (LSF), the figures are uncertain: they oscillate between80,000 and 120,000, depending on the sources.This sign language is too little used because it requires a significantinvestment to learn it and most people do not use it directly in theirlives.

Publicly available datasets are limited in both quantity and quality,making many traditional machine learning algorithms inadequate for thetask of building classifiers. Language recognition (SLR) methods typicallyextract features via deep neural networks and suffer from overlearningdue to their imprecise and limited data. Many of these models are baseddirectly on the analysis of retrieved images and the analysis of imagepixels.Many early SLR systems used gloves for data retrieval and accelerometersto acquire hand features.While these techniques offered the advantage of accurate positions,they did not allow for full natural movement and restricted mobility,altering the signs performed. Further trials with a modified glovelikedevice, which was less restrictive, attempted to address this issueBut recently, skeleton coordinate-based action recognition has beenattracting more and more attention because of its invariance to subject orbackground, whereas skeleton coordinate-based SLR only tak

IA modelThe AI for sign language recognition (SLR) is designed to make the creation easier, facilitating the creation of the dataset, implementing new models, visualization of training results, as well as a dynamic visualisation of recorded movements. The model is composed of a bidirectional LTSM, a linear, a dropout, a batchnorm1d, a relu, and an other Linear. The AI trained on 16 different signs or 1600 sequences. I reach an accuracy of 87% on the test set and 96% on the training.

Project image
Project image
Project image
Project image
Project image
Back to Projects