Sign language recognition and classification using blended ensemble machine learning

Abstract

An efficient sign language recognition system (SLR) is the most significant for hearing-impaired people for communication. The body movements and hand gestures are utilized to characterize the vocabulary in dynamic sign language. The SLR is a challenging problem because the computational model requires simultaneous spatial-temporal modelling for a number of sources. To overcome this problem, this research proposes the blended ensemble machine learning (ML) approaches for SLR. Initially, the Indian sign language (ISL) dataset is collected for evaluating the effectiveness of the model. Then, the pre-processing is done by using data augmentation and normalization techniques. Then, the pre-processed data is provided to the segmentation process which is done by using multi-threshold entropy function. Then, VGG-16 is used for the feature extraction process to extract the features and finally, classification is carried out using ensemble ML. An effectiveness of the proposed method is validated based on accuracy, precision, recall, and F1-score, wherein it achieves better results of 99.57%, 0.92%, 0.95%, and 0.99% as compared to the existing works like support vector machine (SVM) and convolutional neural network (CNN).