Deep transfer learning for classification of ECG signals and lip images in multimodal biometric authentication systems

Abstract

Authentication plays an essential role in diverse kinds of application that requires security. Several authentication methods have been developed, but biometric authentication has gained huge attention from the research community and industries due to its reliability and robustness. This study investigates multimodal authentication techniques utilizing electrocardiogram (ECG) signals and face lip images. Leveraging transfer learning from pre-trained ResNet and VGG16 models, ECG signals and photos of the lip area of the face are used to extract characteristics. Subsequently, a convolutional neural network (CNN) classifier is employed for classification based on the extracted features. The dataset used in this study comprises ECG signals and face lip images, representing distinct biometric modalities. Through the integration of transfer learning and CNN classification, improving the reliability and precision of multimodal authentication systems is the primary objective of the study. Verification results show that the suggested method is successful in producing trustworthy authentication using multimodal biometric traits. The experimental analysis shows that the proposed deep transfer learning-based model has reported the average accuracy, F1-score, precision, and recall as 0.962, 0.970, 0.965, and 0.966, respectively.