Long short-term memory based activity detection using skeleton joints data: a systematic review

Abstract

In today's security and surveillance applications, recognizing abnormal activity is critical component of identifying possible hazardous or unusual human behaviors. There is need for new technologies that can detect abnormal human behaviors precisely. The present review investigates various aspects of detection process, focusing on skeleton joints input data. It then explores adoption of deep learning (DL) architectures, such as long short-term memory (LSTMs) and transformers, to improve accuracy and robustness of recognition models. The review explores synergistic integration of LSTMs and transformers to improve recognition of unusual activity. By integrating LSTMs' processing capabilities and attention mechanisms of transformers, enhanced models can accurately identify intricate patterns of activity. Despite the advancements that have been made in the field, the challenges that remain are still related to recognition of unusual activities. These include lack of scalability for large datasets, need for models that can recognize complex behaviors across diverse applications, and need to ensure that detection is performed in a low-latency manner. The paper explores future directions of developing LSTM-based models that can recognize unusual activity using skeleton joint data in a cloud-based environment. The review emphasizes the potential of such solutions that can take advantage of the processing power of graphics processing units (GPUs) and tensor processing units (TPUs) and enable real-time recognition of activity in large datasets.