Recognizing human activities has become a trend in smart surveillance that contains several challenges, such as performing effective analyses of huge video data streams, while maintaining low computational complexity, and performing this task in real-time. Current activity recognition techniques are using convolutional neural network (CNN) models with computationally complex classifiers, creating hurdles in obtaining quick responses for abnormal activities. To address these challenges in real-time surveillance, this paper proposes a lightweight deep learning-assisted framework for activity recognition. First, we detect a human in the surveillance stream using an effective CNN model, which is trained on two surveillance datasets. The detected individual is tracked throughout the video stream via an ultra-fast object tracker called the ‘minimum output sum of squared error’ (MOSSE). Next, for each tracked individual, pyramidal convolutional features are extracted from two consecutive frames using the efficient LiteFlowNet CNN. Finally, a novel deep skip connection gated recurrent unit (DS-GRU) is trained to learn the temporal changes in the sequence of frames for activity recognition. Experiments are conducted over five benchmark activity recognition datasets, and the results indicate the efficiency of the proposed technique for real-time surveillance applications compared to the state-of-the-art.
Bibliographical noteFunding Information:
This work was supported by the National Research Foundation of Korea and funded by the Korean government (MSIP) under Grant No. 2019R1A2B5B01070067 .
© 2021 The Author(s)
Copyright 2021 Elsevier B.V., All rights reserved.
- Activity recognition
- Artificial intelligence
- Deep learning
- Machine learning
- Pattern recognition
- Video big data analytics
ASJC Scopus subject areas