Efficient activity recognition using lightweight CNN and DS-GRU network for surveillance applications

Amin Ullah, Khan Muhammad, Weiping Ding, Vasile Palade, Ijaz Ul Haq, Sung Wook Baik

    Research output: Contribution to journalArticlepeer-review

    54 Citations (Scopus)
    79 Downloads (Pure)


    Recognizing human activities has become a trend in smart surveillance that contains several challenges, such as performing effective analyses of huge video data streams, while maintaining low computational complexity, and performing this task in real-time. Current activity recognition techniques are using convolutional neural network (CNN) models with computationally complex classifiers, creating hurdles in obtaining quick responses for abnormal activities. To address these challenges in real-time surveillance, this paper proposes a lightweight deep learning-assisted framework for activity recognition. First, we detect a human in the surveillance stream using an effective CNN model, which is trained on two surveillance datasets. The detected individual is tracked throughout the video stream via an ultra-fast object tracker called the ‘minimum output sum of squared error’ (MOSSE). Next, for each tracked individual, pyramidal convolutional features are extracted from two consecutive frames using the efficient LiteFlowNet CNN. Finally, a novel deep skip connection gated recurrent unit (DS-GRU) is trained to learn the temporal changes in the sequence of frames for activity recognition. Experiments are conducted over five benchmark activity recognition datasets, and the results indicate the efficiency of the proposed technique for real-time surveillance applications compared to the state-of-the-art.

    Original languageEnglish
    Article number107102
    JournalApplied Soft Computing
    Early online date15 Jan 2021
    Publication statusPublished - May 2021

    Bibliographical note

    This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).


    Funding Information:
    This work was supported by the National Research Foundation of Korea and funded by the Korean government (MSIP) under Grant No. 2019R1A2B5B01070067 .


    • Activity recognition
    • Artificial intelligence
    • Deep learning
    • IoT
    • Machine learning
    • Pattern recognition
    • Video big data analytics

    ASJC Scopus subject areas

    • Software


    Dive into the research topics of 'Efficient activity recognition using lightweight CNN and DS-GRU network for surveillance applications'. Together they form a unique fingerprint.

    Cite this