Hierarchical transfer learning for online recognition of compound actions

Victoria Bloom, V. Argyriou, D. Makris

    Research output: Contribution to journalArticlepeer-review

    33 Citations (Scopus)
    261 Downloads (Pure)


    Recognising human actions in real-time can provide users with a natural user interface (NUI) enabling a range of innovative and immersive applications. A NUI application should not restrict users’ movements; it should allow users to transition between actions in quick succession, which we term as compound actions. However, the majority of action recognition researchers have focused on individual actions, so their approaches are limited to recognising single actions or multiple actions that are temporally separated. This paper proposes a novel online action recognition method for fast detection of compound actions. A key contribution is our hierarchical body model that can be automatically configured to detect actions based on the low level body parts that are the most discriminative for a particular action. Another key contribution is a transfer learning strategy to allow the tasks of action segmentation and whole body modelling to be performed on a related but simpler dataset, combined with automatic hierarchical body model adaption on a more complex target dataset. Experimental results on a challenging and realistic dataset show an improvement in action recognition performance of 16% due to the introduction of our hierarchical transfer learning. The proposed algorithm is fast with an average latency of just 2 frames (66 ms) and outperforms state of the art action recognition algorithms that are capable of fast online action recognition.
    Original languageEnglish
    Pages (from-to)62-72
    JournalComputer Vision and Image Understanding
    Early online date11 Dec 2015
    Publication statusPublished - Mar 2016

    Bibliographical note

    NOTICE: this is the author’s version of a work that was accepted for publication in Computer Vision and Image Understanding. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Computer Vision and Image Understanding, [VOL 144, (2015)] DOI: 10.1016/j.cviu.2015.12.001

    © 2015, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/


    • Online action recognition
    • Online interaction recognition
    • Hierarchical
    • Transfer learning


    Dive into the research topics of 'Hierarchical transfer learning for online recognition of compound actions'. Together they form a unique fingerprint.

    Cite this