KD-Net: Continuous Keystroke Dynamics Based Human Identification from RGB-D Image Sequences

Xinxin Dai, Ran Zhao, Pengpeng Hu, Adrian Munteanu

Research output: Contribution to journalArticlepeer-review

19 Downloads (Pure)

Abstract

Keystroke dynamics is a soft biometric based on the assumption that humans always type in uniquely characteristic manners. Previous works mainly focus on analyzing the key press or release events. Unlike these methods, we explore a novel visual modality of keystroke dynamics for human identification using a single RGB-D sensor. In order to verify this idea, we create a dataset dubbed KD-MultiModal that contains 243.2K frames of RGB images and depth images, obtained by recording the video of hand typing with a single RGB-D sensor. The dataset comprises RGB-D image sequences of 20 subjects (10 males and 10 females) typing sentences and each subject types around 20 sentences. In the task, only the hand and keyboard region contributes to the person identification, so we also propose methods of extracting Regions of Interest (RoI) for each type of data. Unlike the data of the key press or release, our dataset not only captures the velocity of pressing and releasing different keys and the typing style of specific keys or combinations of keys but also contains rich information on the hand shape and posture. To verify the validity of our proposed data, we adopt deep neural networks to learn distinguishing features from different data representations, including RGB-KD-Net, D-KD-Net and RGBD-KD-Net. Simultaneously, the sequence of point clouds also can be obtained from depth images given the intrinsic parameters of the RGB-D sensor, so we also study the performance of human identification based on the point clouds. Extensive experimental results show that our idea works and the performance of the proposed method based on RGB-D images is the best, which achieved 99.44% accuracy based on the unseen real-world data. To inspire more researchers and facilitate relevant studies, the proposed dataset will be publicly accessible together with the publication of this paper.
Original languageEnglish
Article number8370
Number of pages16
JournalSensors
Volume23
Issue number20
Early online date10 Oct 2023
DOIs
Publication statusPublished - 10 Oct 2023

Bibliographical note

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Funder

This research was funded by Innoviris under Project AI43D, the Fonds Wetenschappelijk Onderzoek (FWO) under Project G094122N, and the FWO and Innoviris under Project MUSCLES.

Keywords

  • keystroke dynamics
  • human identification
  • RGB-D images
  • image sequences

Fingerprint

Dive into the research topics of 'KD-Net: Continuous Keystroke Dynamics Based Human Identification from RGB-D Image Sequences'. Together they form a unique fingerprint.

Cite this