PatientHandNet: 3-D Open-Palm Hand Reconstruction From Sparse Multiview Depth Images

Xinxin Dai, Ran Zhao, Pengpeng Hu, Adrian Munteanu

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
63 Downloads (Pure)

Abstract

Accurately reconstructing 3-D hand shapes of patients is important for immobilization device customization, artificial limb generation, and hand disease diagnosis. Traditional 3-D hand scanning requires multiple scans taken around the hand with a 3-D scanning device. These methods require the patients to keep an open-palm posture during scanning, which is painful or even impossible for patients with impaired hand functions. Once multi-view partial point clouds are collected, expensive post-processing is necessary to generate a high-fidelity hand shape. To address these limitations, we propose a novel deep-learning method dubbed PatientHandNet to reconstruct high-fidelity hand shapes in a canonical open-palm pose from multiple-depth images acquired with a single-depth camera. The hand poses in the depth images may vary, hand movements are allowed, facilitating the 3-D scanning process in particular for patients with difficult conditions. The proposed method has strong operability since it is insensitive to the input pose, allowing for pose variations in the input depth images. We also proposed two novel datasets: a large-scale synthetic dataset to train our model and a real-world dataset with ground-truth hand biometrics extracted by an experienced anthropometrist. Extensive experimental results on the unseen synthetic data and real-world data demonstrate that the proposed method provides robust and easy-to-use hand shape reconstruction and outperforms the state-of-the-art methods in biometric accuracy terms.
Original languageEnglish
Article number5023613
Number of pages15
JournalIEEE Transactions on Instrumentation and Measurement
Volume72
DOIs
Publication statusPublished - 22 Aug 2023

Bibliographical note

This document is the author’s post-print version, incorporating any revisions agreed during the peer-review process. Some differences between the published version and this version may remain and you are advised to consult the published version if you wish to cite from it.

Funder

10.13039/501100004744-Innoviris (Grant Number: AI43D)
10.13039/501100003130-Fonds Wetenschappelijk Onderzoek (FWO) (Grant Number: G094122N)

Keywords

  • 3-D hand reconstruction
  • 3-D hand shape
  • hand biometrics
  • multiscale features
  • multi-view depth processing
  • point cloud processing

Fingerprint

Dive into the research topics of 'PatientHandNet: 3-D Open-Palm Hand Reconstruction From Sparse Multiview Depth Images'. Together they form a unique fingerprint.

Cite this