AlignBodyNet: Deep Learning-based Alignment of Non-overlapping Partial Body Point Clouds from a Single Depth Camera

Pengpeng Hu, Edmond S.L. Ho, Adrian Munteanu

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)

Abstract

This paper proposes a novel deep learning framework to generate omnidirectional 3D point clouds of human bodies by registering the front- and back-facing partial scans captured by a single depth camera. Our approach does not require calibration-assisting devices, canonical postures, nor does it make assumptions concerning an initial alignment or correspondences between the partial scans. This is achieved by factoring this challenging problem into ( i ) building virtual correspondences for partial scans, and ( ii ) implicitly predicting the rigid transformation between the two partial scans via the predicted virtual correspondences. In this study, we regress the SMPL vertices from the two partial scans for building the virtual correspondences. The main challenges are ( i ) estimating the body shape and pose under clothing from single partial dressed body point clouds, and ( ii ) the predicted bodies from front- and back-facing inputs required to be the same. We, thus, propose a novel deep neural network dubbed AlignBodyNet that introduces shape-interrelated features and a shape-constraint loss for resolving this problem.We also provide a simple yet efficient method for generating real-world partial scans from complete models, which fills the gap in the lack of quantitative comparisons based on the real-world data for various studies including partial registration, shape completion, and view synthesis. Experiments based on synthetic and real-world data show that our method achieves state-of-the-art performance in both objective and subjective terms.
Original languageEnglish
Article number2502609
Pages (from-to)(In-Press)
Number of pages9
JournalIEEE Transactions on Instrumentation and Measurement
Volume72
Early online date16 Nov 2022
DOIs
Publication statusPublished - 2023
Externally publishedYes

Keywords

  • 3-D scanning
  • deep learning on point clouds
  • iterative closest point (ICP)
  • non-overlapping registration
  • partial registration
  • virtual correspondence

ASJC Scopus subject areas

  • Instrumentation
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'AlignBodyNet: Deep Learning-based Alignment of Non-overlapping Partial Body Point Clouds from a Single Depth Camera'. Together they form a unique fingerprint.

Cite this