E. Alvarez de la Campa Crespo and B. Spanlang, "A Deep CNN Model for Inferring 3D Human Body Shapes Using Front and Side Images", Proc. of 3DBODY.TECH 2021 - 12th Int. Conf. and Exh. on 3D Body Scanning and Processing Technologies, Lugano, Switzerland, 19-20 Oct. 2021, #32, https://doi.org/10.15221/21.32.

Title:

A Deep CNN Model for Inferring 3D Human Body Shapes Using Front and Side Images

Authors:

Elena ALVAREZ DE LA CAMPA CRESPO, Bernhard SPANLANG

Virtual Bodyworks S.L., Barcelona Health Hub, Barcelona, Spain

Abstract:

Immersive Virtual Reality (IVR) studies indicate that there is some level of the brain that does not distinguish between reality and virtual reality. In this context, a self avatar embodied from first person perspective brings a significant and lasting change to the user. IVR is therefore widely used in research and for psychological and physiological health rehabilitation. We use IVR in a wide range of areas in pain and mobility, emotional health, diversity equity and inclusion, to rehabilitate domestic violence offenders and to promote healthier lifestyles among obese people. A remaining challenge is to accurately and efficiently create avatars with body shapes and appearance that closely match those of the real user’s bodies. This is owing to the huge differences in human body forms, the reduction of the complex human shape by body scanning technology and the complexity of acquiring accurate body measurements.

The primary objective of this work was to construct a cost-effective and accurate model to infer the 3D shape from a front and side image of a person taken with a smartphone. To achieve this, we used a fully morphable human body model to change the body shape using a set of body shape modifying parameters. We create a dataset of thousands of computer generated front and side images varying the shape modifiers of the morphable model. We then train a convolutional neural network (CNN) using that dataset.

Our approach efficiently infers 3D human body shapes from a person's front and side image generating an accurate representation of a person. We made preliminary tests using a set of 10 body scans with known measurements, creating computer generated front and side images of the scans and using these images as input to the CNN and to compare the resulting body shape with the original 3D body scan.

Our results demonstrate the effectiveness of the designed approach. Our proposed model enables us to create a fully movable avatar that can be embodied in IVR from a front and side smartphone photo in a fully automated way. The same inferred shape modifiers can also be used on the clothing of the avatar to enable us to dress the avatar. Although a larger comparative study needs to be performed before the use of this approach can be routinely recommended, we believe that the convenience and ease-of-use of this model will contribute to increase the reach of VR tools with look-alike avatars also in clinical settings.

Keywords:

3D Body Scanning, Body Measurement, 3D Body Shape, Virtual Reality, Embodiment, Deep Learning, Machine Learning, Health Application, Convolutional Neural Networks, Computer Vision

Details:

Full paper: 2132alvarezdelacampa.pdf
Proceedings: 3DBODY.TECH 2021, 19-20 Oct. 2021, Lugano, Switzerland
Paper id#: 32
DOI: 10.15221/21.32
Presentation video: 3DBodyTech2021_32_Alvarez.mp4

Copyright notice

© Hometrica Consulting - Dr. Nicola D'Apuzzo, Switzerland, hometrica.ch.
Reproduction of the proceedings or any parts thereof (excluding short quotations for the use in the preparation of reviews and technical and scientific papers) may be made only after obtaining the specific approval of the publisher. The papers appearing in the proceedings reflect the author's opinions. Their inclusion in these publications does not necessary constitute endorsement by the editor or by the publisher. Authors retain all rights to individual papers.

Note: click the + on the top left of the page to open/close the menu.