Analysis and synthesis of 3D shape families via deep-learned generative models of surfaces



We present a method for joint analysis and synthesis of geometrically diverse 3D shape families. Our method first learns part-based templates such that an optimal set of fuzzy point and part correspondences is computed between the shapes of an input collection based on a probabilistic deformation model. In contrast to previous template-based approaches, the geometry and deformation parameters of our part-based templates are learned from scratch. Based on the estimated shape correspondence, our method also learns a probabilistic generative model that hierarchically captures statistical relationships of corresponding surface point positions and parts as well as their existence in the input shapes. A deep learning procedure is used to capture these hierarchical relationships. The resulting generative model is used to produce control point arrangements that drive shape synthesis by combining and deforming parts from the input collection. The generative model also yields compact shape descriptors that are used to perform fine-grained classification. Finally, it can be also coupled with the probabilistic deformation model to further improve shape correspondence. We provide qualitative and quantitative evaluations of our method for shape correspondence, segmentation, fine-grained classification and synthesis. Our experiments demonstrate superior correspondence and segmentation results than previous state-of-the-art approaches.


paper thumbnail


Supplementary Material


This archive contains the correspondence results from our method for BCHP dataset. Please see readme.txt in this archive for more information

Segmentation.7z, 180MB

This archive contains the ground-truth segmentations of the BCHP dataset and segmentation results from our method. Please see readme.txt in this archive for more information

FineGrainedClassification.7z, 24MB

This archive contains the fine-grained classification dataset with training and test split, our results and learned descriptors. Please see readme.txt in this archive for more information


This archive contains source code for our deep-learned model (Beta Shape Machine)., 3GB

This archive contains part of the training data for our BSM, with 3D mesh obj files, sample points and segmentation. It also provide an example code for sampling the trained BSM.




Kalogerakis gratefully acknowledges support from NSF (CHS-1422441). We thank Qi-xing Huang and Vladimir Kim for sharing data from their methods. We thank Siddhartha Chaudhuri and anonymous reviewers for valuable comments. We thank Szymon Rusinkiewicz for distributing the trimesh2 library.