3D Object Completion via Class-Conditional Generative Adversarial Network

Yu-Chieh Chen; Daniel Stanley Tan; Wen-Huang Cheng; Kai-Lung Hua

doi:10.1007/978-3-030-05716-9_5

3D Object Completion via Class-Conditional Generative Adversarial Network

Yu-Chieh Chen, Daniel Stanley Tan, Wen-Huang Cheng, Kai-Lung Hua^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

Abstract

Many robotic tasks require accurate shape models in order to properly grasp or interact with objects. However, it is often the case that sensors produce incomplete 3D models due to several factors such as occlusion or sensor noise. To address this problem, we propose a semi-supervised method that can recover the complete the shape of a broken or incomplete 3D object model. We formulated a hybrid of 3D variational autoencoder (VAE) and generative adversarial network (GAN) to recover the complete voxelized 3D object. Furthermore, we incorporated a separate classifier in the GAN framework, making it a three player game instead of two which helps stabilize the training of the GAN as well as guides the shape completion process to follow the object class labels. Our experiments show that our model produces 3D object reconstructions with high-similarity to the ground truth and outperforms several baselines in both quantitative and qualitative evaluations.

Original language	English
Title of host publication	MultiMedia Modeling
Subtitle of host publication	25th International Conference, MMM 2019, Thessaloniki, Greece, January 8–11, 2019, Proceedings, Part II
Editors	Benoit Huet, Ioannis Kompatsiaris, Stefanos Vrochidis, Vasileios Mezaris, Wen-Huang Cheng, Cathal Gurrin
Publisher	Springer
Pages	54-66
Number of pages	13
ISBN (Print)	9783030057152, 9783030057169
DOIs	https://doi.org/10.1007/978-3-030-05716-9_5
Publication status	Published - 11 Dec 2018
Externally published	Yes
Event	25th International Conference on MultiMedia Modeling - Thessaloniki, Greece Duration: 8 Jan 2019 → 11 Jan 2019 https://link.springer.com/book/10.1007/978-3-030-05716-9

Publication series

Series	Lecture Notes in Computer Science
Volume	11296
ISSN	0302-9743

Conference

Conference	25th International Conference on MultiMedia Modeling
Abbreviated title	MMM 2019
Country/Territory	Greece
City	Thessaloniki
Period	8/01/19 → 11/01/19
Internet address	https://link.springer.com/book/10.1007/978-3-030-05716-9

Keywords

Generative adversarial network
Object classification
Object reconstruction
Shape completion

Access to Document

10.1007/978-3-030-05716-9_5

Cite this

Chen, Y.-C., Tan, D. S., Cheng, W.-H., & Hua, K.-L. (2018). 3D Object Completion via Class-Conditional Generative Adversarial Network. In B. Huet, I. Kompatsiaris, S. Vrochidis, V. Mezaris, W.-H. Cheng, & C. Gurrin (Eds.), MultiMedia Modeling: 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8–11, 2019, Proceedings, Part II (pp. 54-66). Springer. https://doi.org/10.1007/978-3-030-05716-9_5

Chen, Yu-Chieh ; Tan, Daniel Stanley ; Cheng, Wen-Huang et al. / 3D Object Completion via Class-Conditional Generative Adversarial Network. MultiMedia Modeling: 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8–11, 2019, Proceedings, Part II. editor / Benoit Huet ; Ioannis Kompatsiaris ; Stefanos Vrochidis ; Vasileios Mezaris ; Wen-Huang Cheng ; Cathal Gurrin. Springer, 2018. pp. 54-66 (Lecture Notes in Computer Science, Vol. 11296).

@inproceedings{72faedb74f4a425cbda00dfc103296ad,

title = "3D Object Completion via Class-Conditional Generative Adversarial Network",

abstract = "Many robotic tasks require accurate shape models in order to properly grasp or interact with objects. However, it is often the case that sensors produce incomplete 3D models due to several factors such as occlusion or sensor noise. To address this problem, we propose a semi-supervised method that can recover the complete the shape of a broken or incomplete 3D object model. We formulated a hybrid of 3D variational autoencoder (VAE) and generative adversarial network (GAN) to recover the complete voxelized 3D object. Furthermore, we incorporated a separate classifier in the GAN framework, making it a three player game instead of two which helps stabilize the training of the GAN as well as guides the shape completion process to follow the object class labels. Our experiments show that our model produces 3D object reconstructions with high-similarity to the ground truth and outperforms several baselines in both quantitative and qualitative evaluations.",

keywords = "Generative adversarial network, Object classification, Object reconstruction, Shape completion",

author = "Yu-Chieh Chen and Tan, {Daniel Stanley} and Wen-Huang Cheng and Kai-Lung Hua",

year = "2018",

month = dec,

day = "11",

doi = "10.1007/978-3-030-05716-9_5",

language = "English",

isbn = "9783030057152",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "54--66",

editor = "Benoit Huet and Ioannis Kompatsiaris and Stefanos Vrochidis and Vasileios Mezaris and Wen-Huang Cheng and Cathal Gurrin",

booktitle = "MultiMedia Modeling",

note = "25th International Conference on MultiMedia Modeling, MMM 2019 ; Conference date: 08-01-2019 Through 11-01-2019",

url = "https://link.springer.com/book/10.1007/978-3-030-05716-9",

}

Chen, Y-C, Tan, DS, Cheng, W-H & Hua, K-L 2018, 3D Object Completion via Class-Conditional Generative Adversarial Network. in B Huet, I Kompatsiaris, S Vrochidis, V Mezaris, W-H Cheng & C Gurrin (eds), MultiMedia Modeling: 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8–11, 2019, Proceedings, Part II. Springer, Lecture Notes in Computer Science, vol. 11296, pp. 54-66, 25th International Conference on MultiMedia Modeling, Thessaloniki, Greece, 8/01/19. https://doi.org/10.1007/978-3-030-05716-9_5

3D Object Completion via Class-Conditional Generative Adversarial Network. / Chen, Yu-Chieh; Tan, Daniel Stanley; Cheng, Wen-Huang et al.
MultiMedia Modeling: 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8–11, 2019, Proceedings, Part II. ed. / Benoit Huet; Ioannis Kompatsiaris; Stefanos Vrochidis; Vasileios Mezaris; Wen-Huang Cheng; Cathal Gurrin. Springer, 2018. p. 54-66 (Lecture Notes in Computer Science, Vol. 11296).

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

TY - GEN

T1 - 3D Object Completion via Class-Conditional Generative Adversarial Network

AU - Chen, Yu-Chieh

AU - Tan, Daniel Stanley

AU - Cheng, Wen-Huang

AU - Hua, Kai-Lung

PY - 2018/12/11

Y1 - 2018/12/11

N2 - Many robotic tasks require accurate shape models in order to properly grasp or interact with objects. However, it is often the case that sensors produce incomplete 3D models due to several factors such as occlusion or sensor noise. To address this problem, we propose a semi-supervised method that can recover the complete the shape of a broken or incomplete 3D object model. We formulated a hybrid of 3D variational autoencoder (VAE) and generative adversarial network (GAN) to recover the complete voxelized 3D object. Furthermore, we incorporated a separate classifier in the GAN framework, making it a three player game instead of two which helps stabilize the training of the GAN as well as guides the shape completion process to follow the object class labels. Our experiments show that our model produces 3D object reconstructions with high-similarity to the ground truth and outperforms several baselines in both quantitative and qualitative evaluations.

AB - Many robotic tasks require accurate shape models in order to properly grasp or interact with objects. However, it is often the case that sensors produce incomplete 3D models due to several factors such as occlusion or sensor noise. To address this problem, we propose a semi-supervised method that can recover the complete the shape of a broken or incomplete 3D object model. We formulated a hybrid of 3D variational autoencoder (VAE) and generative adversarial network (GAN) to recover the complete voxelized 3D object. Furthermore, we incorporated a separate classifier in the GAN framework, making it a three player game instead of two which helps stabilize the training of the GAN as well as guides the shape completion process to follow the object class labels. Our experiments show that our model produces 3D object reconstructions with high-similarity to the ground truth and outperforms several baselines in both quantitative and qualitative evaluations.

KW - Generative adversarial network

KW - Object classification

KW - Object reconstruction

KW - Shape completion

U2 - 10.1007/978-3-030-05716-9_5

DO - 10.1007/978-3-030-05716-9_5

M3 - Conference Article in proceeding

SN - 9783030057152

SN - 9783030057169

T3 - Lecture Notes in Computer Science

SP - 54

EP - 66

BT - MultiMedia Modeling

A2 - Huet, Benoit

A2 - Kompatsiaris, Ioannis

A2 - Vrochidis, Stefanos

A2 - Mezaris, Vasileios

A2 - Cheng, Wen-Huang

A2 - Gurrin, Cathal

PB - Springer

T2 - 25th International Conference on MultiMedia Modeling

Y2 - 8 January 2019 through 11 January 2019

ER -

Chen YC, Tan DS, Cheng WH, Hua KL. 3D Object Completion via Class-Conditional Generative Adversarial Network. In Huet B, Kompatsiaris I, Vrochidis S, Mezaris V, Cheng WH, Gurrin C, editors, MultiMedia Modeling: 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8–11, 2019, Proceedings, Part II. Springer. 2018. p. 54-66. (Lecture Notes in Computer Science, Vol. 11296). doi: 10.1007/978-3-030-05716-9_5

3D Object Completion via Class-Conditional Generative Adversarial Network

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this