Adapting Semantic Segmentation of Urban Scenes via Mask-Aware Gated Discriminator

Yong-Xiang Lin, Daniel Stanley Tan, Wen-Huang Cheng, Kai-Lung Hua

Research output: Chapter in Book/Report/Conference proceedingConference Article in proceedingAcademicpeer-review


Training a deep neural network for semantic segmentation relies on pixel-level ground truth labels for supervision. However, collecting large datasets with pixel-level annotations is very expensive and time consuming. One workaround is to utilize synthetic data where we can generate potentially unlimited data with their corresponding ground truth labels. Unfortunately, networks trained on synthetic data perform poorly on real images due to the domain shift problem. Domain adaptation techniques have shown potential in transferring the knowledge learned from synthetic data to real world data. Prior works have mostly leveraged on adversarial training to perform a global aligning of features. However, we observed that background objects have lesser variations across different domains as opposed to foreground objects. Using this insight, we propose a method for domain adaptation that models and adapts foreground objects and background objects separately. Our approach starts with a fast style transfer to match the appearance of the inputs. This is followed by a foreground adaptation module that learns a foreground mask that is used by our gated discriminator in order to adapt the foreground and background objects separately. We demonstrate in our experiments that our model outperforms several state-of-the-art baselines in terms of mean intersection over union (mIoU).
Original languageEnglish
Title of host publicationProceedings - 2019 IEEE International Conference on Multimedia and Expo, ICME 2019
Subtitle of host publicationProceedings
Number of pages6
ISBN (Electronic)978-1-5386-9552-4
ISBN (Print)978-1-5386-9553-1
Publication statusPublished - Jul 2019
Externally publishedYes
EventIEEE International Conference on Multimedia and Expo - Shanghai, China
Duration: 8 Jul 201912 Jul 2019


ConferenceIEEE International Conference on Multimedia and Expo
Abbreviated titleICME 2019
Internet address


  • Domain adaptation
  • Gated-convolution
  • Semantic segmentation


Dive into the research topics of 'Adapting Semantic Segmentation of Urban Scenes via Mask-Aware Gated Discriminator'. Together they form a unique fingerprint.

Cite this