Deep Convolutional Neural Networks (DCNNs) have demonstrated impressive achievements in computer vision tasks, such as image recognition [20], object detection [17], and semantic segmentation [32]. As one of the most fundamental tasks, semantic segmentation predicts pixel-level semantic labels for given images. It plays an extremely important role in autonomous agent applications such as self-driving techniques.
Existing supervised semantic segmentation methods, however, largely rely on pixel-wise annotations which require tremendous time and labor efforts. To overcome this limitation, publicly available synthetic datasets (e.g., GTA [41] and SYNTHIA [42]) which are densely-annotated, have been considered recently. Nevertheless, the most obvious drawback of such a strategy is the poor knowledge generalization caused by domain shift issues (e.g., appearance and spatial layout differences), giving rise to dramatic performance degradation when directly applying models learned from synthetic data to real-world data of interest. In consequence, domain adaptation has been exploited in recent studies for cross-domain semantic segmentation, where the most common strategy is to learn domain-invariant representations by minimizing distribution discrepancy between source and target domains [56,29], designing a new loss function [61], considering depth information [7,49], or alternatively generating highly confident pseudo labels and re-training models with these labels through a self-training manner [62,28,22,38,51,25,54]. Following the advances of Generative Adversarial Nets (GAN) [19], adversarial learning has been used to match cross-domain representations by minimizing an adversarial loss on the source and target representations [21,34,35,13], or adapting structured output space across two domains [45,28]. Recent studies further consider the pixel-level (e.g., texture and lighting) domain shift to enforce source and target images to be domain-invariant in terms of visual appearance [58,1,52,36,9,53]. This is achieved by translating images from the source domain to the target domain by using image-to-image translation models such as CycleGAN [60] and UNIT [30].
Despite these painstaking efforts, we are still far from being able to fully adapt cross-domain knowledge mainly stemming from two limitations. First, adversarial-based image-to-image translation introduces inevitable bias to the translated images, as we cannot fully guarantee that the translated source domain ) is identical to the target domain
and
denote two domains, and F indicates an image-to-image translation model). This limitation is especially harmful to the source-to-target translation [58,1,52,36,28], since the data size of the source domain is much larger than the target domain in most of domain adaptation problems. Moreover, source-to-target translation is more computationally expensive than target-to-source translation. Second, simply aligning cross-domain representations in the feature space [21,1,45] ignores the joint distribution shift (i.e.,
), where G is used for feature extraction, while
and
indicate ground truth labels). These limitations give rise to severe false positive and false negative issues in the target prediction. This problem can get even worse when there is a significant discrepancy in layout or structure between the source and target domains, such as adapting from synthetic to real urban traffic scenes.
In this paper, we propose an innovative domain adaptation framework for semantic segmentation. The key idea is to reduce the image translation bias and align cross-domain feature representations through image reconstruction. As opposed to performing source-to-target translation [1,52,28], for the first time, we conduct the target-to-source translation to make target images indistinguishable from source images. This enables us to substantially reduce the bias in translated images and allows us to use original source images and their corresponding ground truth to train a segmentation network. Compared to the source-to-target translation, our method is also much more efficient. Besides, a reconstruction network is designed to reconstruct both source and target images from their predicted labels. It is noteworthy that we reconstruct images directly from the
Fig. 1. An example of our method on synthetic-to-real urban scene adaptation. Given a target-domain (or real) image (a), we first make target-to-source translation to obtain source-like (or synthetic) image (b), and then perform segmentation on these translated images. Our method improves the segmentation accuracy in the target domain by reconstructing both source and target images from their predicted labels (c). (d) illustrates the image reconstructed from (c), while (e) indicates the ground truth label.
label space, rather than the feature space as reported in previous studies. This is essential to guide the segmentation network by penalizing the reconstructed image that semantically deviates from the corresponding input image. Most importantly, this strategy enforces cross-domain features with the same category close to each other.
The performance of our method is evaluated on synthetic-to-real scenarios of urban scene understanding, i.e., GTA5 to Cityscapes and SYNTHIA to Cityscapes. Our results demonstrate that the proposed method achieves significant improvements compared with existing methods. Figure 1 demonstrates an example of our model in adapting cross-domain knowledge in semantic segmentation tasks and reconstructing the input image from its output label. We also carry out comprehensive ablation studies to analyze the effectiveness of each component in our framework.
The contribution of this paper is threefold.
– For the first time, we propose and investigate the target-to-source translation in domain adaptation. It reduces the image translation bias and is more computationally efficient compared to the widely-used source-to-target translation.
– To enforce semantic consistency, we introduce a label-driven reconstruction module that reconstructs both source and target images from their predicted labels.
– Extensive experiments show that our method achieves the new state-of-the-art performance on adapting synthetic-to-real semantic segmentation.
Semantic Segmentation Recent achievements in semantic segmentation mainly benefit from the technical advances of DCNNs, especially the emergence of Fully Convolutional Network (FCN) [32]. By adapting and extending contemporary deep classification architectures fully convolutionally, FCN enables pixel-wise semantic prediction for any arbitrary-sized inputs and has been widely recognized as one of the benchmark methods in this field. Numerous methods inspired by FCN were then proposed to further enhance segmentation accuracy, which have exhibited distinct performance improvement on the well-known datasets (e.g., PASCAL VOC 2012 [14] and Cityscapes [11]) [5,31,59,6,4].
However, such methods heavily rely on human-annotated, pixel-level segmentation masks, which require extremely expensive labeling efforts [11]. In consequence, weakly-supervised methods, which are based on easily obtained annotations (e.g., bounding boxes and image-level tags), were proposed to alleviate the need for effort-consuming labeling [12,40]. Another alternative is to resort to freely-available synthetic datasets (e.g., GTA5 [41] and SYNTHIA [42]) with pixel-level semantic annotations. However, models learned on synthetic datasets suffer from significant performance degradation when directly applied to the real datasets of interest, mainly owing to the domain shift issue.
Domain Adaptation Domain adaptation aims to mitigate the domain discrepancy between a source and a target domain, which can be further divided into supervised adaptation, semi-supervised adaptation, and unsupervised adaptation, depending on the availability of labels in the target domain. The term unsupervised domain adaptation refers to the scenario where target labels are unavailable and have been extensively studied [33,48,15,46,47,55,57].
Recent publications have highlighted the complementary role of pixel-level and representation-level adaptation in semantic segmentation [1,37,58,52,7], where the pixel-level adaptation is mainly achieved by translating images from the source domain to the target domain (source-to-target translation). Specifically, unpaired image-to-image translation is used in CyCADA [1] to achieve pixel-level adaptation by restricting cycle-consistency. Similarly, FCAN achieves the image translation by combining the image content in the source domain and the ”style” from the target domain [58]. I2IAdapt [37] further considers to align source and target representations based on an image-to-image translation strategy, attempting to adapt domain shift. Instead of using the adversarial learning for image translation, DCAN performs source-to-target translation by leveraging target images for channel-wise alignment [52]. Driven by the fact that geometry and semantics are coordinated with each other, GIO-Ada augments the standard image translation network by integrating geometric information [7]. However, source-to-target translation introduces substantial bias to the translated images, given that the size of the source domain is usually much larger than the target domain. To address this problem, we propose the first-of-its-kind target-to-source image translation to reduce pixel-level domain discrepancy. Compared to the source-to-target translation, it is more computationally efficient and enables us to remove the uncertainty by training the segmentation network with original source images and their corresponding labels.
Motivated by the observation that cross-domain images (e.g., GTA5 and Cityscapes) often share tremendous structural similarities, ASN [45] adapts structured output based on the adversarial learning. The strength of this method is its ability to provide weak supervision to target images by enforcing target outs to be indistinguishable from source outputs. However, it is limited to the scenario
Fig. 2. An overview of our framework. Inspired by the self-training,
[28,22,38,51,25,54] generate pseudo labels for target images and then re-training the segmentation model with these labels. It outperforms the existing methods by a large margin. However, such a strategy underestimates the side effect of pseudo labels that are incorrectly predicted. As a consequence, the segmentation model fails to increasingly improve itself using these wrong ground truth. Instead, our method reconstructs source and target input images from the label space to ensure these outputs are semantically cor-
rect. The image-to-image translation network in [28] uses a reconstruction loss and a perceptual loss to maintain the semantic consistency between the input image and the reconstruction from the translated image. Different from [28], we design a cycle-reconstruction loss in our reconstruction network to enforce the semantic consistency between the input image and the reconstruction from the
predicted label.
Reconstruction-based strategy for unsupervised domain adaptation has received considerable attention recently [16,2]. The key idea is to reconstruct input images from their feature representations to ensure that the segmentation model can learn useful information. Chang et al. [3] follow a similar idea to first disentangle images into the domain-invariant structure and domain-specific texture representations, and then reconstruct input images. LSD-seg [43] first reconstructs images from the feature space, and then apply a discriminator to the reconstructed images. Rather than performing reconstruction from feature representations, we reconstruct both source and target images from their predicted labels.
3.1 Overview
The overall design of our framework is illustrated in Figure 2, mainly containing three complementary modules: a translation network F, a segmentation network G, and a reconstruction network M. Given a set of source domain images with labels
and a set of target domain images
without any annotations.
We first perform the image-to-image translation to reduce the pixel-level discrepancy between source and target domains. As opposed to the source-to-target translation reported in previous domain adaptation methods, we conduct the target-to-source translation through an unsupervised image translation network (Figure 3). Our goal is to learn a mapping such that the distribution of images from
) is indistinguishable from the distribution of
. As a counterpart, the inverse mapping
:
, which maps images from
to
, is introduced to prevent the mode collapse issue [18]. Two adversarial discriminators
are employed for distribution match, where
indistinguishable distribution between
) and
, and
encourages indistinguishable distribution between
(More details can be found in the Supplementary).
Based on the trained model F, we first translate images from to source-like images
). Specifically, each image in
preserves the same content as the corresponding image in
while demonstrating the common style (e.g., texture and lighting) as
are then fed into a segmentation network for semantic label prediction.
Compared to translating images from the source domain to the target domain, the target-to-source translation has three benefits. First, it allows full supervision on the source domain by training the segmentation network with original source images and their corresponding labels. Second, it enables to reduce the bias in translated images. Third, it is computationally efficient lying in the fact that
3.3 Semantic Segmentation
Given that source-like images preserves all semantic information from
we apply a shared segmentation network G to
and
to predict their segmentation outputs with the loss function given by,
Fig. 3. Schematic overview of our framework which has three modules: (i) a translation network for pixel-level discrepancy reduction by translating target images to source-like images, where source-like images are indistinguishable from source images, (ii) a segmentation network that predicts segmentation outputs for source images and source-like images, and (iii) a reconstruction network for reconstructing source and source-like images from their corresponding label space.
where indicates the typical segmentation objective,
is pseudo labels of
which is derived from [28],
)) is an adversarial loss, and
leverages the importance of these losses. Specifically,
defined as,
which enforces G to learn domain-invariant features by confusing the discriminator D. It is noteworthy that we regard the segmentation outputs as features in our study. This is based on the observation that
significant similarities in terms of spatial layouts and structures [45].
3.4 Image Reconstruction from the Label Space
To encourage G to generate segmentation outputs that are semantic consistent, we introduce a reconstruction network M to reconstruct from
, where (
) indicates image size, C represents the number of label classes, and the subscript
can be either s or
to denote the source or the target domain. However, directly reconstructing images from the feature space fails to provide semantic consistency constraint to G. On the one hand,
) encodes rich information which makes the image reconstruction
Fig. 4. A comparison between the image reconstruction from feature space and label space (ours). For each input image (first column), the first and second row indicate the images reconstructed from the feature and label space, respectively.
quite straightforward. As illustrated in Figure 4, in just a few epochs, the reconstructed images derived from M are almost identical to the input images. On the other hand, to enforce cross-domain features with the same category close to each other, it is essential to perform the reconstruction based on the label space. Unfortunately, ) lies in the feature space instead. To overcome these limitations, the most clear-cut way is to convert
) to have zeros everywhere except where the index of each maximum value in the last dimension. Doing so formulates the categorical representation of the predicted label that corresponds to
). Nevertheless, such conversion is non-differentiable and cannot be trained using standard backpropagation.
Driven by the softmax action selection which is commonly used in the reinforcement learning, we apply Boltzmann distributed probabilities to approximate the semantic label map of ), which is defined as,
where is a temperature parameter. This conversion is continuous and differen-tiable, therefore, we use M to reconstruct input images
(Figure 4).
To synthesize high-resolution images from the semantic label map, we use conditional GANs [23] to model the conditional distribution of given
. To this end, we introduce M and multi-scale domain discriminators
for k = 1, 2, 3. M is designed to reconstruct
aims to distinguish
). Specifically, M follows the architecture proposed in [24], while
is based on PatchGAN [23] that penalizes structure at the scale of image patches. All
follow the same network architecture. Besides
and
) themselves, they are downsampled by a factor of 2 and 4 to obtain pyramid of 3 scales for
, and
, respectively. It is worth mentioning that
is essential to differentiate real and reconstructed images with high resolution [50], owing to its ability in providing large receptive field. The objective function is
given by,
To further enforce semantic consistency between ), we introduce a cycle-reconstruction loss
to match their feature representations.
contains a VGG perceptual loss and a discriminator feature matching loss, which is defined as,
where V is a VGG19-based model for extracting high-level perceptual information [24], M and N represent the total number of layers in V and for matching intermediate representations. Note that
when it deviates from the corresponding image
in terms of semantic consistency. In this way, M enables to map features from
closer to the features from
with the same label.
Taken together, the training objective of our framework is formulated as,
where and
leverage the importance of losses above. Notably, our method is able to implicitly encourage G to generate semantic-consistent segmentation labels for the target domain.
In this section, a comprehensive evaluation is performed on two domain adaption tasks to assess our framework for semantic segmentation. Specifically, we consider the large distribution shift of adapting from synthetic (i.e., GTA5 [41] and SYNTHIA [42]) to the real images in Cityscapes [11]. A thorough comparison with the state-of-the-art methods and extensive ablation studies are also carried out to verify the effectiveness of each component in our framework.
4.1 Datasets
Cityscapes is one of the benchmarks for urban scene understanding, which is collected from 50 cities with varying scene layouts and weather conditions. The 5,000 finely-annotated images from this dataset are used in our study, which contains 2,975 training images, 500 validation images, and 1,525 test images. Each image with a resolution of 2048 1024. The GTA5 dataset is synthesized from
Table 1. A performance comparison of our method with other state-of-the-art models on ”GTA5 to Cityscapes”. The performance is measured by the intersection-over-union (IoU) for each class and mean IoU (mIoU). Two base architectures, i.e., VGG16 (V) and ResNet101 (R) are used in our study.
Source only R 75.8 16.8 77.2 12.5 21.0 25.5 30.1 20.1 81.3 24.6 70.3 53.8 26.4 49.9 17.2 25.96.525.3 36.036.6 SIBAN [34] R 88.5 35.4 79.5 26.3 24.3 28.5 32.5 18.3 81.2 40.0 76.5 58.1 25.8 82.6 30.3 34.43.421.6 21.542.6 CLAN [35] R 87.0 27.1 79.6 27.3 23.3 28.3 35.5 24.2 83.6 27.4 74.2 58.6 28.0 76.2 33.1 36.76.731.9 31.443.2 DISE [3] R 91.5 47.5 82.5 31.3 25.6 33.0 33.7 25.8 82.7 28.8 82.7 62.4 30.8 85.2 27.7 34.56.425.2 24.445.4 IntraDA [38] R 90.6 37.1 82.6 30.1 19.1 29.5 32.4 20.6 85.7 40.5 79.7 58.7 31.1 86.3 31.5 48.30.030.2 35.846.3 BDL [28] R 91.0 44.7 84.2 34.6 27.6 30.2 36.0 36.0 85.0 43.6 83.0 58.6 31.6 83.3 35.3 49.73.328.8 35.648.5 CrCDA [22] R 92.4 55.3 82.3 31.2 29.1 32.5 33.2 35.6 83.5 34.8 84.2 58.9 32.2 84.7 40.6 46.12.131.1 32.748.6 SIM [51] R 90.6 44.7 84.8 34.3 28.7 31.6 35.0 37.6 84.7 43.3 85.3 57.0 31.5 83.8 42.6 48.51.930.4 39.049.2 Kim et al. [25] R 92.9 55.0 85.3 34.2 31.1 34.9 40.7 34.0 85.2 40.1 87.1 61.0 31.1 82.5 32.3 42.90.3 36.4 46.150.2 FDA-MBT [54] R 92.5 53.3 82.4 26.5 27.6 36.4 40.6 38.9 82.3 39.8 78.0 62.6 34.4 84.9 34.1 53.1 16.9 27.7 46.4 50.45
Ours R 90.8 41.4 84.7 35.1 27.5 31.2 38.0 32.8 85.6 42.1 84.9 59.6 34.4 85.0 42.8 52.73.430.9 38.149.5
Source only V 26.0 14.9 65.15.512.98.9 6.0 2.570.02.947.0 24.50.040.0 12.11.5 0.0 0.0 0.0 17.9 SIBAN [34] V 83.4 13.0 77.8 20.4 17.5 24.6 22.89.681.3 29.6 77.3 42.7 10.9 76.0 22.8 17.95.714.22.0 34.2 ASN [45] V 87.3 29.8 78.6 21.1 18.2 22.5 21.5 11.0 79.7 29.6 71.3 46.86.580.1 23.0 26.90.010.60.3 35.0 CyCADA [1] V 85.2 37.2 76.5 21.8 15.0 23.8 22.9 21.5 80.5 31.3 60.7 50.59.076.9 17.1 28.24.5 9.8 0.0 35.4 CLAN [35] V 88.0 30.6 79.2 23.4 20.5 26.1 23.0 14.8 81.6 34.5 72.0 45.87.980.5 26.6 29.90.010.70.0 36.6 CrDoCo [9] V 89.1 33.2 80.1 26.9 25.0 18.3 23.4 12.8 77.0 29.1 72.4 55.1 20.2 79.9 22.3 19.51.020.1 18.738.1 CrCDA [22] V 86.8 37.5 80.4 30.7 18.1 26.8 25.3 15.1 81.5 30.9 72.1 52.8 19.0 82.1 25.4 29.2 10.1 15.83.7 39.1 BDL [28] V 89.2 40.9 81.2 29.1 19.2 14.2 29.0 19.6 83.7 35.9 80.7 54.7 23.3 82.7 25.8 28.02.325.7 19.941.3 FDA-MBT [54] V 86.1 35.1 80.6 30.8 20.4 27.5 30.0 26.0 82.1 30.3 73.6 52.5 21.7 81.7 24.0 30.5 29.9 14.6 24.042.2 Kim et al. [25] V 92.5 54.5 83.9 34.5 25.5 31.0 30.4 18.0 84.1 39.6 83.9 53.6 19.3 81.7 21.1 13.6 17.7 12.36.5 42.3 SIM [51] V 88.1 35.8 83.1 25.8 23.9 29.2 28.8 28.6 83.0 36.7 82.3 53.7 22.8 82.3 26.4 38.6 0.019.6 17.142.4
Ours V 90.1 41.2 82.2 30.3 21.3 18.3 33.5 23.0 84.1 37.5 81.4 54.2 24.3 83.0 27.6 32.08.1 29.7 26.9 43.6
the game Grand Theft Auto V (GTA5), including a total of 24,966 labeled images whose annotations are compatible with Cityscapes. The resolution of each image is 1914 1052. The SYNTHIA-RAND-CITYSCAPES (or SYNTHIA for short) contains 9,400 pixel-level annotated images (1280
760), which are synthesized from a virtual city. Following the same setting reported in the previous studies, we use the labeled SYNTHIA or GTA5 dataset as the source domain, while using the unlabeled training dataset in the CITYSCAPES as the target domain. Only the 500 labeled validation images from CITYSCAPES are used as test data in all of our experiments.
4.2 Network Architecture
We use two segmentation baseline models, i.e., FCN-VGG16 and DeepLab-ResNet101 to investigate the effectiveness and generalizability of our framework. Specifically, FCN-VGG16 is the combination of FCN-8s [32] and VGG16 [44], while DeepLab-ResNet101 is obtained by integrating DeepLab-V2 [6] into ResNet101 [20]. These two segmentation models share the same discriminator which has 5 convolution layers with channel number 64,128, 256, 512, 1. For each layer, a leaky ReLU parameterized by 0.2 is followed, except the last one. The kernel size and stride are set to 44 and 2, respectively. The reconstruction
Table 2. A performance comparison of our method with other state-of-the-art models on ”SYNTHIA to Cityscapes”. The performance is measured by the IoU for each class and mIoU. Two base architectures, i.e., VGG16 (V) and ResNet101 (R) are used in our study.
model follows the architecture in [24], containing 3 convolution layers (kernel 33 and stride 1), 9 ResNet blocks (kernel 3
3 and stride 2), and another 3 transposed convolution layers (kernel 3
3 and stride 2) for upsampling. The 3 multi-scale discriminators share the identical network, each of which follows the architecture of PatchGAN [23]. More details regarding the architecture of discriminators in both segmentation and reconstruction models can be found in the Supplementary.
4.3 Implementation Details
Our framework is implemented with PyTorch [39] on two TITAN Xp GPUs, each of which with 12GB memory. The batch size is set to one for training all the models discussed above. Limited by the GPU memory space, the translation network is first trained to perform target-to-source image translation by using Adam optimizer [26]. The initial learning rate is set to 0.0001, which is reduced by half after every 100,000 iterations. We use momentum {0.5, 0.999} with weight decay 0.0001. The maximum training iteration is 1000k.
DeepLab-ResNet101 is trained using Stochastic Gradient Descent optimizer with initial learning rate 210
. The polynomial decay with power 0.9 is applied to the learning rate. The momentum and weight decay are set to 0.9 and 5
10
, respectively. For FCN-VGG16, the Adam optimizer with momentum {0.9, 0.99} and initial learning rate 1
is used for training. The learning rate
Table 3. Ablation study of the target-to-source translation and the reconstruction network. S S indicate source-to-target and target-to-source translation.
is decreased using step decay with step size 50000 and drop factor 0.1. In equation 1, is set to 1
for DeepLab-ResNet101 and 1
for FCN-VGG16.
The reconstruction network is first pre-trained by reconstructing source images from the corresponding labels
. We use the Adam optimizer with initial learning rate 2
10
and momentum {0.5, 0.999}, where the learning rate is linearly decreased to zero. In equation 6, we set
is set to 0.01 and 0.001 for FCN-VGG16 and DeepLab-ResNet101 respectively.
4.4 GTA5 Cityscapes
We carry out the adaptation from GTA5 to Cityscapes by following the same evaluation protocol as previously reported in [45,28]. The overall quantitative performance is assessed on 19 common classes (e.g., road, wall, and car) between these two domains. As shown in Table 1, we demonstrate competitive performance against ResNet101-based methods, but are inferior to two newly published models [25,54]. For the VGG16-based backbone, however, we are able to achieve the best results compared to existing state-of-the-art methods including [25,54]. Specifically, our method surpasses the source-only model (without adaptation) by 12.9% and 25.7% on ResNet101 and VGG16, respectively. Compared with CyCADA [1] and BDL [28] that rely on source-to-target translation, we demonstrate significant improvements (i.e., 8.2% and 2.3% on VGG16) by reducing image translation bias. CLAN [35] aims to enforce local semantic consistency by a category-level adversarial network. However, such a strategy fails to account for the global semantic consistency. Our reconstruction network shares a similar spirit with CLAN in terms of joint distribution alignment but enables us to enforce semantic consistency from a global view. As a consequence, we get 6.3% and 7.0% improvement on ResNet101 and VGG16, respectively.
4.5 SYNTHIA Cityscapes
We then evaluate our framework on the adaptation from SYNTHIA to Cityscapes based on 13 classes on ResNet101 and 16 classes on VGG16. The results exhibit
Fig. 5. Qualitative examples of semantic segmentation results in Cityscapes. For each target-domain image (first column), its ground truth and the corresponding segmentation prediction from the baseline model (source-to-target translation) are given. The following are predictions of our method by incorporating target-to-source translation and reconstruction, together with the reconstructed image.
Table 4. Ablation study of the temperature Cityscapes.
that our method outperforms other competing methods on average as shown in Table 2. Both ASN [45] and BDL [28] adapt output space in their models. However, simply aligning segmentation outputs may lead to negative transfer issue, owing to the dramatic differences of the layout and structure between SYNTHIA and Cityscapes. We achieve 6.4% and 1.7% improvement than ASN and BDL on ResNet101, respectively. It is noteworthy that we also outperform [54] on both ResNet101 and VGG16-based backbone.
4.6 Ablation Study
Target-to-source Translation and Reconstruction For GTA5 to Cityscapes, 0.6% improvement is achieved by considering target-to-source translation on ResNet101 compared to the source-to-target translation model (Table 3). By further enforce semantic consistency through a reconstruction network, our method achieves 49.5 mIoU. Similar improvements are also observed on VGG16, with 1.0% improvement by performing target-to-source translation. The prediction power of our method is further boosted by combining translation and reconstruc-
Table 5. Ablation study of the feature space vs. label space reconstruction.
tion, giving rise to another 1.3% mIoU improvement. The qualitative study of each module in our method is showcased in Figure 5.
Table 6. Ablation study of the reconstruction loss on GTA5 Cityscapes with VGG16 backbone.
achieve a performance boost of 0.6% and 1.1% by considering target-to-source translation on ResNet101 and VGG16, respectively (Table 3). The performance gain is 1.1% and 1.0% by incorporating the reconstruction network. Our results prove the effec-tiveness of target-to-source translation and reconstruction in adapting domain knowledge for semantic segmentation.
eter in this section and find that
= 0.001 achieves the best performance (Table 4). Therefore,
is set to 0.001 in all of our experiments to approximate semantic label maps.
feature space reconstruction based on the VGG16-based backbone. Table 5 highlights the benefits of our label-driven reconstruction that enforces semantic consistency of target images and their predicted labels.
ceptual loss and discriminator feature matching loss (equation 5) in maintaining semantic consistency.
We propose a novel framework that exploits cross-domain adaptation in the context of semantic segmentation. Specifically, we translate images from the target domain to the source domain to reduce image translation bias and the computational cost. To enforce cross-domain features with the same category close to each other, we reconstruct both source and target images directly from the label space. Experiments demonstrate that our method achieves significant improvement in adapting from GTA5 and SYNTHIA to Cityscapes.
Acknowledgments This work was partially supported by US National Science Foundation IIS-1718853, the CAREER grant IIS-1553687 and Cancer Prevention and Research Institute of Texas (CPRIT) award (RP190107).
1. Cycada: Cycle consistent adversarial domain adaptation. In: International Conference on Machine Learning (ICML) (2018)
2. Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., Erhan, D.: Domain separation networks. In: Advances in neural information processing systems (NIPS). pp. 343–351 (2016)
3. Chang, W.L., Wang, H.P., Peng, W.H., Chiu, W.C.: All about structure: Adapting structural information across domains for boosting semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1900–1909 (2019)
4. Chen, L.C., Collins, M.D., Zhu, Y., Papandreou, G., Zoph, B., Schroff, F., Adam, H., Shlens, J.: Searching for efficient multi-scale architectures for dense image prediction. In: Advances in neural information processing systems (NIPS) (2018)
5. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected crfs. In: International Conference on Learning Representations (ICLR) (2014)
6. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence (TPAMI) 40(4), 834–848 (2018)
7. Chen, Y., Li, W., Chen, X., Gool, L.V.: Learning semantic segmentation from synthetic data: A geometrically guided input-output adaptation approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1841–1850 (2019)
8. Chen, Y., Li, W., Van Gool, L.: Road: Reality oriented adaptation for semantic segmentation of urban scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 7892–7901 (2018)
9. Chen, Y.C., Lin, Y.Y., Yang, M.H., Huang, J.B.: Crdoco: Pixel-level domain transfer with cross-domain consistency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1791–1800 (2019)
10. Choi, J., Kim, T., Kim, C.: Self-ensembling with gan-based data augmentation for domain adaptation in semantic segmentation. In: Proceedings of the IEEE international conference on computer vision (ICCV) (2019)
11. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 3213–3223 (2016)
12. Dai, J., He, K., Sun, J.: Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision (ICCV). pp. 1635–1643 (2015)
13. Du, L., Tan, J., Yang, H., Feng, J., Xue, X., Zheng, Q., Ye, X., Zhang, X.: Ssfdan: Separated semantic feature based domain adaptation network for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision (ICCV) (2019)
14. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascalnetwork.org/challenges/VOC/voc2012/workshop/index.html
15. Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: International Conference on Machine Learning (ICML) (2015)
16. Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D., Li, W.: Deep reconstruction-classification networks for unsupervised domain adaptation. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 597–613 (2016)
17. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision (ICCV). pp. 1440–1448 (2015)
18. Goodfellow, I.: Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160 (2016)
19. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in neural information processing systems (NIPS). pp. 2672–2680 (2014)
20. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770–778 (2016)
21. Hoffman, J., Wang, D., Yu, F., Darrell, T.: Fcns in the wild: Pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649 (2016)
22. Huang, J., Lu, S., Guan, D., Zhang, X.: Contextual-relation consistent domain adaptation for semantic segmentation. Proceedings of the European Conference on Computer Vision (ECCV) (2020)
23. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1125–1134 (2017)
24. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 694–711 (2016)
25. Kim, M., Byun, H.: Learning texture invariant representation for domain adaptation of semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 12975–12984 (2020)
26. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2014)
27. Lee, K.H., Ros, G., Li, J., Gaidon, A.: Spigan: Privileged adversarial learning from simulation. In: International Conference on Learning Representations (ICLR) (2019)
28. Li, Y., Yuan, L., Vasconcelos, N.: Bidirectional learning for domain adaptation of semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
29. Lian, Q., Lv, F., Duan, L., Gong, B.: Constructing self-motivated pyramid curriculums for cross-domain semantic segmentation: A non-adversarial approach. In: Proceedings of the IEEE international conference on computer vision (ICCV) (2019)
30. Liu, M.Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. In: Advances in neural information processing systems (NIPS). pp. 700–708 (2017)
31. Liu, Z., Li, X., Luo, P., Loy, C.C., Tang, X.: Semantic image segmentation via deep parsing network. In: Proceedings of the IEEE international conference on computer vision (ICCV). pp. 1377–1385 (2015)
32. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 3431–3440 (2015)
33. Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning transferable features with deep adaptation networks. In: International Conference on Machine Learning (ICML) (2015)
34. Luo, Y., Liu, P., Guan, T., Yu, J., Yang, Y.: Significance-aware information bottleneck for domain adaptive semantic segmentation. In: Proceedings of the IEEE international conference on computer vision (ICCV) (October 2019)
35. Luo, Y., Zheng, L., Guan, T., Yu, J., Yang, Y.: Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2507–2516 (2019)
36. Murez, Z., Kolouri, S., Kriegman, D., Ramamoorthi, R., Kim, K.: Image to image translation for domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 4500–4509 (2018)
37. Murez, Z., Kolouri, S., Kriegman, D., Ramamoorthi, R., Kim, K.: Image to image translation for domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). vol. 13 (2018)
38. Pan, F., Shin, I., Rameau, F., Lee, S., Kweon, I.S.: Unsupervised intra-domain adaptation for semantic segmentation through self-supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 3764–3773 (2020)
39. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017)
40. Pinheiro, P.O., Collobert, R.: From image-level to pixel-level labeling with convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1713–1721 (2015)
41. Richter, S.R., Vineet, V., Roth, S., Koltun, V.: Playing for data: Ground truth from computer games. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 102–118 (2016)
42. Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.M.: The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 3234–3243 (2016)
43. Sankaranarayanan, S., Balaji, Y., Jain, A., Lim, S.N., Chellappa, R.: Learning from synthetic data: Addressing domain shift for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
44. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR) (2015)
45. Tsai, Y.H., Hung, W.C., Schulter, S., Sohn, K., Yang, M.H., Chandraker, M.: Learning to adapt structured output space for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
46. Tzeng, E., Hoffman, J., Darrell, T., Saenko, K.: Simultaneous deep transfer across domains and tasks. In: Proceedings of the IEEE international conference on computer vision (ICCV). pp. 4068–4076 (2015)
47. Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
48. Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014)
49. Vu, T.H., Jain, H., Bucher, M., Cord, M., P´erez, P.: Dada: Depth-aware domain adaptation in semantic segmentation. In: Proceedings of the IEEE international conference on computer vision (ICCV) (2019)
50. Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
51. Wang, Z., Yu, M., Wei, Y., Feris, R., Xiong, J., Hwu, W.m., Huang, T.S., Shi, H.: Differential treatment for stuff and things: A simple unsupervised domain adaptation method for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 12635–12644 (2020)
52. Wu, Z., Han, X., Lin, Y.L., Gokhan Uzunbas, M., Goldstein, T., Nam Lim, S., Davis, L.S.: Dcan: Dual channel-wise alignment networks for unsupervised scene adaptation. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 518–534 (2018)
53. Yang, J., An, W., Yan, C., Zhao, P., Huang, J.: Context-aware domain adaptation in semantic segmentation. arXiv preprint arXiv:2003.04010 (2020)
54. Yang, Y., Soatto, S.: Fda: Fourier domain adaptation for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 4085–4095 (2020)
55. Ying, W., Zhang, Y., Huang, J., Yang, Q.: Transfer learning via learning to transfer. In: International Conference on Machine Learning (ICML). pp. 5072–5081 (2018)
56. Zhang, Y., David, P., Gong, B.: Curriculum domain adaptation for semantic segmentation of urban scenes. In: Proceedings of the IEEE international conference on computer vision (ICCV) (2017)
57. Zhang, Y., Wei, Y., Wu, Q., Zhao, P., Niu, S., Huang, J., Tan, M.: Collaborative unsupervised domain adaptation for medical image diagnosis. IEEE Transaction on Image Processing (TIP) (2020)
58. Zhang, Y., Qiu, Z., Yao, T., Liu, D., Mei, T.: Fully convolutional adaptation networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6810–6818 (2018)
59. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2881–2890 (2017)
60. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision (ICCV) (2017)
61. Zhu, X., Zhou, H., Yang, C., Shi, J., Lin, D.: Penalizing top performers: Conservative loss for semantic segmentation adaptation. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 568–583 (2018)
62. Zou, Y., Yu, Z., Kumar, B.V., Wang, J.: Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 297–313. Springer (2018)
1.1 Target-to-source Translation
Inspired by image-to-image translation networks [24,14], existing domain adaptation methods [1,17,13] translate images from the source domain to the target domain (source-to-target) to reduce pixel-level domain discrepancy. This is achieved by an unsupervised image translation model such as CycleGAN [24] to learn a mapping
:
. However, such strategy introduces inevitable bias to the translated images
), stemming from that
and
cannot be guaranteed to follow the exactly identical distribution through the adversarial learning [8]. This problem can get even worse in the source-to-target translation, as
in most of domain adaptation problems. For example, GTA5 [19] (i.e.,
) contains 24,966 images, while Cityscapes (i.e.,
[7] has only 2,975 images. As a consequence,
) contains massive amounts of translation bias which will further induces negative effects when adapting domain knowledge between
) and
. To alleviate this problem, for the first time, we translate images from the target domain to the source domain through the mapping
(where F is the reverse function of
).
and
) are then used for further domain knowledge transfer. Doing so significantly reduces the translation bias in the translated images
) and is much more computationally efficient than the source-to-target translation. Another advantage is that the segmentation network can be trained using original source images
and their corresponding labels.
1.2 Network Architecture
The three multi-scale discriminators (i.e., , and
) used in our reconstruction model follow the identical network architecture. Each of them is a 70
70 PatchGAN [10] containing five convolution layers with kernel number {64, 128, 256, 512, 1}. The kernel size for each layer is 4
4. The first three layers use stride 2, while the last two layers with stride 1. A leaky ReLU parameterized by 0.2 is applied to the first four layers. We also apply BatchNorm to each layer, except the first and last one.
1.3 Implementation Details
For multi-scale discriminators, Adam optimizer with initial learning rate 2and momentum { 0.5, 0.999} are used in our study. The learning rate is linearly decayed to zero with step size 100.
1.4 mIoU Gap
We investigate our model’s ability in narrowing the mIoU gap between the model (Oracle) that is trained in a fully-supervised matter. Compared to existing state-of-the-art methods, we significantly recover the performance loss based
Table 1. A performance comparison of our method with other state-of-the-art models on ”GTA5 to Cityscapes”. The performance is measured by the mIoU gap between each model and the fully-supervised model (Oracle). Two base architectures, i.e., VGG16 (V) and ResNet101 (R) are used in our study.
on the ResNet101 backbone on GTA5 to Cityscapes. Although we are inferior to [11,23] which are published simultaneously with our work, we outperforms these two methods on VGG16-based backbone by a large margin (Table 1). Similar improvements can also be observed on the adaptation from SYNTHIA to Cityscapes as shown in Table 2.
1.5 Qualitative Comparison
As shown in Figure 1, we present the qualitative comparison in Cityscapes based on the VGG16 model from GTA5
Cityscapes. Our results reveal the effectiveness of target-to-source translation and joint distribution alignment in adapting cross-domain knowledge.
The qualitative comparison for ResNet101 and VGG16 model from SYNTHIA
Cityscapes are showcased in Figure 2 and Figure 3, respectively. Similarly, each component in our framework contributes to the overall performance improvement.
Table 2. A performance comparison of our method with other state-of-the-art models on ”SYNTHIA to Cityscapes”. The performance is measured by the mIoU gap between each model and the fully-supervised model (Oracle). Two base architectures, i.e., VGG16 (V) and ResNet101 (R) are used in our study.
Fig. 1. Qualitative examples of semantic segmentation results in Cityscapes (GTA5Cityscapes with VGG16). For each target-domain image (first column), its ground truth and the corresponding segmentation output from the baseline model (source-to-target) are given. The following are predictions of our method by incorporating target-to-source translation and reconstruction, together with the reconstructed image.
Fig. 2. Qualitative examples of semantic segmentation results in Cityscapes (SYNTHIACityscapes with ResNet101). For each target-domain image (first column), its ground truth and the corresponding segmentation output from the baseline model (source-to-target) are given. The following are predictions of our method by incorporating target-to-source translation and reconstruction, together with the reconstructed image.
Fig. 3. Qualitative examples of semantic segmentation results in Cityscapes (SYNTHIACityscapes with VGG16). For each target-domain image (first column), its ground truth and the corresponding segmentation output from the baseline model (source-to-target) are given. The following are predictions of our method by incorporating target-to-source translation and reconstruction, together with the reconstructed image.
1. Cycada: Cycle consistent adversarial domain adaptation. In International Conference on Machine Learning (ICML), 2018.
2. Wei-Lun Chang, Hui-Po Wang, Wen-Hsiao Peng, and Wei-Chen Chiu. All about structure: Adapting structural information across domains for boosting semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1900–1909, 2019.
3. Yuhua Chen, Wen Li, Xiaoran Chen, and Luc Van Gool. Learning semantic segmentation from synthetic data: A geometrically guided input-output adaptation approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1841–1850, 2019.
4. Yuhua Chen, Wen Li, and Luc Van Gool. Road: Reality oriented adaptation for semantic segmentation of urban scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7892–7901, 2018.
5. Yun-Chun Chen, Yen-Yu Lin, Ming-Hsuan Yang, and Jia-Bin Huang. Crdoco: Pixel-level domain transfer with cross-domain consistency. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1791–1800, 2019.
6. Jaehoon Choi, Taekyung Kim, and Changick Kim. Self-ensembling with gan-based data augmentation for domain adaptation in semantic segmentation. In Proceedings of the IEEE international conference on computer vision (ICCV), 2019.
7. Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3213–3223, 2016.
8. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems (NIPS), pages 2672–2680, 2014.
9. Jiaxing Huang, Shijian Lu, Dayan Guan, and Xiaobing Zhang. Contextual-relation consistent domain adaptation for semantic segmentation. Proceedings of the European Conference on Computer Vision (ECCV), 2020.
10. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1125– 1134, 2017.
11. Myeongjin Kim and Hyeran Byun. Learning texture invariant representation for domain adaptation of semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12975– 12984, 2020.
12. Kuan-Hui Lee, German Ros, Jie Li, and Adrien Gaidon. Spigan: Privileged adver-
13. Yunsheng Li, Lu Yuan, and Nuno Vasconcelos. Bidirectional learning for domain
14. Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In Advances in neural information processing systems (NIPS), pages 700–708, 2017.
15. Yawei Luo, Ping Liu, Tao Guan, Junqing Yu, and Yi Yang. Significance-aware information bottleneck for domain adaptive semantic segmentation. In Proceedings of the IEEE international conference on computer vision (ICCV), October 2019.
16. Yawei Luo, Liang Zheng, Tao Guan, Junqing Yu, and Yi Yang. Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2507–2516, 2019.
17. Zak Murez, Soheil Kolouri, David Kriegman, Ravi Ramamoorthi, and Kyungnam Kim. Image to image translation for domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 13, 2018.
18. Fei Pan, Inkyu Shin, Francois Rameau, Seokju Lee, and In So Kweon. Unsupervised intra-domain adaptation for semantic segmentation through self-supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3764–3773, 2020.
19. Stephan R Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun. Playing for data: Ground truth from computer games. In Proceedings of the European Conference on Computer Vision (ECCV), pages 102–118, 2016.
20. Yi-Hsuan Tsai, Wei-Chih Hung, Samuel Schulter, Kihyuk Sohn, Ming-Hsuan Yang,
21. Tuan-Hung Vu, Himalaya Jain, Maxime Bucher, Matthieu Cord, and Patrick P´erez.
22. Zhonghao Wang, Mo Yu, Yunchao Wei, Rogerio Feris, Jinjun Xiong, Wen-mei Hwu, Thomas S Huang, and Honghui Shi. Differential treatment for stuff and things: A simple unsupervised domain adaptation method for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12635–12644, 2020.
23. Yanchao Yang and Stefano Soatto. Fda: Fourier domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4085–4095, 2020.
24. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-