Weakly-supervised learning [19, 38] focuses on learning a pixel-level lesion segmentation model for medical images with only weakly-labeled (image-level) annotations. Due to the slight requirements for large-scale, high-quality fully-labeled (pixel-level) annotations, it has been widely-explored in a number of medical diagnosis tasks, e.g., automated glaucoma detection [43], thoracic disease localization [39], histopathology segmentation [19], etc.
However, weakly-supervised learning is a huge challenge for semantic lesions segmentation since 1) effective constraints or domain expertise are needed to learn the internal representation related to image-level annotations, which can produce inaccurate and coarse lesion regions; 2) it ignores the strong probabilistic dependencies between target lesions segmentation task and well-to-annotated source diseases, where such dependencies are treated as semantic knowledge. For example, diseases detected by both gastroscope and enteroscopy tend to share similar appearances,
and further have similar prior distributions. Based on such dependencies, in this paper, we explore how to transfer semantic knowledge from closely-related fully-annotated source dataset (e.g., gastroscope images) to weakly-labeled target dataset (e.g., enteroscopy images). To take advantage of the semantic transferable knowledge, we propose a new weakly-supervised semantic lesions representation transfer model as shown in Figure 1, and its goal is to learn the transferable semantic knowledge from fully-labeled source diseases dataset to improve the segmentation performance on target weakly-labeled lesions segmentation task. The core idea of our model is a pseudo pixel-label generator, which can leverage seed information by incorporating class balance with super-pixel prior [1] to further prevent the dominance of well-to-transfer categories. The hard-to-transfer samples can be incrementally introduced from the target dataset into training set. Afterwards, to mitigate the mapping features gaps of same class among source and target datasets, we endeavor to learn transferable knowledge by aligning the dynamically-searched feature centroids, which are gradually reckoned with previously-learned features and highly-confident pseudo labels. Meanwhile, adversarial learning is utilized in the output space to drive the segmentation outputs of the source and target datasets to share closer global distribution. Finally, we conduct the experiments on our built medical endoscopic dataset and several benchmark datasets to justify the superiority of our model. The experimental results can strongly support the effectiveness of our proposed model. The contributions of our work are as follows: • We develop a new semantic lesion representation transfer model for weakly-supervised lesions segmentation. To our best knowledge, this is an earlier exploration about semantic transfer for endoscopic lesions segmentation in the medical image analysis field.
• A pseudo pixel label generator is proposed to progressively mine more highly-confident pseudo labels, which can not only include more hard-to-transfer samples from the target dataset into training set, but also achieve class balance with super-pixel priors.
• A new medical endoscopic dataset with 3659 images collected from more than 1100 volunteers is built. We demonstrate the effectiveness of our model against several state-of-the-arts on our endoscopic dataset and several benchmark datasets.
In this section, we discuss some representative related works about semantic lesion segmentation and semantic representation transfer.
Semantic Lesion Segmentation: Computer aided diagnosis (CAD) [31, 9, 37, 7] is developed to assist clinician to improve the efficiency and accuracy of medical lesions segmentation. Traditional methods rely on local image features handcrafted by domain experts [18, 6]. To further improve the segmentation quality, most advanced methods [28, 20, 10] based on convolutional neural networks [14, 32, 4] are proposed, which can achieve state-of-the-arts performance but acquire lots of pixel-level annotations. Thus, weakly-supervised semantic lesions segmentation methods [19, 38] are proposed to save annotation efforts. However, there is currently still a large segmentation performance gap between models trained only with image tags and models trained with pixel annotations.
Semantic Knowledge Transfer: Learning the semantic transferable representation from source dataset to target dataset for classification task via generative adversarial network [13] has been widely-explored [23, 34, 35, 24, 15]. As pointed out in [42], methods addressing classification transfer do not translate well to the semantic segmentation task, which is still a significant challenge. Recently, Bousmalis et al. [2] propose to learn transferable knowledge via transferring the source images to target dataset. [42] utilizes a curriculum learning approach to mitigate the gap between source and target dataset. Several researches [16, 5, 15, 12, 33] focus on employing adversarial learning to semantic segmentation transfer in the feature space. [17] introduces an additional generator conditioned on the extra auxiliary information for target dataset. [44] exploits a self-training strategy for semantic representation transfer. However, existing models cannot be directly applied to semantic lesion transfer, since 1) they cannot ensure the features in same class but in different datasets are mapped nearby due to non-valid labeled information for target samples; 2) the model tends to transfer some easier-to-learned classes instead of balancing all the classes.
Therefore, we focus on learning semantic transferable knowledge by highly-confident class-balanced pseudo labels and dynamically-searched feature centroids with previously-learned experience.
In this section, we provide a brief overview about our semantic lesion representation transfer model. Then, the details about model formulation, training and testing procedures are elaborated.
3.1. Overview of Our Proposed Model
The overview architecture of our model is shown in Figure 2. Two subnets marked as and
are designed for classification and segmentation tasks, respectively, where the prediction of subnet
is refined by classification probability via convolution operation, as shown in dashed arrows of Figure 2. Suppose that the source dataset (e.g., gas-
Figure 2: Framework of our proposed model, where six components of our model include ResNet-50 network for feature extraction, adversarial learning for enforcing various lesion segmentation to share closer distribution, pseudo label generator for weakly-labeled enteroscopy dataset, semantic representation transfer loss for aligning feature centroids among source and target datasets, and two subnets denoted as
and
for classification
and segmentation
, respectively.
troscope images) and target dataset (e.g., enteroscopy images) are denoted as and
, respectively, where
and
are the corresponding image and pixel annotations of
, and
is the corresponding image annotation of
. We firstly forward image
of source dataset
to optimize the whole net- work excluding discriminator D. The segmentation output for image
of target dataset
is then predicted by subnet
. Since our goal is to encourage the segmentation outputs of source dataset
and target dataset
to share closer distribution, discriminator D takes these two predictions as the input to distinguish whether the input is from
or
.
Although we employ generative adversarial objective to narrow the gap of segmentation outputs between and
, it cannot ensure the features of same class in different datasets (i.e.,
and
) are mapped nearby. Inspired by this key observation, we endeavor to learn the semantic representation transfer by aligning the feature centroid for each class. However, we do not have pixel annotations as guidance to compute centroids for target dataset
. To address this issue, we propose a new method to generate pseudo pixel labels, which takes into account class balance and super-pixel segmentation priors. Based on the pseudo labels of target dataset, we utilize exponentially-weighted features based on previously-learned experience to compute semantic centroid for each class. Furthermore, the target image
assigned with pseudo pixel labels
is then for- warded into our model to fine-tuning the whole network.
3.2. Model Formulation
In order to learn transferable knowledge for target disease segmentation task, we formulate our proposed model as the following objective:
where and
are trade-off parameters and the definitions of each loss function are shown as follows:
Classification Loss represents the classification loss of both target and source datasets (e.g., gastroscope and enteroscopy datasets). The subnet
is utilized to discriminate whether the input image has lesion or not by the loss
:
where denotes the parameters of the subnet
.
and
are the classification softmax outputs for source and target datasets, respectively, and J(., .) is the typical cross-entropy loss.
Segmentation Loss : For the subnet
with softmax outputs,
can be formulated as the segmentation loss for dataset
with supervised pixel annotation
, and dataset
with assigned pseudo pixel label
. It can then be formulated as:
where denotes the parameters of
and
are the segmentation softmax outputs of sub- net
at pixel
) and
), respectively.
denotes one-hot encoding of ground truth label for the a-th pixel position in image
, and
is as- signed pseudo label for the b-th pixel position in image
. K and
are the number of classes and one-hot vector, respectively. Notice that assigning
as 0 can neglect this pseudo pixel label in training procedure. We thus expect the
-norm regularization on
can serve as a negative sparse constraint to prevent the trivial solution from ignoring all pseudo pixel labels.
is a global weight to control the amount of selected pseudo labels, and a larger
can promote the selection of more pseudo labels for model training.
Similar to self-paced learning [21], Eq. (3) in our model can iteratively produce pseudo pixel labels corresponding to large confidence. However, the optimization of the second term in Eq. (3) can result in two issues: (i) our model will tend to be biased towards initially easily-learned classes and neglect other hard-to-transfer classes in the training procedure; (ii) the generated pseudo labels with highly-confident scores are spatially discrete. To address the issue (i), the second term in Eq. (3) can be formulated as Eq. (4) where class-wise confidence levels are normalized.
where ) are class balance parameters that determine the proportion of generated pseudo labels for each class k. In order to avert dominance of large amount of pixel classes, we develop a new method for the determination of
as summarized in Algorithm 1: after obtaining maximum predicted probability
of each pixel for all target images, we sort the probabilities of all pixels predicted as class
can be determined when
equals to the probability ranked at
. The value of p is starting from 25% and empirically added by 5% in each training epoch, and the maximum portion p is set as 55%. Furthermore, the optimal solution of Eq. (4) is:
To handle the issue (ii), the pseudo labels that are produced by Eq. (5) can be refined with super-pixel spatial priors [1], which ensures spatial continuity of generated pseudo labels. Moreover, Algorithm 2 presents the details about how to apply super-pixel spatial refinement for the assignment of pseudo labels : the super-pixel priors
is applied for each target image
. When the (h, w)-th pixel which has same spatial priors among its 8-neighborhoods has no valid pseudo labels, its pixel label can be decided via voting the pseudo labels of its 8-neighborhoods.
Adversarial Loss : To drive lesion seg-mentation outputs between
and
to share similar distribution, we utilize generative adversarial objective
in this paper. Discriminator D in Figure 2 takes these two segmentation softmax outputs of subnet
as input to distinguish whether the input is from
or
, and
is trained to fool D. Formally, it can be defined as:
where and
indi- cate the output of discriminator D for image
and
, respectively, and
indicates the parameters of discriminator D.
Semantic Transfer : To ensure that thefeatures of same class in different datasets
and
are mapped nearby, is proposed for semantic representation transfer via feature centroid alignment, which can be defined as:
where and
are the centroids of the class k in datasets
and
, respectively.
is a trade-off parameter. Considering that the centroids of same class in different datasets have similar sparse property, we utilize the second term of Eq. (7). Specifically, motivated by exponential reward design in reinforcement learning [22, 25], we propose a new method to search centroids for each class based on exponentially-weighted previously learned features, which resort to history learned experience. Furthermore, pseudo
labels generated by Algorithm 2 are used to guide semantic alignment for dataset . The details of computing centroid for each class are shown in Algorithm 3.
Instead of aligning those newly obtained centroids in each iteration directly, we propose to align the centroids via resorting previously-leaned experience to overcome two practical limitations: 1) Categorical information in each batch is often insufficient, e.g., it is possible that some classes are missing in the current training batch since the samples are randomly selected; 2) If the batch size is small, even one false pseudo label will lead to the enormous deviation between the true centroid and pseudo-labeled centroid.
3.3. Details of Network Architecture
Baseline, Subnet and
: We utilize DeepLab-v3 [4] architecture based on ResNet-50 [14] as the backbone network, which is pre-trained with ImageNet [11]. For the ResNet-50 [14], we remove the last classification layer and modify the stride of the last two convolutional blocks from 2 to 1 for higher dimensional output. Moreover, three dilated convolutional filters with stride of {1, 2, 4} are utilized in the last convolutional block to enlarge receptive field. As shown in Figure 2, the output feature map generated by baseline ResNet-50 is passed into subnet
for image clas-sification. It is forwarded into subnet
as well for pixel segmentation, which contains an Atrous Spatial Pyramid Pooling(ASPP) [3] block and a pixel classifier layer.
Algorithm 3 Optimizing Semantic Representation Transfer Loss
Input: Max-iteration N, classes number K, the feature centroids and
of each class k for
and
;
; 4:
via Algorithm 2; 5: Extracting pixel feature maps
and
by subnet
10: ; (Exponentially-weighted) 11: end for
Discriminator (D): Inspired by [26], for the discriminator D, we employ a fully convolutional networks for retaining global information compared with multi-layer perception. It consists of 5 convolutional layers with stride of 2 and kernel of 3. In more detail, the channels of 5 convolutional filters are {16, 32, 64, 64, 1}, respectively. Excluding the last convolution layer, the activation function of each filter is Leaky RELU with the parameter as 0.2.
3.4. Training and Testing
Training: In each training step, for losses and
, we firstly forward the source image
(e.g., gastroscope) with the image-level label
and the pixel-level annotation
to the network and generate the segmentation softmax output
. We then obtain the target softmax output
for image
(e.g., enteroscopy) only with the image-level label
, and ulti- mate pseudo pixel labels
are generated via Algorithm 2. In addition, these two segmentation outputs are passed into discriminator D for optimizing
. For training the objective
, the centroids
and
for each class k are computed via Algorithm 3, which resorts to previously learned features.
Testing: In testing phase, a target image (e.g., en- teroscopy) is passed into feature extractor ResNet-50 followed by subnet
and
for classification and segmentation. The discriminator D and other algorithmic designs would not be involved. As for implementation details, we use a single Titan XP GPU with 12 GB memory. The Adam
Table 1: Comparison performance between our proposed model and the state-of-the-arts on our medical dataset. Models with the best performance are bolded.
optimizer is used to train whole networks with the batch size as 4. The initial learning rate is set as and it is exponential decay with the rate and step size as 0.7 and 950, respectively.
In this section, we give detailed descriptions about our built dataset, and both source code and built dataset are available at http://ai.sia.cn/lwfb/. Although our model is mainly designed for medical image analysis, the experiments on other benchmark datasets are also conducted to validate its generalization performance.
4.1. Dataset and Evaluation
The datasets in our experiments include one our own medical dataset, and three benchmark datasets.
Medical Endoscopic Dataset: this dataset is built by ourself, which has total 3659 images that collected from more than 1100 volunteers with various lesions, including gastritis, polyp, cancer, bleeding and ulcer. Specifically, it contains 2969 gasteroscope images and 690 enteroscopy images. In the training phase, we treat the gasteroscope images as the source dataset, whose 2400 images have the image-level labels and 569 images have both image-level labels and pixel-level annotations; enteroscopy images are treated as target dataset, whose 300 images are with their image-level labels. For the test phase, the other 390 enteroscopy images are utilized to evaluate the performance.
Cityscapes [8] is a real-world dataset about urban street scenes, which is collected in 50 cities. It consists of three disjoint subsets: training subset with 2993 images, validation subset with 503 images and test subset with 1531 images. There are total 34 distinct categories in the dataset.
GTA [27] contains 24996 images w.r.t synthetic street scenes, which are collected from realistic computer game Grand Theft Auto V based on the city of Los Angeles. The segmentation annotations are compatible with the Cityscapes dataset [8].
SYNTHIA [29] is a large synthetic dataset whose images are collected in virtual city without corresponding to any real city. For the experiments, we use its subset called SYNTHIA-RANDCITYSCAPES with 9400 images, including 12 automatically annotated object categories and some unnamed classes.
For the evaluation, we use intersection over union (IoU)
as basic metric. Additionally, three derived metrics are also used, i.e., IoU of normal (), IoU of disease (
) and mean IoU (mIoU). The larger of the corresponding metric is, the better of the corresponding model will be.
4.2. Experiments on Medical Endoscopic Dataset
In this subsection, we validate the superiority of our model by comparing it with several state-of-the-arts on our built medical dataset:
• Baseline (BL) model utilizes DeepLab-v3 [4] as backbone for segmentation without semantic transfer.
• Constrained Deep Weak Supervision (CDWS) [19] exploits multi-scale learning with weak supervision by applying area constraint for segmentation predictions.
• No More Discrimination (NMD) [5] refines segmentation module by leveraging soft pseudo labels and static object priors with multiple class-wise adaptation.
• FCNs in the Wild (Wild) [16] designs a adversarial loss with prior constraint on pixel-level output to optimize intermediate convolutional layers.
• Discriminative Feature Network (DFN) [40] designs both Smooth Network and Border Network to learn discriminative semantic feature.
• Learning to Adapt (LtA) [33] exploits multi-level adaptation in the context of semantic segmentation.
• Conditional GAN (CGAN) [17] proposes to integrate conditional GAN into the segmentation network for feature space adaptation. For a fair comparison, we use ResNet-50 [14] as the backbone architecture and add an additional classification head to refine segmentation in this experiment. The evaluation results of our model against state-of-the-arts are presented in Table 1. As shown in Table 1, we have the following observations: 1) Compared with the state-of-the-arts [33, 17], our proposed model outperforms them by a large margin around 2.543.14%, which validates the effectiveness of our model, i.e., a pseudo label generator can mine more accurate and highly-confident pseudo labels. 2) As for mIoU, all models [5, 16, 40, 33, 17] with semantic transfer outperform the models [4, 19] without semantic transfer. Ablation Study: To validate the effectiveness of different components of our model, we also conduct experiment on our medical dataset with different components ablation, i.e., Baseline network DeepLab-v3 (BL), Adversarial Learning (AL), Pseudo Labels (PL) and Semantic Representation Transfer (SRT). As the results shown in Table 2,
Figure 3: Visualization of the learned representations using t-SNE [36], where blue and red points are source gasteroscope samples and target enteroscopy samples, respectively. Two separated clusters denote two categories, i.e., lesion and normal.
Table 2: Ablation study and different pseudo labels designs of our model on medical dataset with Baseline network DeepLab-v3 [4] (BL), Adversarial Learning (AL), Pseudo Labels (PL), Semantic Representation Transfer (SRT) and training without pseudo labels (Ours-woPL), class balance (Ours-woCB) or super-pixel spatial priors (Ours-woSP).
we can observe that when one or more components are removed, the performance degrades, e.g., the performance decreases 0.38%4.42% in terms of mIoU after removing the pseudo labels selection or semantic representation transfer. In addition, we also demonstrate the learned transferable representations in Figure 3. Notice that our model can well map the features of same class in different datasets nearby along the learning process when compared with Baseline (Figure 3 (a)) and Adversarial Learning (Figure 3 (b)), which validates that highly-confident pseudo pixel labels and previously-learned feature can further improve the performance for enteroscopy lesions segmentation.
Effect of Pseudo Labels Selection: We intend to study how different designs for pseudo labels selection affect the performance of our model, i.e., training without pseudo labels (denoted as Ours-woPL), training without class balance (denoted as Ours-woCB) and training without super-pixel spatial priors (denoted as Ours-woSP). As the results shown in Table 2, our model which is only with class balance can achieve 1.61% improvement when comparing with OurswoPL, while the training model with both class balance and super-pixel spatial priors can improve 2.27%. This observation indicates that the pseudo labels component is designed reasonably. In addition, as depicted in Figure 4, the pseudo pixel label generator can iteratively generate more highly-confident pseudo pixel labels by incorporating class balance and super-pixel spatial prior.
Effect of Hyper-Parameters: This subsection investigates the effect of parameters and
. As the results illustrated in Figure 5, we can choose the optimal
and
by empirically conducting extensive parameter experiments. Notice that the performance of our model has great stability when tuning the value of different parameters. Moreover, it also validates the importance of incorporating previously-learned features and sparsity property of medical endoscopic dataset.
Figure 4: The illustration of intuitive propagation of pseudo labels, where input images are from enteroscopy dataset.
4.3. Experiments on Benchmark Datasets
In this subsection, we conduct experiments on several benchmark datasets that has compatible annotations with each other to further justify the effectiveness of our model. For a fair comparison, we remove the classification head and adopt the same experimental data configuration with the completing methods [16, 42, 15, 30, 33, 17]. For the ablation studies shown in Table 3 and Table 4, BL, AL, PL, SRT and Ours-woSP indicate baseline, adversarial learning, pseudo labels, semantic lesions transfer components of our model and training without super-pixel priors, respectively.
Transfer from SYNTHIA to Cityscapes: In this experiment, our model is used to learn transferable knowledge from SYNTHIA [29] to Cityscapes [8]. For the training phase, SYNTHIA dataset with finely-annotated 9400 im-
Table 3: Comparisons performance of learning transferable knowledge from SYNTHIA dataset to Cityscapes dataset. Models with best and runner-up performance are marked with red and blue colors, respectively.
Table 4: Comparison performance of learning transferable representation from GTA dataset to Cityscapes dataset. Models with best and runner-up performance are marked with red and blue colors, respectively.
ages is regarded as . The Cityscapes without pixel labels has 2993 images is regarded as
. For the test, we use validation subset with 500 images of Cityscapes, which is disjoint with training subset. Notice that we consider 16 common classes for two datasets: road, sidewalk, building, wall, fence, pole, traffic light, traffic sign, vegetation, sky, person, rider, car, bus, motorbike and bike. From the presented results in Table 3, we can conclude that: 1) Our model outperforms state-of-the-arts [30, 17, 33] by
for the remaining classes in terms of mIoU, which verifies the effectiveness of our model; 2) Ablation studies of both PL, SRT and SP also validates these components are designed reasonably; 3) Although the appearances of the hard-to-transfer classes (e.g., wall, pole, motorbike and bike) are extremely different between these two datasets, our model can also achieve comparable performance.
Transfer from GTA to Cityscapes: When conducting experiments to learn transferable representation from GTA [27] to Cityscapes [8], in the training process, GTA with finely-annotated 24996 images and the training subset with 2993 images of Cityscapes without using pixel labels are treated as and
, respectively. The remaining validation subset with 500 images of Cityscapes is used for evaluation. As the results presented in Table 4, we consider 19 shared classes: road, sidewalk, building, wall, fence, traffic light, traffic sign, vegetation, terrain, sky, person, rider, car, truck, bus, train, motorbike and bike. Notice that: 1) Other semantic transfer models can be easily partial towards easy-to-transfer classes (e.g., road, building, sky, vegetation and car), while our model can achieve better performance for both initially hard-to-transfer classes and easy-to-transfer classes. 2) The ablation studies of PL, SRT and SP verify that previously-learned experience and pseudo labels play a significant role when comparing with [15, 30, 33, 17].
In this paper, we explore a new semantic lesions representation transfer model for weakly-supervised endoscopic lesions segmentation. More specifically, a pseudo pixel label generator is presented to progressively mine more samples from target data into training set, which incorporates super-pixel priors and class balance to prevent dominance of well-to-transfer categories. We also align the dynamically-searched feature centroids for each class of different datasets with previously-learned features. Experiments on our built dataset and several benchmark datasets show the effectiveness and superiority of our model.
Thanks for the medical endoscopic data provided by Prof. Yunsheng Yang from the Chinese PLA General Hospital.
[1] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk. Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2274–2282, Nov. 2012. 2, 4
[2] Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. Unsupervised PixelLevel Domain Adaptation with Generative Adversarial Networks. In In CVPR, 2017. 2
[3] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):834– 848, Apr. 2018. 5
[4] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV, 2018. 2, 5, 6, 7
[5] Yi-Hsin Chen, Wei-Yu Chen, Yu-Ting Chen, Bo-Cheng Tsai, Yu-Chiang Frank Wang, and Min Sun. No More Discrimination: Cross City Adaptation of Road Scene Segmenters. In ICCV, Oct 2017. 2, 6, 8
[6] Jie-Zhi Cheng, Yi-Hong Chou, Chiun-Sheng Huang, Yeun- Chung Chang, Chui-Mei Tiu, Kuei-Wu Chen, and ChungMing Chen. Computer-aided US Diagnosis of Breast Lesions by Using Cell-based Contour Grouping. Radiology, 255(3):746–754, June 2010. 2
[7] Yang Cong, Shuai Wang, Ji Liu, Jun Cao, Yunsheng Yang, and Jiebo Luo. Deep sparse feature selection for computer aided endoscopy diagnosis. Pattern Recognition, 48(3):907 – 917, 2015. 2
[8] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016. 6, 7, 8
[9] Jianan Cui, Haiqing Yu, Shuhang Chen, Yunmei Chen, and Huafeng Liu. Simultaneous estimation and segmentation from projection data in dynamic pet. Medical physics, 46(3):1245–1259, 2019. 2
[10] Avijit Dasgupta and Sonam Singh. A fully convolutional neural network based structured prediction approach towards the retinal vessel segmentation. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pages 248–251, Melbourne, Australia, Apr. 2017. IEEE. 2
[11] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR, 2019. 5
[12] Qi Dou, Cheng Ouyang, Cheng Chen, Hao Chen, and Pheng- Ann Heng. Unsupervised cross-modality domain adaptation of convnets for biomedical image segmentations with adversarial loss. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI’18, pages 691– 697. AAAI Press, 2018. 2
[13] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, pages 2672–2680, Cambridge, MA, USA, 2014. MIT Press. 2
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2015. 2, 5, 6
[15] Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. CyCADA: Cycle-consistent adversarial domain adaptation. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1989–1998. PMLR, 10–15 Jul 2018. 2, 7, 8
[16] Judy Hoffman, Dequan Wang, Fisher Yu, and Trevor Darrell. Fcns in the wild: Pixel-level adversarial and constraint-based adaptation. arXiv:1612.02649, 2016. 2, 6, 7, 8
[17] Weixiang Hong, Zhenzhen Wang, Ming Yang, and Junsong Yuan. Conditional generative adversarial network for structured domain adaptation. In CVPR, June 2018. 2, 6, 7, 8
[18] Karla Horsch, Maryellen L. Giger, Luz A. Venta, and Carl J. Vyborny. Automatic segmentation of breast lesions on ultrasound. Medical Physics, 28(8):1652–1659, Aug. 2001. 2
[19] Zhipeng Jia, Xingyi Huang, Eric I.-Chao Chang, and Yan Xu. Constrained Deep Weak Supervision for Histopathology Image Segmentation. IEEE Transactions on Medical Imaging, 36(11):2376–2388, Nov. 2017. 1, 2, 6
[20] Zhexin Jiang, Hao Zhang, Yi Wang, and Seok-Bum Ko. Reti- nal blood vessel segmentation using fully convolutional network with transfer learning. Computerized Medical Imaging and Graphics, 68:1–15, Sept. 2018. 2
[21] M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self- paced learning for latent variable models. In Advances in Neural Information Processing Systems, pages 1189–1197, 2010. 4
[22] Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In ICLR, 2016. 4
[23] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. Learning Transferable Features with Deep Adaptation Networks. In ICML, Feb. 2015. 2
[24] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. Unsupervised domain adaptation with residual transfer networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, pages 136–144, USA, 2016. Curran Associates Inc. 2
[25] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. Playing atari with deep reinforcement learning. In NIPS, 2013. 4
[26] Alec Radford, Luke Metz, and Soumith Chintala. Unsuper- vised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434, 2015. 5
[27] Stephan R. Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun. Playing for data: Ground truth from computer games. In ECCV, 2016. 6, 8
[28] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234– 241, Cham, 2015. Springer International Publishing. 2
[29] German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M. Lopez. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In CVPR, June 2016. 6, 7
[30] Swami Sankaranarayanan, Yogesh Balaji, Arpit Jain, Ser- Nam Lim, and Rama Chellappa. Learning from synthetic data: Addressing domain shift for semantic segmentation. In CVPR, June 2018. 7, 8
[31] Junji Shiraishi, Qiang Li, Daniel Appelbaum, and Ku- nio Doi. Computer-Aided Diagnosis and Artificial Intelligence in Clinical Imaging. Seminars in Nuclear Medicine, 41(6):449–462, Nov. 2011. 2
[32] Karen Simonyan and Andrew Zisserman. Very deep con- volutional networks for large-scale image recognition. In CVPR, 2014. 2
[33] Yi-Hsuan Tsai, Wei-Chih Hung, Samuel Schulter, Ki- hyuk Sohn, Ming-Hsuan Yang, and Manmohan Chandraker. Learning to adapt structured output space for semantic segmentation. In CVPR, 2018. 2, 6, 7, 8
[34] Eric Tzeng, Judy Hoffman, Trevor Darrell, and Kate Saenko. Simultaneous deep transfer across domains and tasks. In ICCV, 2015. 2
[35] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Dar- rell. Adversarial discriminative domain adaptation. In ICCV, 2017. 2
[36] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579–2605, 2008. 7
[37] Shuai Wang, Yang Cong, Huijie Fan, Lianqing Liu, Xiao- qiu Li, Yunsheng Yang, Yangdong Tang, Huaici Zhao, and Haibin Yu. Computer-aided endoscopic diagnosis without human-specific labeling. IEEE Transactions on Biomedical Engineering, 63(11):2347–2358, Nov 2016. 2
[38] Yan Xu, Jun-Yan Zhu, Eric I-Chao Chang, Maode Lai, and Zhuowen Tu. Weakly supervised histopathology cancer image segmentation and classification. Medical Image Analysis, 18(3):591–604, Apr. 2014. 1, 2
[39] Chaochao Yan, Jiawen Yao, Ruoyu Li, Zheng Xu, and Jun- zhou Huang. Weakly supervised deep learning for thoracic disease classification and localization on chest x-rays. In Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, pages 103–110. ACM, 2018. 1
[40] Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. Learning a discriminative feature network for semantic segmentation. CoRR, abs/1804.09337, 2018. 6
[41] Fisher Yu and Vladlen Koltun. Multi-scale context aggrega- tion by dilated convolutions. In ICLR, 2016. 8
[42] Yang Zhang, Philip David, and Boqing Gong. Curriculum domain adaptation for semantic segmentation of urban scenes. In ICCV, Oct 2017. 2, 7, 8
[43] Rongchang Zhao, Wangmin Liao, Beiji Zou, Zailiang Chen, and Shuo Li. Weakly-supervised simultaneous evidence identification and segmentation for automated glaucoma diagnosis. 2019. 1
[44] Yang Zou, Zhiding Yu, B.V.K. Vijaya Kumar, and Jinsong Wang. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In The European Conference on Computer Vision (ECCV), September 2018. 2