Learning Adaptive Loss for Robust Learning with Noisy Labels

2020·Arxiv

Abstract

Abstract

Robust loss minimization is an important strategy for handling robust learning issue on noisy labels. Current robust loss functions, however, inevitably involve hyperparameter(s) to be tuned, manually or heuristically through cross validation, which makes them fairly hard to be generally applied in practice. Besides, the non-convexity brought by the loss as well as the complicated network architecture makes it easily trapped into an unexpected solution with poor generalization capability. To address above issues, we propose a meta-learning method capable of adaptively learning hyperparameter in robust loss functions. Specifi-cally, through mutual amelioration between robust loss hyperparameter and network parameters in our method, both of them can be simultaneously finely learned and coordinated to attain solutions with good generalization capability. Four kinds of SOTA robust loss functions are attempted to be integrated into our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its accuracy and generalization performance, as compared with conventional hyperparameter tuning strategy, even with carefully tuned hyperparameters.

1. Introduction

DNNs have recently obtained remarkable performance on various applications [15, 22]. Its effective training, however, often requires to pre-collect large scale finely annotated samples. When the training dataset contains certain amount of noisy (incorrect) labels, the overfitting issue tends to easily occur, naturally leading to their poor performance in generalization [58]. In fact, such biased training data are commonly encountered in practice, since the data are generally collected by coarse annotation sources, like crowdsourcing systems [4] or search engines [26, 61]. Such robust deep learning issue is thus critical in machine learning and computer vision.

One of the most classical methods for handling this issue

Figure 1. Comparison of different loss functions. In each figure, the 0-1 loss, CE loss, original robust loss, and those learned by our method under three different noise rates on CIFAR-10 are shown. The robust losses included in (a)-(d) are PolySoft [13], GCE [59], SL [53] and Bi-Tempered [1], respectively.

is to employ robust losses, that are not unduly affected by noisy labels, to replace the conventional noise-insensitive ones to guide the training process [31]. For example, as compared with the commonly used cross entropy (CE) loss, the mean absolute error (MAE), as well as the simplest 0-1 loss for classification, can be more robust against noisy labels [10] due to their evident suppression to large loss values (as clearly depicted in Fig.1), and thus inclines to reduce the negative influence brought by the outlier samples with evidently corrupted labels. Beyond other robust learning techniques for defending noisy labels, like sample reweighting [23, 6, 52, 20, 40, 42], loss correction [12, 45, 37, 16], and label correction [24, 25, 51, 47], such robust-loss-designing methodology is superior in its concise implementation scheme and solid theoretical basis of robust statistics and generalization theory [18, 27, 32, 37]

Along this research line, besides the aforementioned MAE and 0-1 loss, various forms of robust losses have been designed against such robust learning issue on noisy labels. For example, 0-1 loss is verified to be robust for binary clas-sification [31, 11]. However, since 0-1 loss is not continuous and the corresponding learning algorithm is hardly to be efficiently and accurately executed, many surrogate loss functions have been proposed to approximate it [2, 32, 35], such as ramp loss [5] and unhinged loss [50], which are also proved to be robust to label noise under certain conditions [11]. Specifically, [31] formally defines a “noise-tolerant” robust loss if the minimization under it with noisy labels would achieve the same solution as that with noise-free labels. [10] further relaxes the definition with loss bounded conditions to make the definition useful in guiding construction of rational robust losses in practice. Inspired by this formulation, several losses have been designed very recently and proved to satisfy loss bounded conditions, like the GCE [59] and SL [53], both originated from the CE loss by introducing some noise-robust factor.

Although these robust loss functions help improve the robustness of a learning algorithm on noisy labels, they still have evident deficiencies in practice. On the one hand, they inevitably involve hyperparameter(s) to control their robustness extents against different noise rates. These hyperparameters need to be manually preset or tuned by heuristic strategies like cross-validation, naturally conducting efficiency issue and difficulty in practical implementations. On the other hand, the relatively complex form of robust loss will increase the non-convexity of the objective function used for training the network. Together with the non-convexity brought by the complicated network architectures, the minimization with all involved network parameters is highly non-convex, making the problem easily trapped to an unexpected solution with poor generalization capability even under properly preset hyperparameters in the robust loss. It is thus still challenging to construct a generally useful algorithm for the problem.

To alleviate the aforementioned issues, this paper presents an adaptive hyperparameter learning strategy to automatically tune the hyperparameter and thus learn the robust loss from data. Specifically, this study mainly made three-fold contributions.

• The proposed algorithm realizes a mutual amelioration between automatically tuning proper hyperparameters involved in a robust loss and learning suitable network parameters. To the best of our knowledge, this is the first work to handle the robust loss (with explicit and concise forms) adaptive learning under noisy labels.

• Four kinds of STOA robust loss functions, including GCE [59], SL [53], Bi-Tempered [1] and PolySoft [13], are attempted to be integrated into our algorithm, showing the generality of our algorithm on adaptive robust loss learning with noisy labels. Especially, besides GCE and SL, which have been proved to be theoretically robust under loss bounded condition, we also prove that the loss bounded conditions for BiTempered and PolySoft, implying their intrinsic ro-

bustness. These robust losses are thus all with sound theoretical guarantee and potentially useful.

• Comprehensive experiments substantiate the effectiveness of the proposed algorithm, especially its superiority beyond conventional hyperparameter setting strategy. Specifically, from the experiments, it is interestingly seen that through iteratively ameliorating both robust loss hyperparameters and deep network parameters, our algorithm is capable of exploring good solution for the problem with evidently better generalization than that extracted by conventional hyperparameter tuning strategy with even carefully tuned hyperparameters. This might show a new potential way for exploring solutions with better generalization for such highly non-convex robust learning problems.

The paper is organized as follows. Section 2 reviews the related works. Section 3 introduces the four robust loss forms used in this paper, and proves the theoretical robustness theory for Bi-Tempered and PolySoft. Our main algorithm for adaptive robust loss learning is also presented in this section. Experiments are demonstrated in Section 4, and a conclusion is finally made.

2. Related Work

Deep learning with noisy labels. There are various approaches raised for handling robust learning issues under noisy labels, which can be roughly divided into four categories: label correction, loss correction, sample reweighting and robust loss setting.

The label correction approach aims to correct noisy labels to their true ones via a supplemental clean label inference step, characterized by directed graphical models [55], conditional random fields [49] or knowledge graphs [25]. Comparatively, the loss correction approach assumes that there exists a noise transition matrix defining the probability of one class changed to another. Typically, [45, 12] modeled the matrix using a linear layer on the top of the DNNs, Forward [37] used the noise transition matrix to modify the loss function, and GLC [16] used additional meta data to estimate the transition matrix.

Robust loss approach. Based on the noise-tolerant definition given by [34], it has been proven that 0-1 loss, sigmoid loss, ramp loss, and probit loss are all noise-tolerant under some conditions [11]. [11] further relaxed this definition as bounded loss condition to make the theory better feasible in practice. Recently, curriculum loss, a tighter upper bound of the 0-1 loss was proposed in [28], which can adaptively select samples for training as a curriculum learning process. Generalized to multi-class problem, MAE (Mean Absolute Error) is proved to be robust to symmetric label noise and class-conditional noise.

Under the advanced noise-robust understanding pro- vided by [10], some new robust losses have been designed on the basis of classical CE loss very recently expected to be finely performed in real practice. Zhang et al. [59] demonstrated that it is hard to train DNN with MAE, and proposed to combine MAE and CE losses to obtain a new loss function, GCE, which behaves very like a weighted MAE, to handle the noisy label issue. [53] observed that the learning procedure of DNNs with CE loss is class-biased, and proposed a Reverse Cross Entropy (RCE) to help robust learning. Besides, [1] also presented a robust loss called BiTempered by introducing two tunable temperatures to the traditional softmax layer and CE loss, which makes the loss be bounded and heavy-tail. Xu et al., [57] also provided a novel information-theoretic robust loss function different from the distance-based loss as aforementioned. Sample reweighting approach. The main idea of this approach is to assign weights to losses of all training samples, and iteratively update these weights based on the loss values during the training process [23, 19, 60]. Such a lossweight function is generally set as monotonically decreasing, enforcing a learning effect that samples with larger loss, more possible to be noisy labeled as compared with small loss samples, are with smaller weights to suppress their effect to training. In this manner, the negative influence of noisy labels can be possibly alleviated. An interesting result is that when this monotonically decreasing weighting function makes this re-weighting learning process equivalent to solving an implicit robust loss function [33], which constructs a close relationship between this strategy with robust loss approach. Very recently, some advanced sample reweighting methods have been raised inspired by the idea of meta-learning [42], which possesses a much more complicated weighting scheme to the conventional reweighting strategies. This makes them able to deal with more general data bias cases other than noisy labels, like class imbalance. The fine theoretical basis, like noise tolerant, however, is also lost and almost impossible to be further prompted due to their complex implementation formats. Learning adaptive loss. Some other methods have also attempted to directly learn a good proxy for an underlying evaluation loss. For example, learning to teach [54] dynamically learned the loss through a teacher network outputting the coefficients matrix of the general loss function. Xu et al., [56] learned a discrete optimization schedule that alternates between different loss functions at different time-points. Adaptive Loss Alignment [17] extended work in [54] to loss-metric mismatch problem. [14] tried to learn the surrogate losses for non-differentiable and nondecomposable loss function. These methods, however, generally attain losses with complicated forms, making them hardly to be theoretically analyzed with robust loss theory. Hyperparameter optimization. Hyperparameter opti-

mization was historically investigated by selecting proper values for each hyperparameter to obtain better performance on validation set. Typical methods include grid search, random search [3], Bayesian optimization [44, 46], etc. Recently, meta-learning based strategy has been gradually more investigated [9, 8, 30, 38]. This paper can be seen as a specific exploration of this methodology on adaptive robust loss learning issue on noisy labels.

3. Adaptive Robust Loss Learning

3.1. Preliminaries

We consider the problem of c-class classification. Let be the feature space, and be the label space. Assume the DNN architecture is with a softmax output layer. Denote the network as a function with input and output as f(x; w), where , where w represent the network parameters. , representing the j-th component () of f(x; w), then satisfies . Given training data , for any loss function L, the (empirical) risk of the network classifier is defined as:

The commonly used CE loss can be written as:

where denotes the j-th component of . Generally in all components of , only one is 1 and all others are 0.

3.2. Typical Robust Loss Forms

We first introduce the forms of some typical robust losses.

Generalized Cross Entropy (GCE). To exploit the ben-efits of both the noise-tolerant property of MAE and the implicit weighting scheme of CE for better learning, Zhang et al., [59] proposed the GCE loss as follows:

where denotes the j’s index of the term for each i, and . GCE loss degenerates to the CCE when q approaches to 0 and becomes to MAE loss when q = 1.

Symmetric Cross Entropy (SL). Wang et al. [53] proposed an extra term for CE to make it noise tolerant and designed the Reverse Cross Entropy (RCE):

where A < 0 is a preset constant. The SL loss is defined as:

Bi-Tempered logistic Loss (Bi-Tempered). Amid et al. [1] replaced the logarithm and exponential of the logistic loss with corresponding “tempered” versions function , to make the loss functions bounded to handle large-margin outliers and softmax function heavy-tailed to handle small-margin mislabeled examples. Specifically, they define and where . The BiTempered Loss function is then defined as [1]:

where , is the input of softmax layer, and is calculated by letting .

Polynomial Soft Weighting loss (PolySoft). Self-paced learning (SPL) is a typical sample reweighting strategy to handle noisy labels by setting monotonically decreasing weighting function [23, 19, 60]. It has been proved that such re-weighting learning process is equivalent to minimizing an latent robust loss [33], and it thus can also be seen as a standard robust loss method. Recently, Gong et al. [13] proposed a polynomial soft weighting scheme for SPL, which can generally approximate monotonically decreasing weighting functions. By setting the CE loss as the basis loss form, the latent robust loss of this method is:

where is defined as in Eq. (1).

3.3. Adaptive Robust Loss Learning Algorithm

It can be observed that all aforementioned robust loss functions contain hyperparameter(s), e.g., q in (Eq.(2)), in (Eq.(3)), in (Eq.(4)) and in (Eq.(5)). Instead of manually presetting or tuning them by cross-validation, we provide the following algorithm to adaptively learn these hyperparameter(s), by borrowing the idea of recent meta-learning techniques [41, 48, 7, 9, 43, 42].

The Meta-learning Objective. Given training dataset D, the net parameters are trained by optimizing the following minimization problem under certain robust loss :

where denotes the hyperparameter set of .

Our method aims to automatically learn the hyperparameters in a meta-learning manner [7, 40, 42]. Specifically, assume that we have a small amount meta-data set (i.e., with clean labels) , representing the meta-knowledge of ground-truth sample-label distribution, where M is the number of meta-samples, and . We can then formulate a meta-loss minimization problem with respect to as:

where represents the loss imposed on meta data. Since meta data are all clean, it is employed as the conventional loss forms without hyperparameter, like CE loss.

Learning Algorithm. Calculating the optimal and requires two nested loops of optimization, which is expensive to obtain the exact solution [9]. Here we adopt an online approximation strategy [7, 42] to jointly update both sets of parameters in an iterative manner to guarantee the efficiency of the algorithm.

At iteration step t, we need to update hyperparameter on the basis of the net parameter and hyperparameter obtained in the last iteration by minimizing the meta loss defined in Eq.(7). To guarantee efficency and general feasibility, SGD is employed to optimize the parameters on m mini-batch samples from , i.e.,

where the following equation is used to formulate on a mini-batch training samples from D:

which is inspired from MAML [7], and is the step size. When obtained the parameter , the network parameters can then be updated by:

The Adaptive Robust Loss (ARL) Algorithm can then be summarized in Algorithm 1. All computations of gradients can be efficiently implemented by automatic differentiation techniques and easily generalized to any deep learning architectures. The algorithm can be easily implemented using popular deep learning frameworks like PyTorch [36]. The algorithm can then be easily integrated with any robust loss to make their hyperparameter automatically learnable. Specifically, we denote the ARL algorithms on robust losses defined in Eq.(2),(3),(4) and (5) as A-GCE, A-SL, A-Bi-Tempered and A-PolySoft, respectively.

3.4. Noise-robust Properties of Utilized Losses

It can be seen from Fig.1 the adopted GCE, SL, BiTempered and PolySoft losses are all robust amelioration from CE loss to noisy labels. All of them tend to be flat when loss becomes larger so as to suppress the negative in-fluence by large losses brought by noisy samples with incorrect labels. In theory, actually they all satisfy loss bounded conditions, and are noise-tolerant under certain conditions. This property has been proved for GCE [59] and SL [53], and we then provide the related results for the other two.

Denote the true label of x as , in contrast to its noisy label y, and and as the underlying distributions of clean and noisy data, respectively. Let be the risk of classifier f under clean labels, and as the risk of classifier f under label noise rate . A loss function L is defined to be noise tolerant [31, 10] if on noisy data has the same mis-classification probability as that of on clean data, where and are the global minimizers of and , respectively. To make this definition more feasible in practice, Ghosh et al. [10] further relaxed this definition as the bounded loss condition and proved that under it the loss also possesses certain robustness capacity in theory [59], as proved for GCE [59] and SL [53]. We can further prove that the PolySoft and Bi-Tempered also possess these properties, as provided in the following theorems. The detailed proofs are listed in the supplementary material.

Theorem 1 Under the symmetric noise with , and , the PolySoft loss in Eq. (5) satisfies

A, A,

where . Especially, when , we have .

The theorem clarifies that the PolySoft loss is with loss bounded condition, and noise tolerant when .

Theorem 2 Under the symmetric noise with , and , the Bi-Tempered loss in Eq.(4) satisfies

A, A,

This theorem illustrates that the Bi-Tempered loss satisfies loss bounded condition. Albeit not noise tolerant, it still possesses certain theoretical robustness [59].

4. Experimental Results

To evaluate the capability of the ARL algorithm, we implement experiments on CIFAR-10, CIFAR-100, TinyImageNet, as well as a large-scale real-world noisy dataset Clothing1M.

4.1. Experimental Setup

Datasets. We first verify the effectiveness of our method on two benchmark datasets: CIFAR-10 and CIFAR-100 [21], consisting of color images arranged in 10 and 100 classes, respectively. Both datasets contain 50,000 training and 10,000 test images. We random select 1,000 clean images in the validation set as meta data. Then, we verify our method on a larger and harder dataset called TinyImageNet (T-ImageNet briefly), containing 200 classes with 100K training, 10K validation, 10K test images of . We random sample 10 clean images per class as meta data. These datasets are popularly used for evaluation of learning with noisy labels in the previous literatures [39, 37, 12].

Noise setting. We test two types of label noise: symmetric noise and asymmetric (class-dependent) noise. Symmetric noisy labels are generated by flipping the labels of a given proportion of training samples to one of the other class labels uniformly [58]. For asymmetric noisy labels, we use the setting in [42], where the label of each sample is independently flipped to two classes with same probability. Also, we consider a more realistic hierarchical corruption in CIFAR-100 as described in [16], which applies uniform corruption only to semantically similar classes.

Baselines. We compare ARL algorithm with the following state-of-art methods, and implement all methods with default settings in the original paper by PyTorch. 1) CE, which uses CE loss to train the DNNs on noisy datasets. 2) Forward [37], which corrects the prediction by the label transition matrix. 3) DMI [57], which uses mutual information based robust loss to train the DNNs. 4) Meta-Weight-Net [42], which uses a MLP net to learn the weighting function in a data-driven fashion, representing the SOTA sample weighting methods. 5) PolySoft [13], 6) GCE [59], 7) SL [53], 8) Bi-Tempered [1] represent the STOA robust loss methods. The meta-data in these methods are used as validation set for cross-validation to search the best hyperparameters except for Meta-Weight-Net.

Network structure. We use ResNet-32 [15] as our classifier network models for CIFAR-10 and CIFAR-100 dataset, and a 18-layer Preact ResNet [15] for T-ImageNet.

Table 1. Test accuracy (%) of all competing methods on CIFAR-10 and CIFAR-100 under different noise rates. The best results are in bold.

Experimental setup. We train the models with SGD, at an initial learning rate 0.1 and a momentum 0.9, a weight decay with mini-batch size 100. For our proposed methods, ResNet-32 models, the learning rate decays 0.1 at 40 epochs and 50 epochs for a total of 60 epochs; Preact ResNet-18 models, the learning rate decays 0.1 at 30 epochs and 60 epochs for a total of 90 epochs. We use SGD to optimize hyperparameters, and the learning rate setting is the same as the classifier for different experiments.

Hyperparameter setting. For the methods in PolySoft, GCE, SL, Bi-Tempered, we used the optimal hyperparameter in the original paper or carefully searched by cross-validation. For our method, those hyperparameters are automatically learned.

4.2. Robustness Performance Evaluation

Results on CIFAR-10 and CIFAR-100. The classifica-tion accuracies of CIFAR-10 and CIFAR-100 under symmetric and asymmetric noise are reported in Table 1 with 5 random runs. As can be seen, our ARL algorithm (in color blue) improves on the original algorithm via a large margin for almost all noise rates and all datasets. Table 2 shows classification accuracies of more realistic hierarchical label corruption on CIFAR-100 dataset. Our ARL algorithm can also improve the accuracy of the original algorithm and APolySoft outperforms all other baselines methods.

Combining Table 1 and 2, it can be observed that: 1) PolySoft and Bi-Tempered’s performance drop quickly as the noise rate exceeds 0.4, and A-PolySoft and A-Bi-Tempered improve the accuracy around 5% on CIFAR-100 thereafter. 2) SL drops more quickly on CIFAR-100 than CIFAR-10, and A-SL improves the accuracy more evidently, over 10% on CIFAR-100 with 20% noise rate. 3) A-PolySoft outperforms the STOA sample reweighting method Meta-Weight-Net, possibly attributed to its monotonically decreasing form of robust loss, making it noise robust, as illustrated in Section 4.3.

Results on T-ImageNet. To verify our approach on a more complex scenario, we summarize in Table 3 the test accuracy on T-ImageNet with different noise setting. As we can see, for both noise settings with different noise rates, A-PolySoft outperforms other baselines. Meanwhile, our ARL algorithm improves the original algorithm stably.

4.3. Towards Understanding of ARL Algorithm

How ARL adapt to noise extents. To understand how the ARL algorithm automatically fit noise extents, we plot the learned loss under different noise rates on CIFAR10 dataset in Fig.1. It is easy to see that when the loss value is small, learned loss is almost the same as CE loss; while when loss becomes larger and exceeds a certain threshold, learned loss tends to be flat thereafter, behaving as suppressing the effect of samples with incorrect labels often with large losses. Furthermore, the learned loss tends to be flat earlier when the noisy rate is higher, which implies ARL algorithm can adjust loss function according to noise amounts to better encode and alleviate the noisy label effectly. Such noisy-rate adapting capability of our algorithm finely ex-

Table 2. Test accuracy (%) of ResNet-32 on CIFAR-100 with hierarchical noisy labels. The best results are in bold.

Table 3. Test accuracy (%) on T-ImageNet under different noise fractions. The best results are in bold.

Figure 2. Sample weight distribution on CIFAR-10 dataset under 40% symmetric noise experiments during training process. (a)-(b) present the sample weights produced by Meta-Weight-Net and A-PolySoft, respectively.

plains its superior robustness depicted in our experiments.

Reweighting mechanism visualization. To better understand why our algorithm contributes to learn more robust models during training, we plot the weight distribution variations of clean and noisy training samples during the learning process of A-PolySoft in Fig.2. To better visualize this point, we also compare such weight distribution of Meta-Weight-Net. It can be seen the weights extracted by A-PolySoft clearly distinguish clean and noisy samples,

Figure 3. 2D representations extracted by A-Bi-Tempered and baselines on CIFAR-10 dataset with 60% symmetric noisy labels.

much more evident than those obtained by Meta-Weight-Net. Specifically, through the iteration of our algorithm, the weights of clean samples are with larger values gradually than those on noisy ones (most approximates 0), and thus the negative influence of these noisy labels tends to be effectively reduced. This clearly explains why our algorithm is able to consistently outperform the Mete-Weight-Net.

Representation Demonstration. We further investigate the representations learned by ARL algorithm compared to other baselines. We extract the high-dimensional representations of data at the second last dense layer of the learned classifiers by different methods, and then project them to a 2D embedding by using t-SNE [29]. As shown in Fig.3, it is evident that the representations learned by A-Bi-Tempered algorithm (other methods are presented in the supplemental file) are obviously better than that by CE and Bi-Tempered with more separated and clearly bounded clusters.

4.4. Ablation Study

ARL algorithm tries to mutually ameliorate robust loss hyperparameters and net parameters in an iterative manner. An important problem is whether or not the mutual amelioration process helps explore better generalization solution. To clarify this, Fig.4 compares four strategies as: 1) SL: conventional SL method using the hyperparameter optimally tuned by cross-validation; 2) SL-Opt1: conventional SL method using the hyperparameter learned by A-SL at the last step; 3) SL-Opt2: run conventional SL in each step of A-SL by using the hyperparameter obtained by the latter in

Table 4. Test accuracy (%) of different models on real-world noisy dataset Clothing1M. The best results are in bold.

(d) CIFAR-100 60% Symmetric Noise Figure 4. Test accuracy vs. number of epochs of A-SL and other comparison methods, SL, SL-Opt1, SL-Opt2 with different noise amounts on CIFAR-10 and CIFAR-100 datasets under symmetric noise.

its current step as its initialization; 4) A-SL: our algorithm. It can be easily observed: 1) SL-Opt1 performs worse than SL, which means the hyperparameter adaptively learned by A-SL is actually not the optimal one for SL, with fixed hyperparameter throughout its iteration. 2) SL-Opt2 outperforms SL, implying the A-SL adaptively finds a proper hyperparameter for its robust loss and simultaneously explores a good initialization net parameter for this loss under its current hyperparameter in a dynamical mutual updating way. 3) ASL outperforms SL-Opt2, showing such adaptive learning manner for both robust loss hyperparameter and net parameters should be a more suitable manner for simultaneously obtaining optimal values for both of them rather than only updating one under the other fixed, even the fixed one could be set possibly optimal. All of these observations inspire such a meta-learning regime might provide a rational learning manner for exploring better generalization solutions for such non-convex robust learning problem.

4.5. Experiments on Real-world Noisy Dataset

We then verify the applicability of our algorithm on a real-world large-scale noisy dataset: Clothing1M, which contains 1 million images of clothing from online shopping websites with 14 classes, e.g., T-shirt, Shirt, Knitwear. The labels are generated by the surrounding text of images and are thus extremely noisy. The dataset also provides 50k, 14k, 10k manually refined clean data for training, validation and testing respectively, but we did not use the 50k clean data. We use the validation dataset as the meta dataset.

Experimental setup. Following the previous works[37, 47], we used ResNet-50 pre-trained on ImageNet. For preprocessing, we resize the image to , crop the center as input, and perform normalization. We used SGD with a momentum 0.9, a weight decay , an initial learning rate 0.01, and batch size 32. The learning rate of ResNet-50 is divided by 10 after 5 epochs (for a total 10 epochs). We use SGD to optimize hyperparameters with an initial learning rate 0.1, and divided by 10 after 5 epoch.

Results. The results are summarized in Table 4. The conventional algorithm need to search a proper hyperparameter from a candidate set by cross-validation to obtain a sat-isfied result, which is often expensive and hard to be reproduced in real-world settings. Our ARL algorithm provides a new way to mutually ameliorate between hyperparameter and network parameters to reduce the barrier of practical implementations. It can be seen that ARL algorithm can consistently improve the performance of the original algorithm, and A-PolySoft obtains the highest performance compared to the baselines.

5. Conclusion

In this paper, we have proposed an adaptive hyperparameter learning strategy to learn the form of robust loss function directly from data by automatically tuning the hyper-parameter. Four STOA robust loss functions are chosen to be integrated into our ARL framework, to verify its validity. Comprehensive experiments have been conducted, and the empirical results show that the propose method can perform superior than conventional hyperparameter setting strategy. The learning fashion of iterative amelioration betwen hyper-parameter and network parameter has shown good potential for providing a new thought to explore solutions with better generalization for such highly non-convex robust loss optimization problems.

References

[1] Ehsan Amid, Manfred K Warmuth, Rohan Anil, and Tomer Koren. Robust bi-tempered logistic loss based on bregman divergences. In NeurIPS, 2019. 1, 2, 3, 4, 5

[2] Peter L Bartlett, Michael I Jordan, and Jon D McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473):138–156, 2006. 2

[3] James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. JMLR, 2012. 3

[4] Wei Bi, Liwei Wang, James T Kwok, and Zhuowen Tu. Learning to predict from crowdsourced data. In UAI, 2014. 1

[5] J Paul Brooks. Support vector machines with the ramp loss and the hard margin loss. Operations research, 59(2):467– 479, 2011. 2

[6] Haw-Shiuan Chang, Erik Learned-Miller, and Andrew Mc- Callum. Active bias: Training more accurate neural networks by emphasizing high variance samples. In NeurIPS, 2017. 1

[7] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Modelagnostic meta-learning for fast adaptation of deep networks. In ICML, 2017. 4

[8] Luca Franceschi, Michele Donini, Paolo Frasconi, and Mas- similiano Pontil. Forward and reverse gradient-based hyper-parameter optimization. In ICML, 2017. 3

[9] Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In ICML, 2018. 3, 4

[10] Aritra Ghosh, Himanshu Kumar, and PS Sastry. Robust loss functions under label noise for deep neural networks. In AAAI, 2017. 1, 2, 3, 5

[11] Aritra Ghosh, Naresh Manwani, and PS Sastry. Making risk minimization tolerant to label noise. Neurocomputing, 160:93–107, 2015. 1, 2

[12] Jacob Goldberger and Ehud Ben-Reuven. Training deep neural-networks using a noise adaptation layer. In ICLR, 2017. 1, 2, 5

[13] Maoguo Gong, Hao Li, Deyu Meng, Qiguang Miao, and Jia Liu. Decomposition-based evolutionary multiobjective optimization to self-paced learning. IEEE Transactions on Evolutionary Computation, 23(2):288–302, 2018. 1, 2, 4, 5

[14] Josif Grabocka, Randolf Scholz, and Lars Schmidt-Thieme. Learning surrogate losses. arXiv preprint arXiv:1905.10108, 2019. 3

[15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016. 1, 6

[16] Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep networks on labels corrupted by severe noise. In NeurIPS, 2018. 1, 2, 5

[17] Chen Huang, Shuangfei Zhai, Walter Talbott, Miguel Bautista Martin, Shih-Yu Sun, Carlos Guestrin, and Josh Susskind. Addressing the loss-metric mismatch with adaptive loss alignment. In ICML, 2019. 3

[18] Peter J Huber. Robust statistics. Springer, 2011. 1

[19] Lu Jiang, Deyu Meng, Teruko Mitamura, and Alexander G Hauptmann. Easy samples first: Self-paced reranking for zero-example multimedia search. In Proceedings of the 22nd ACM international conference on Multimedia, pages 547– 556, 2014. 3, 4

[20] Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In ICML, 2018. 1

[21] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009. 5

[22] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurIPS, 2012. 1

[23] M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self- paced learning for latent variable models. In NeurIPS, 2010. 1, 3, 4

[24] Kuang-Huei Lee, Xiaodong He, Lei Zhang, and Linjun Yang. Cleannet: Transfer learning for scalable image classi-fier training with label noise. In CVPR, 2018. 1

[25] Yuncheng Li, Jianchao Yang, Yale Song, Liangliang Cao, Jiebo Luo, and Li-Jia Li. Learning from noisy labels with distillation. In ICCV, 2017. 1, 2

[26] Junwei Liang, Lu Jiang, Deyu Meng, and Alexander G Hauptmann. Learning to detect concepts from webly-labeled video data. In IJCAI, 2016. 1

[27] Anqi Liu and Brian Ziebart. Robust classification under sam- ple selection bias. In NeurIPS, 2014. 1

[28] Yueming Lyu and Ivor W Tsang. Curriculum loss: Robust learning and generalization against label corruption. arXiv preprint arXiv:1905.10045, 2019. 2

[29] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9:2579–2605, 2008. 7

[30] Dougal Maclaurin, David Duvenaud, and Ryan Adams. Gradient-based hyperparameter optimization through reversible learning. In ICML, 2015. 3

[31] Naresh Manwani and PS Sastry. Noise tolerance under risk minimization. IEEE transactions on cybernetics, 43(3):1146–1151, 2013. 1, 2, 5

[32] Hamed Masnadi-Shirazi and Nuno Vasconcelos. On the de- sign of loss functions for classification: theory, robustness to outliers, and savageboost. In NeurIPS, 2009. 1, 2

[33] Deyu Meng, Qian Zhao, and Lu Jiang. A theoretical un- derstanding of self-paced learning. Information Sciences, 414:319–328, 2017. 3, 4

[34] Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Raviku- mar, and Ambuj Tewari. Learning with noisy labels. In NeurIPS, 2013. 2

[35] Richard Nock and Frank Nielsen. On the efficient minimiza- tion of classification calibrated surrogates. In NeurIPS, 2009. 2

[36] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS Workshop, 2017. 5

[37] Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In CVPR, 2017. 1, 2, 5, 8

[38] Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In ICML, 2016. 3

[39] Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy labels with bootstrapping. In ICLR, 2015. 5

[40] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urta- sun. Learning to reweight examples for robust deep learning. In ICML, 2018. 1, 4

[41] J¨urgen Schmidhuber. Learning to control fast-weight mem- ories: An alternative to dynamic recurrent networks. Neural Computation, 4(1):131–139, 1992. 4

[42] Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta-weight-net: Learning an explicit mapping for sample weighting. In NeurIPS, 2019. 1, 3, 4, 5

[43] Jun Shu, Zongben Xu, and Deyu Meng. Small sample learn- ing in big data era. arXiv preprint arXiv:1808.04572, 2018. 4

[44] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Prac- tical bayesian optimization of machine learning algorithms. In NeurIPS, 2012. 3

[45] Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdev, and Rob Fergus. Training convolutional networks with noisy labels. In ICLR Workshops, 2015. 1, 2

[46] Kevin Swersky, Jasper Snoek, and Ryan P Adams. Multi- task bayesian optimization. In NeurIPS, 2013. 3

[47] Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, and Kiy- oharu Aizawa. Joint optimization framework for learning with noisy labels. In CVPR, 2018. 1, 8

[48] Sebastian Thrun and Lorien Pratt. Learning to learn. Springer, 1998. 4

[49] Arash Vahdat. Toward robustness against label noise in train- ing deep discriminative neural networks. In NeurIPS, 2017. 2

[50] Brendan Van Rooyen, Aditya Menon, and Robert C Williamson. Learning with symmetric label noise: The importance of being unhinged. In NeurIPS, 2015. 2

[51] Andreas Veit, Neil Alldrin, Gal Chechik, Ivan Krasin, Abhi- nav Gupta, and Serge Belongie. Learning from noisy large-scale datasets with minimal supervision. In CVPR, 2017. 1

[52] Yixin Wang, Alp Kucukelbir, and David M Blei. Robust probabilistic modeling with bayesian data reweighting. In ICML, 2017. 1

[53] Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey. Symmetric cross entropy for robust learning with noisy labels. In ICCV, 2019. 1, 2, 3, 5

[54] Lijun Wu, Fei Tian, Yingce Xia, Yang Fan, Tao Qin, Lai Jian- Huang, and Tie-Yan Liu. Learning to teach with dynamic loss functions. In NeurIPS, 2018. 3

[55] Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. In CVPR, 2015. 2

[56] Haowen Xu, Hao Zhang, Zhiting Hu, Xiaodan Liang, Ruslan Salakhutdinov, and Eric Xing. Autoloss: Learning discrete schedules for alternate optimization. In ICLR, 2019. 3

[57] Yilun Xu, Peng Cao, Yuqing Kong, and Yizhou Wang. L dmi: An information-theoretic noise-robust loss function. In NeurIPS, 2019. 3, 5

[58] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In ICLR, 2017. 1, 5

[59] Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In NeurIPS, 2018. 1, 2, 3, 5

[60] Qian Zhao, Deyu Meng, Lu Jiang, Qi Xie, Zongben Xu, and Alexander G Hauptmann. Self-paced learning for matrix factorization. In AAAI, 2015. 3, 4

[61] Bohan Zhuang, Lingqiao Liu, Yao Li, Chunhua Shen, and Ian Reid. Attend in groups: a weakly-supervised deep learning framework for learning from web data. In CVPR, 2017.