IN the past few years, deep neural networks have beengreatly developed and successfully used in a vast of fields, such as pattern recognition, intelligent robots, automatic control, medicine [1]. Despite the great success, researchers have found the vulnerability of deep neural networks to adversarial attacks, which has been a big concern recently [2]. Briefly speaking, for a well-trained neural network f that correctly classifies an example x, adversarial attacks produce a slight and usually invisible perturbation that the output for is different from that of x, which is incorrect. Mathematically, the above adversarial attack on the classifier f can be described as the following problem:

There have been many discussions about adversarial attack since its proposal by Szegedy et al. [3], which generates adversarial examples with the box-constrained L-BFGS method. The basic idea of (1) is to find a small magnitude of perturbation to change the output. The intuitive idea is to search on the direction of gradient, which is the basic idea of FGSM [3] and BIM (iterative version of FGSM, [4]). Further improvement comes from embedding geometric information, e.g., C&W [5], [6], Projected Gradient Descent (PGD, [4]). The latter is currently regarded as the strongest attack. Many defense methods are also focusing on the difference between the original and the adversarial example on image [7], [8], feature [9], [10], [11], [12], [13], or network structure [3], [14], [15].

Adversarial attack and its defense are revolving around a small and a big resulting difference between and f(x). In other words, the existence of adversarial examples is rooted in large gradients of the output f with respect to the input x. Since one usually does not care about df/dx when training neural networks, the gradient could become unreasonably large and thus draw forth adversarial attacks. Given this, Drucker and LeCun proposed to restrict df/dx by minimizing its magnitude via backpropagation, which is hence named as "double backpropagation" (DBP, [16]). This idea has been recently revisited by [17] for enhancing the robustness to adversarial attacks. Although the basic idea of pursuing small gradients for robustness sounds promising, DBP has not achieved the expectation for classification tasks. This is because in classification tasks, the label y is discontinues, so forcing the neural network to have small df/dx may lead to f’s inability to approach y.

In this paper, we are going to use DBP in training autoencoders. For an autoencoder, the encoder part first encodes the input into a lower dimensional code z = e(x). Then the decoder reconstructs the input from the latent code . Unlike the discontinuity of labels, the latent space of an encoder should be continuous with respective to the input, i.e., if the inputs are similar, their latent representations, and reconstruction results should be similar.

However, to the best of our knowledge, the existing autoencoders do not control the magnitude of the , which leads autoencoders to be easily attacked. The attack on an autoencoder in output space can be described as the following,

where and are the encoder and decoder to be attacked.

Notice that x is a clean example so the distance between x and its reconstructed output is within a reconstruction error bound D, i.e., . Then according to the requirement of (2), we could find that the reconstructed output of adversarial example is different from the input. Instead, it could resemble another target image or could be totally meaningless, the former of which is targeted attack and the latter is non-target. An example of target attack is shown in top of Fig. 1, where the left gives the original image and target image and the top-right plots the reconstruction result of an adversarial example. The attacked autoencoder is Deep Recurrent Attentive Writer (DRAW) [18] and the attack is the one proposed by [19]. With this attack, one can slightly disturb an original input to obtain an adversarial input, that is similar to the origin but its reconstruction is totally different.

Fig. 1: An example of adversarial attack on DRAW. The left column shows attack target (from "1" to "0"). The adversarial example of AE (DRAW) and the reconstruction results are shown in the top line. The adversarial example looks like the original "1", however, after encoding and decoding, the reconstruction result, shown in the top right, becomes to another digit "0". The bottom line shows the performance of DBP. For the same model and the same attack, the reconstruction result of the adversarial example still resembles digit "1".

The autoencoders’ safety is of great importance, because they are widely used as compression schemes for data storage and transmission. Given that the attack results from the large gradients of autoencoders, we propose to use DBP to control the gradient and improve their robustness to adversarial attacks.

Therefore we propose to train an autoencoder by minimizing the reconstruction loss, the KL-divergence, as the existing training process, and the magnitude of the gradients. By training with DBP, an autoencoder with small could be obtained, which prevents a slight perturbation on x to lead significant difference on the latent space and the reconstruction result. For the previous example shown in Fig. 1, when we training DRAW with DBP, the attack fails, i.e., the reconstruction result resembles the input.

The above describes the basic idea training autoencoders by DBP. In the following sections, we specifically train DBP autoencoders in the framework of Variational Autoencoder (VAE) [20], [21] and DRAW. The experiments are on MNIST [22] and CelebA [23] dataset and AUDDC is used to measure their robustness. Comparison results show that autoencoder trained with DBP is more robust against adversarial attacks, in both visual effect and quantitative measurements.

Via training with DBP, we can obtain an autoencoder with controllable gradients . According to that, we further design a distribution estimator based on GMM, which attempts to reconstruct continuous probability p from discrete labels y. This procedure also could be regarded as label smoothing [24], [25], [26], [27], [28]. In the proposed GMM, dp/dz is kept small and its cooperation with an encoder trained by DBP leads to a classifier with reasonable gradients from the output to the input, which is hence robust to attacks. A prominent example is that going alone the obtained classifier, one could transfer the images rather than generate adversarial examples which is an example for application of double backpropagation.

The remaining parts of the paper are organized as follows. In Section II, we give a brief review of autoencoders and describe their vulnerability under adversarial attacks. Section III describes the design of double backpropagation for autoencoders. Section IV discusses the cooperation of autoencoder with DBP and GMM. Experiments that validate the robustness of the proposed DBP are given in Section V. Section VI concludes the paper briefly.

A typical autoencoder consists of two parts, encoder e which compresses the input into a low dimensional latent representation, and decoder d which reconstructs the input from the latent space. Autoencoders and their variations have been widely applied [19]. There have been many different encoding and decoding architectures, e.g., sparse [29], denoising [30], variational [20], [21], Wasserstein [31], symmetric [32]. In this paper, we focus on VAE and DRAW. The former is a classical autoencoder framework and the latter is a newly designed one. We implement double backpropagation on these two models as a representative study for their variants.

VAE is a powerful deep generative model, which is effective in generating data in many complicated distributions. VAE aims to maximize , the distribution to generate x from latent variables z as follows,

In VAE, an arbitrary distribution could be used in the latent space. But in practice, the standard Gaussian distribution is often employed [20], i.e., q(z) = N(0, 1). In VAE training process, we optimize the following lower bound for

is the encoder part which encodes x into z, and represents decoder which reconstructs from and are corresponding parameters. DRAW is a newly proposed structure with a sequential variational auto-encoding framework that allows for the iterative construction of complex images. DRAW uses LSTMs with an attention mechanism and a metaphor of "painting"

the image in a canvas step by step [19]. The training procedure could be summarized as the following for t = 1, 2, ..., T

where denotes the error image, is the logistic sigmoid function, represents the canvas matrix, denotes the output of , and denotes the output of . is used to parameterize the distribution , with which the parameters of the Gaussian distribution in the latent can be calculated as follows,

The total loss L of DRAW can be represented as:

Here, is the final canvas matrix. The negative log probability of x under D is , which represents reconstruction loss and the other loss is for z.

Since the proposal of adversarial attack [14], its design, defense, and analysis have attracted much attention recently. Except for a new type of adversarial attack [33], the majority of the existing adversarial attacks aim at the over-sensitive part of a neural network such that slight distortions on the input lead to significant changes on the output.

As discussed previously, the gradients of autoencoders have not been considered during its training neither, thus the existence of adversarial examples is not surprising. Here, we introduce the targeted attack on autoencoders designed by [19]. It aims to change the input from within a small range such that the reconstruction output is similar to a target Mathematically,

where is a trade-off parameter between the similarity to the target and the restriction on the change. In some point of view, the code is more important to an autoencoder and then the distance is measured in the latent space, i.e.,

Notice that the norm here could be any distance measurement, e.g., pixel-wise distance for the image space and KL-divergence for the latent space.

The robustness of autoencoders against adversarial attacks could be measured by the area under the distortion–distortion curve (AUDDC) defined in [19].

For an autoencoder, two similar inputs should share similar codes ideally, which actually implies that the magnitude of should be small. Therefore, we use double backpropagation scheme to train autoencoders. Concisely, we minimize together with the reconstruction loss and the latent space loss, i.e.,

where stands for the reconstruction loss, for the latent loss in the latent space. are hyper parameters specifying the penalty strength. In (10), the autoencoder could be smooth and imperceptible perturbation of the input cannot significantly change the latent codes and the reconstruction output, which greatly improves the robustness of autoencoders against adversarial attacks.

When training autoencoders, different architectures should adopt different learning rate and . Large learning rate and lead to a divergent reconstruction result. In controlling proportions of different item, the total loss focuses on the loss of the reconstruction in the first half of the time and loss of DBP in the other.

The DBP scheme for training an autoencoder is previously defined by (10). For a specific model, the structures of e, d, the loss for reconstruction, and the latent space loss are different. First, we consider VAE(DBP), which can be represented as:

In practice, we simultaneously train the encoder and the decoder to minimize the (11). The encoder input x to a Gaussian distribution with its mean and variance to be . Then a latent variable z is sampled from such Gaussian distribution by the reparameterization trick [20], which is used for recovery. and represent the parameter of encoder and decoder respectively. and are the hyper parameters that balance the reconstruction performance and the magnitude of the gradient. denotes gradient descent method which is used for updating training parameters . This optimization algorithm is described in Algorithm 1, where

For DRAW with double back-propagation, the optimization objective function becomes

At each time step t, the encoder maps the input from the image x, the previous hidden vector operation. Then a latent variable is sampled from the Gaussian distribution . The decoder receives and previous hidden vector to get the output which is added (via a write operation) to canvas matrix . After T steps, the final canvas matrix is used to reconstruct the image. So the reconstruction loss is and the DBP loss is . The training parameters are and , which are accumulation of and at each time step. This optimization algorithm to train DRAW with DBP is described in Algorithm 2, where

log ), = + log ) 2, and log

Adopting DBP in training of autoencoders could certainly decrease the sensitivity. Here, we give an example in Fig.2 to illustrate the difference. The top row of it shows examples of attacking VAE without DBP. Though the input is similar to the original example, its output is totally different, i.e., very similar to the target. As explained, this phenomenon results from very large gradient in the original VAE, which is 131.874. With DBP, we successfully control the gradients to 7.356, nearly 1/12 of the original one, while keeping the reconstruction error similar, which is 2.771 for VAE(NO DBP) and 2.796 for VAE(DBP). The comparative images could be found in the bottom row of Fig.2, where the reconstruction for clean image is quite good and the attack fails. To produce output similar to target, the input sample transforms to something like the target already, which goes against the principle of adversarial attack.

Fig. 2: Attack performance on VAE(NO DBP) and VAE(DBP). From the original image in the middle of the left column, one can generate an adversarial example (shown in the top) for which the reconstruction result (shown in the top of the right column), is totally different. If we train VAE with DBP, the reconstruction result for the original image is similar to that for VAE; see the middle two images in the right column, but the attack fails, i.e., the adversarial example (the bottom left image) and its reconstruction result (the bottom right image) are similar.

The idea of "double backpropagation" has been introduced for a long time but its performance on classification is far from satisfactory. The basic reason is that the labelling process changes continuous probabilities p to discrete labels y, e.g., and +1 in binary classification. And it is not reasonable to require neural networks that approach the labels to have small gradients. In this paper, we propose to train autoencoder with double backpropagation which ensures dz/dx to be small and meaningful. Therefore, to establish a smooth classifier f, one should first recover the probability to make the other part df/dz to be reasonable, for which the recently proposed idea, label smoothing, [27] may help. For example, soft labels acquired by a distillation mechanism [26] could improve the robustness of model by re-training [34]. Label smoothing also has remarkable advantages for obtaining good within-class distributions, e.g., [24], [28]. The above label smoothing methods only focus on changing the discrete label to distribution to make the df/dx more meaningful. However, we can utilize the smooth encoder to compress the input to a low dimensional space to make the label smoothing process simpler.

In this paper, we model the latent space Z by Gaussian Mixture Models (GMM) to learn distribution instead of discrete labels. The idea of establishing GMM in the latent has been proposed in clustering [35], [36] and novel detection [37]. With GMMs in a latent space, we could keep df/dz small and reasonable. Furthermore, is smooth and meaningful with smooth encoder, so we obtain a classifier expected to be robust.

Suppose there are N classes in a classification problem. We establish N GMMs in z to fit them. For the GMMs, the distribution of z can be represented as:

are parameters of class. K is the number of components of every GMM. denotes component’s prior probability in GMM. Similarly, is mean vector and is covariance matrix. Suppose the dataset of class is . We first train an autoencoder with double backpropagation, and then the encoder compresses in the latent. Then Expectation-Maximization (EM) algorithm [38] is adopted to estimate the parameters in every GMM including prior probability , mean vector and covariance matrix based on

After learning parameters of GMM, the probability density value belonging to every class can be calculated for any point in latent space. We concatenating and normalize them to get the probability that the point belongs to each class, which we regard as the output of a classifier.

Combining the encoder part and the GMM part, we get the smooth classifier f(x) = g(e(x)). So, the transformation between two classes can be implemented by running several iterations of the following equations.

where is the target output and denotes the step size.

As shown in the following transformation, new examples are generated based on gradient as same to the adversarial attack. However, instead of obtaining noisy images with incorrect label, we successfully transfer the image, the probability, and the label jointly, from "9" to "7". Figure 3 illustrates the images and the probabilities.

Transformation and adversarial attacks do not co-exist. The success of transformation implies the robustness of the established GMM. Therefore, the classifier we obtained by GMM estimation and smooth encoder is relatively robust to adversarial attacks. Gaussian mixture variational autoencoder [39] is similar to our model in the structure. Nevertheless, this defense method does not maintain a good reconstruction result. Furthermore, it also fails in white box when the hyper parameter that trades off between reconstruction fidelity and latent space prior changes.

As previous discussed, training autoencoder with DBP could restrict the magnitude of the gradient so that improve the performance against adversarial examples. In this section, we validate the robustness of proposed method by securing

Fig. 3: By modeling a GMM over an autoencoder with DBP, one can obtain a smooth classifier such that changing the image in the gradient direction could really transfer the image from "9" and "7", which also means that adversarial attack in gradient direction fails. The bottom plots the smooth change of the probabilities of "9" and "7".

autoencoder against adversarial attack. First, we train autoencoder with DBP and validate that it improves robustness against adversarial examples greatly comparing the autoencoder without DBP on MNIST and CelebA dataset. The autoencoders we use are VAE and DRAW, where adopting "double backpropagation" method makes a difference in defending. After that, we train an autoencoder with DBP and build GMMs in latent space to obtain robust classifier on MNIST and SVHN. Then we validate its transferability and robustness against white and black attack.

5.1 Reconstruction Performance

The autoencoders we choose to research are VAE and the DRAW without and with its attention mechanism. For autoencoders’ structure, we adopt the models in [19]. The details about the models and their implementation are given in the supplemental materials.

Table 1 shows that "double back propagation" restricts the magnitude of gradient , most of which reaches nearly 1/100 of the original one. At the same time, the quality of reconstruction does not decrease too much. In Fig. 4, we show reconstruction results of AE(DBP) and AE(NO DBP) on MNIST and CelebA for the qualitative analysis. For each pair, the left image is an example from MNIST and the right one is from CelebA.

Fig. 4: Reconstruction results of DRAW without and with DBP trained on MNIST and CelebA. The left column shows two original images and the reconstruction results are given in the right, indicating that DBP only slightly discards the reconstruction performance.

TABLE 1: Reconstruction loss and magnitude of gradient(norm) for all autoencoders of different architecture with and without double back-propagation on MNIST and CelebA.

5.2 Robustness of VAE with DBP

In the previous part, we show that applying DBP in the autoencoder can restrict the amplitude of signifi-cantly without decreasing the reconstruction performance. Naturally, we expect that the smoother autoencoder could improve the resistance to adversarial attack. So in this part, we evaluate the robustness of VAE with DBP comparing with VAE without DBP. We achieve targeted attack both on the latent space and output space.

Here AUDDC is used to evaluate the robustness of autoencoders. The details about calculating the AUDDC are provided in supplemental material. Table 2 shows VAE(NO DBP) and VAE(DBP) performance on adversarial attacks. The AUDDC of VAE(DBP) is much larger than that of VAE(NO DBP), which means applying double back-propagation in VAE successfully improves the robustness against adversarial examples. Then we visually analyzed the attack results for VAE(DBP) and VAE(NO DBP) on each dataset. Fig. 5 shows the attack on MNIST and CelebA dataset respectively. For MNIST, when attacking VAE(NO DBP), the adversarial examples are generated by adding imperceptible perturbations on the original image, leading the output to change to something similar to the target. The adversarial examples of VAE(DBP) are always similar to the target, which means the restriction of the gradient improves the resistance of the VAE to attack. For CelebA, when the reconstruction results approach the target, the adversarial examples of VAE(NO DBP) are noisy image but they are similar to target in VAE(DBP).

5.3 Robustness of DRAW with DBP

In this part, we evaluate the robustness of DRAW(DBP) and DRAW(NO DBP) on MNIST and CelebA dataset. The architectures of DRAW are different in attention window, time steps, size of latent and the LSTM. We achieve targeted attack both on the latent space and output space.

Table 2 illustrates the performance of these autoencoders against adversarial attacks. The AUDDCs of different DRAWs(DBP) are much higher than that of DRAW(NO DBP), which means for autoencoders with more complex architecture such as DRAW, applying double back-propagation can also strengthen the resistance against adversarial examples. In Fig. 5, we show some examples of the attack. Imperceptible additive perturbations on the input image fool the DRAW(NO DBP) to generate something similar to the target. However, adversarial examples can only fool the DRAW(DBP) when they actually look like the target.

5.4 Image Transition based on GMM

The above experiments validate the robustness of equipping double backpropagation on VAE and DRAW. As previously

Fig. 5: Robustness of autoencoder against attacks. The top shows the attack process: a. the original image; b. the target image; c. adversarial example of autoencoder without DBP; d. adversarial example of autoencoder with DBP; e. reconstruction result when there is no DBP; f. reconstruction result when there is DBP. The left column indicates the specific autoencoder (VAE/DRAW) and the dataset (MNIST/CelebA). With additionally considering DBP, the robustness of autoencoders is visually enhanced such that unless the adversarial example is very similar to the target, the reconstruction result can not approach the target image.

discussed, training AE(DBP) can control the magnitude of dz/dx. We build GMMs in the latent space to make df/dz meaningful, so as for df/dx. Then we validate its transferability and robustness against while and black attack. In this part, we consider the dataset MNIST and SVHN. The networks we choose to generate adversarial examples are a simple MLP, LeNet [22] and designed convolutional network CvNet. Attack methods are FGSM and PGD. The details about the networks are provided in supplemental material.

For MNIST dataset, we experiment with two classifiers for demonstration of DBP. combines the autoencoder trained without DBP and with GMM. applies DBP on the autoencoder whose and in (10) are both 1. We then build GMM in the latent space for classification. Four examples of transition in the input space are shown in Figure 6. During the transition, the confidence increases gradually. Transition in the input space of is shown in the left of Figure 6. The confidence increases and output of decoder transforms to the target gradually, but the input just has

TABLE 2: AUDDC for DRAW different structure with and without DBP on MNIST and CelebA. Higher values indicate higher resistance to the attacks.

Fig. 6: Image transition performance with and without DBP. In each of the four groups, we transfer the images by searching on the gradient direction of the classifier. The first line is the reconstruction result of the autoencoder and the second line is the input.

TABLE 3: Accuracies of the Encoder(DBP)+GMM, MLP, LeNet and CvNet for the white box examples on MNIST and SVHN.

some noise and does not change to the target. Then we implement transition in the input space of . The results are visualized in the right of Figure 6. The input changes greatly and shows some apparent features of the target as the confidence changing for the reason that the GMM is smooth and the magnitude of dz/dx is restricted.

So transformation in the input space can only be realized when both the GMMs and encoder are smooth. However, the input images are still fuzzy and hard to recognize because the GMM does not reach high accuracy and encoder part is not smooth enough. GMM may provide the wrong gradient direction which leads the input change to fuzzy one.

As previous discussed, transformation and adversarial attacks do not co-exist. To validate this, we analyze the performance of the classifier we design against white and black box examples. For MNIST, MLP and LeNet are used to generate black box examples [4] where the accuracy of MLP is 97.84% and LeNet is 99.37%. The classifier with encoder and GMMs reaches 95.28% accuracy rate. For SVHN, we train CvNet to generate specific adversarial examples which reaches 91.14% accuracy rate. The classifier consisting of encoder and GMM reaches 83.8% accuracy. Table 3 illustrates the performance of classifiers against white attack. We find that the classifiers combining smooth encoder and GMM shows robustness against the FGSM and PGD attack while the MLP, LeNet and CvNet can be easily attacked. The

classifiers combining smooth encoder and GMM can also defend the adversarial black box attack which is illustrated in Table 4.

TABLE 4: Accuracies of the Encoder(DBP)+GMM for the black box examples on MNIST and SVHN. The adversarial examples are generated by MLP, LeNet and CvNet.

In this paper, we are the first to propose to secure autoencoder such as VAE and DRAW with double backpropagation. It is shown that adversarial attack towards an autoencoder trained with DBP will fail and produce sample similar to the target image. Accordingly, autoencoders with DBP are much more robust without loss of reconstruction reality. Furthermore, we obtain a robust and accurate classifier by modeling the distribution of latent vectors in robust autoencoder by GMM. Our research shows that smoothing the gradient and label contributes to defense of adversarial attack.

This work was partially supported by National Key Research Development Project (No. 2018AAA0100702), National Natural Science Foundation of China (No. 61977046), and 1000-Talent Plan (Young Program).

[1] M. Z. Alom, T. M. Taha, C. Yakopcic, S. Westberg, P. Sidike, M. S. Nasrin, B. C. Van Esesn, A. A. S. Awwal, and V. K. Asari, “The history began from alexnet: A comprehensive survey on deep learning approaches,” arXiv preprint arXiv:1803.01164, 2018.

[2] N. Akhtar and A. Mian, “Threat of adversarial attacks on deep learning in computer vision: A survey,” IEEE Access, vol. 6, pp. 14 410–14 430, 2018.

[3] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” arXiv preprint arXiv:1312.6199, 2013.

[4] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083, 2017.

[5] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 2017, pp. 39–57.

[6] X. Jia, X. Wei, X. Cao, and H. Foroosh, “Comdefend: An efficient image compression model to defend adversarial examples,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 6084–6092.

[7] A. Raghunathan, J. Steinhardt, and P. Liang, “Certified defenses against adversarial examples,” arXiv preprint arXiv:1801.09344, 2018.

[8] X. Li and S. Ji, “Defense-vae: A fast and accurate defense against adversarial attacks,” arXiv preprint arXiv:1812.06570, 2018.

[9] M. Cisse, P. Bojanowski, E. Grave, Y. Dauphin, and N. Usunier, “Parseval networks: Improving robustness to adversarial examples,” in Proceedings of the 34th International Conference on Machine LearningVolume 70. JMLR. org, 2017, pp. 854–863.

[10] F. Liao, M. Liang, Y. Dong, T. Pang, X. Hu, and J. Zhu, “Defense against adversarial attacks using high-level representation guided denoiser,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1778–1787.

[11] A. Prakash, N. Moran, S. Garber, A. DiLillo, and J. Storer, “Deflect-ing adversarial attacks with pixel deflection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8571–8580.

[12] Y. Guo, C. Zhang, C. Zhang, and Y. Chen, “Sparse dnns with improved adversarial robustness,” in Advances in neural information processing systems, 2018, pp. 242–251.

[13] Z. Yan, Y. Guo, and C. Zhang, “Deep defense: Training dnns with improved adversarial robustness,” in Advances in Neural Information Processing Systems, 2018, pp. 419–428.

[14] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014.

[15] F. Tramèr, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, and P. McDaniel, “Ensemble adversarial training: Attacks and defenses,” arXiv preprint arXiv:1705.07204, 2017.

[16] H. Drucker and Y. Le Cun, “Double backpropagation increasing generalization performance,” in IJCNN-91-Seattle International Joint Conference on Neural Networks, vol. 2. IEEE, 1991, pp. 145–150.

[17] A. S. Ross and F. Doshi-Velez, “Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients,” in Thirty-second AAAI conference on artificial intelligence, 2018.

[18] K. Gregor, I. Danihelka, A. Graves, D. J. Rezende, and D. Wierstra, “Draw: A recurrent neural network for image generation,” arXiv preprint arXiv:1502.04623, 2015.

[19] G. Gondim-Ribeiro, P. Tabacof, and E. Valle, “Adversarial attacks on variational autoencoders,” arXiv preprint arXiv:1806.04646, 2018.

[20] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.

[21] D. J. Rezende, S. Mohamed, and D. Wierstra, “Stochastic backpropagation and approximate inference in deep generative models,” arXiv preprint arXiv:1401.4082, 2014.

[22] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner et al., “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.

[23] Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 3730–3738.

[24] R. Müller, S. Kornblith, and G. Hinton, “When does label smoothing help?” arXiv preprint arXiv:1906.02629, 2019.

[25] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.

[26] N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami, “Distillation as a defense to adversarial perturbations against deep neural networks,” in 2016 IEEE Symposium on Security and Privacy (SP). IEEE, 2016, pp. 582–597.

[27] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.

[28] X. Geng, C. Yin, and Z.-H. Zhou, “Facial age estimation by learning from label distributions,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 10, pp. 2401–2412, 2013.

[29] A. Ng et al., “Sparse autoencoder,” CS294A Lecture notes, vol. 72, no. 2011, pp. 1–19, 2011.

[30] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,” Journal of machine learning research, vol. 11, no. Dec, pp. 3371–3408, 2010.

[31] I. Tolstikhin, O. Bousquet, S. Gelly, and B. Schoelkopf, “Wasserstein auto-encoders,” arXiv preprint arXiv:1711.01558, 2017.

[32] Y. Pu, W. Wang, R. Henao, L. Chen, Z. Gan, C. Li, and L. Carin, “Adversarial symmetric variational autoencoder,” in Advances in Neural Information Processing Systems, 2017, pp. 4330–4339.

[33] S. Tang, X. Huang, M. Chen, C. Sun, and J. Yang, “Adversarial attack type i: Cheat classifiers by significant changes,” IEEE transactions on pattern analysis and machine intelligence, 2019.

[34] A. Mustafa, S. H. Khan, M. Hayat, J. Shen, and L. Shao, “Image super-resolution as a defense against adversarial attacks,” arXiv preprint arXiv:1901.01677, 2019.

[35] X. Guo, X. Liu, E. Zhu, and J. Yin, “Deep clustering with convolutional autoencoders,” in International Conference on Neural Information Processing. Springer, 2017, pp. 373–382.

[36] A. Alqahtani, X. Xie, J. Deng, and M. Jones, “A deep convolutional auto-encoder with embedded clustering,” in 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 2018, pp. 4058–4062.

[37] B. Zong, Q. Song, M. R. Min, W. Cheng, C. Lumezanu, D. Cho, and H. Chen, “Deep autoencoding gaussian mixture model for unsupervised anomaly detection,” 2018.

[38] T. K. Moon, “The expectation-maximization algorithm,” IEEE Signal processing magazine, vol. 13, no. 6, pp. 47–60, 1996.

[39] P. Ghosh, A. Losalka, and M. J. Black, “Resisting adversarial attacks using gaussian mixture variational autoencoders,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 541–548.

Chengjin Sun received her BS degree in Nanjing University, Nanjing, China, in 2018. She is now a master student at the Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China. Her research interests are adversarial robustness for deep learning.

Sizhe Chen is a senior student in Department of Automation, Shanghai Jiao Tong University, Shanghai, China. He is now doing research at the Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University. His research interests are adversarial attack, attack transferability, and interpretability of DNN.

Xiaolin Huang (S’10-M’12-SM’18) received the B.S. degree in control science and engineering, and the B.S. degree in applied mathematics from Xi’an Jiaotong University, Xi’an, China in 2006. In 2012, he received the Ph.D. degree in control science and engineering from Tsinghua University, Beijing, China. From 2012 to 2015, he worked as a postdoctoral researcher in ESAT-STADIUS, KU Leuven, Leuven, Belgium. After that he was selected as an Alexander von Humboldt Fellow and working in Pattern Recognition Lab, the Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany. From 2016, he has been an Associate Professor at Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China. In 2017, he was awarded by "1000-Talent Plan" (Young Program). His current research areas include machine learning and optimization.