Recent research on interpretability explained the predictions in a post-hoc manner: given a trained predictive model with predicted scores F (x), use extracted information to explain how the model made predictions. Such extracted information can be analyzed and displayed by data analysis techniques, such as gradient-based methods [Tsang et al., 2018a; Sundararajan et al., 2017; Shrikumar et al., 2017; Yosinski et al., 2015], and sensitivity analysis [Ribeiro et al., 2016; Lundberg and Lee, 2017], or by using a mimic model to minimize
, such as tree- and rule-based models [Li et al., 2019; Che et al., 2016; Letham et al., 2015; Wang and Lin, 2019]. Summing up, the post-hoc methods do not change or improve the underlying predictive model and use simpler forms, such as linear models, to explain the relationships in a complex predictive model, thereby making it easier for users to understand the interpretations.
Post-hoc methods, though effective in interpreting models in an easy-to-understand way, have two limitations: (a) When we use different post-hoc methods to explain the same predictions, the explanations may be contradictory with each other [Alvarez-Melis and Jaakkola, 2018]. The human cost for determining these methods and the samples to train the post-hoc models are prohibitive with larger datasets. Moreover, it is unclear whether we can aggregate different post-hoc explanations [Tan et al., 2018]; (b) The post-hoc methods focus on using simpler models to provide a higher descriptive accuracy, which measures the degree to which a post-hoc method properly describes the patterns learned by a predictive model. Usually, the simplicity of post-hoc methods helps users understand the patterns, but provides imperfect representations of nonlinear relationships among variables in complex black-box models. Their interpretability is obtained at the expense of prediction accuracy [Murdoch et al., 2019].
To overcome these limitations, there is a need for modelbased interpretations, which come from the construction of the prediction model [Murdoch et al., 2019]. Such models (a) have a predefined structure and can readily describe relationships between input features and predictions once the models are learned. It does not require the determination of the post-hoc models and is estimated on all training data, therefore it avoids the interpretations being changed dramatically when different methods or subsets of the data are used. (b) Such models should interpret the relationships between input features and predictions in a simple and explicit way, such as describing feature contributions via providing feature shapes that can be easily understood by users. In the meanwhile, it should be expressive enough to properly fit the data and achieve good prediction performance.
In this work, we propose a hybrid Piecewise Linear and Deep (PiLiD) model under a Wide & Deep (short for W&D) scheme [Cheng et al., 2016] as shown in Figure 1. The proposed model is comprised of a piecewise linear component and a nonlinear component. The first component uses piecewise linear functions to approximate the complex relationships between the input features and predictions. Such a form is explicit enough to describe the feature contributions by providing feature shapes, yet increases the expressiveness of the model. The other component uses a multi-layer perceptron (MLP) to capture the feature interactions and increase the prediction performance. The two components are jointly trained and the interpretability (in the form of piecewise linear functions) is obtained once the model is learned. This work has the following contributions:
• We propose the PiLiD model that enhances the interpretability under a W&D scheme. Different from the post-hoc methods, the interpretability is obtained once the prediction model is learned.
• The proposed PiLiD model is flexible for adaptations. We develop the PiLiB model, a variant to extract interpretable higher-order interactions.
• We integrate a predefined interpretable structure into the linear component to decipher the contribution of features by piecewise linear approximation. Such a form provides explicit feature shapes while preserving high expressiveness.
• As a result of the joint-training scheme, we show that the model can describe complex feature shapes while improving model performance.
2.1 Wide and Deep Scheme
The Wide & Deep (W&D) scheme jointly trains a linear model and a deep neural network to benefit from memorization and generalization [Cheng et al., 2016]. This framework takes advantage of both the Wide (linear model) and Deep (deep neural network) components by jointly training both together, and thus outperforms the models with either Wide or Deep component in a large-scale recommender system [Li et al., 2019; Guo et al., 2018; Zheng et al., 2017]. The Wide component of the W&D scheme also provides interpretations of the features’ main effect. Hence W&D has been used to develop interpretable models. Tsang et al. [2018a] developed a model that modified the Deep component to only extract pairwise interactions among features. Similarly, Kraus and Feuerriegel [2019] changed the Deep component to a recurrent neural network to capture the temporal information. Wang and Lin [Wang and Lin, 2019] proposed a hybrid model, in which an interpretable model and a black-box model function together to obtain a good trade-off between model transparency and performance. Effective as they are, these W&D-based interpretable models’ prediction performance is worse than the complex black-box model (the Deep part alone).
2.2 Interpretable Methods
There are mainly two categories of interpretable machine learning methods. The first refers to post-hoc methods which initially learn an original predictive model and then explain how the model obtains the underlying predictions. This type of models usually fall into two sub-categories, prediction-level and dataset-level interpretations [Murdoch et al., 2019].
The prediction-level methods focus on locally explaining why the model makes a particular prediction. They tell users what individual features or feature interactions are the most important by standard data analysis such as the gradient-based methods [Tsang et al., 2018a; Sundararajan et al., 2017; Shrikumar et al., 2017; Yosinski et al., 2015], the sensitivity analysis [Ribeiro et al., 2016; Lundberg and Lee, 2017], the step-wise feature removal approaches [Schwab and Karlen, 2019], and the mimic models [Li et al., 2019; Che et al., 2016; Letham et al., 2015; Wang and Lin, 2019]. In contrast, the dataset-level methods are interested in global explanations that explore more general relationships learned by the models. Tan et al. [2018] described feature contributions by providing a global additive value function. Both prediction- and dataset-level methods suffer from the aforementioned drawbacks of post-hoc scheme.
Another category of approaches predefines an interpretable structure, and provides insight into relationships between input features and predictions once the model is learned [Kraus and Feuerriegel, 2019]. Generalized additive model (GAM) can model extreme nonlinearity on individual features [Hastie and Tibshirani, 1986; Lou et al., 2012], but it cannot model feature interactions. In this regard, Lou et al. [2013] developed the GAM model, which first learns a base GAM without any interactions and then selects a number of pairwise interactions that minimize residuals. Unfortunately, the obtained individual and interacting feature contributions can be extremely complex because there is no regularization on their shapes, and this model is slow to converge. For this reason, Tsang et al. [2018b] divided a neural network into several same-sized blocks and used
regularization to model both uni-variate and high-order interactions, however, only a subset of individual features can be interpreted and the accuracy decreases because the model does not capture the nonlinearity other than the learned feature interactions. The main challenge, as the main purpose of this paper, is to come up with a structure that is simple enough to help users understand the rationale behind the predictions, while keeping the model sophisticated to take care of the non-interpretable nonlinearity in the data.
3.1 Problem Setting
Vectors are represented by boldface lower-case letters, for example x. Note that the index for i-th vector is in brackets, such as xis the i-th data point, and the j-th element of a vector w is w
; matrices are denoted by boldface capital letters, for example W and the element (i, j) of W is
. Let
be a set of N data samples, where x
is the i-th data point described by m features, and
is a real value (class label) to be predicted in regression (classification) problems.
We assume that there exists a mapping function describing the relationship between the prediction and the individual features (main effects) and interacting features (interac-
Figure 1: The proposed PiLiD (left) and PiLiB (right) models. In PiLiB, the whole neural network in the nonlinear component of PiLiD is replaced with B same-sized smaller neural networks (blocks).
tion effects). To predict , we define:
where is a constant term1,
is a marginal value function of j-th feature, and
is a function of all features. Eq.(1) describes (a) a regression model if
is the identity, and (b) a classification model if
is the logistic function of the identity.
Given the following definitions, we use piecewise linear functions to approximate the marginal value functions. Definition 1. The characteristic points are some predefined values to partition the whole feature value scale into several pieces.
where
1, . . . , m, and Definition 3. The marginal value vector u is defined as:
where and
, is the difference of marginal values between two consecutive characteristic points.
We use an MLP to approximate the interaction effects . Here it is a feed-forward neural network with L hidden layers. There are
hidden units in the l-the layer and
. The layer matrices are denoted by W
, and bias vectors are denoted by b
. The non-linear activation function is denoted as
. Let w
and
be the final weight vector and bias for the output y. In this way, the MLP with L layers can be represented by:
3.2 Model Description and Training Process
Given Definition 3 and Eq.(6), the Eq.(1) can be reformulated as follows:
According to Definition 2, the vectorized input layer in piecewise linear component transforms the original data into a ‘wider’ vector by piecewise linear partition. The input of each unit in this layer corresponds to an element in vector , and every
units are decomposed from one feature. The first layer in the piecewise linear component has
units and has a form y
b, where w
and b
. There is no any activation function. The next layer groups every
units (m groups in total), and then sums the outputs of units in each group. Note that when
, the piecewise linear component PiLiD degenerates to a simple linear model providing a single value describing feature contributions, i.e., feature attributions. When
, all feature values become characteristic points, PiLiD has to fit the curve point-by-point. Thus, it can fit any curve given sufficient data. Without the constrain in
, it may use an extremely complex shape to fit the curve, causing the over-fitting problem.
The outputs of two components are fed into a specific loss function for joint training. The joint training process uses the mini-batch stochastic optimization algorithm to backpropagate the gradients from both components at the same time. The general loss function is:
(8) where
is a neural network with
parameters,
b are the vectors of parameters in the piecewise linear component,
are the parameters, including weights matrices W
, bias vectors b
1, . . . , L, weight vector w
and bias term
for the output of nonlinear component,
is the loss function. Specifically,
is the cross-entropy loss for classification problems and the mean square error for regression problems.
is a prede-fined coefficient for the regularization function
that can be
or
regularization. The gradient-based algorithms can make the optimal solution robust and fast to converge by initializing the parameters in a neural network [LeCun et al., 2015]. In this study, the parameters in the piecewise linear component are not randomly initialized because their optimal values are supposed to indicate the contributions of individual features to the predictions. For this reason, we use the parameters obtained by a simple linear regression to initialize the parameters in the piecewise linear component. Such initialization forces the optimization algorithm starts from a more promising and interpretable solution region. As for the nonlinear component, we use a standard Gaussian initialization. The pseudo code for learning the proposed model is presented in Algorithm 1 (for brevity, we raise a regression problem as an example):
Algorithm 1 Training process for the proposed PiLiD.
Ensure: Parameters
1: Vectorize the feature values in xinto
according to Definition 2. Let
and y =
.
2: Initialize , . . . ,
, w
y, b
(0, . . . ,
, W
, σ), and b
, . . . ,
, l = 1, . . . , L.
3: Standard mini-batch stochastic gradient optimization algorithm.
3.3 A Variant
To explore interacting effects in nonlinear component and demonstrate the flexibility of the proposed model, we substitute the nonlinear component with the state-of-the-art Neural Interaction Transparency (NIT) model to decipher the high-order feature interactions [Tsang et al., 2018b].
The original NIT model partitions a whole feed-forward neural network into several ‘blocks’ with the same size. It uses the regularization to force some hidden units to have zero weights in each ‘block’, and adds an extra term
in the loss function to encourage the model to learn smallersized interactions. We refer interested readers to Tsang et al.[2018b]. In this work, we propose a variant of PiLiD, namely the Piecewise Linear and Blocks (PiLiB) to model all individual features and important interacting features.
The piecewise linear component in PiLiB is same as that in PiLiD. The nonlinear component is divided into B same-sized block (shown on the right in Figure 1). The loss function of PiLiD is:
T (G()))
)) (9)
= max
max
Note that G if a feature is active in a block, thus the estimated interaction order is defined as
1, . . . , B. The first term in limits the maximum interaction order to be equal to or less than K. Different with Tsang et al. [2018b], the second term encourages the model to learn pairwise interactions. In this way, we push the modelling of the main effects to the piecewise linear component and the pairwise interactions to the nonlinear component. The algorithm for training parameters in PiLiB is in supplementary materials.
Our experiments are conducted on synthetic and real datasets. The simulations aim to answer the following questions: (a) Can the proposed PiLiD accurately describe the actual marginal value functions while outperforming other predictive models? (b) Does the initialization process help improve interpretability and accuracy? The experiments on real datasets aim to show the advantages of PiLiD and PiLiB over several state-of-the-art interpretable methods.
4.1 Simulations
The synthetic data is generated as follows: (a) Randomly determine the coefficients of m polynomial functions (in 10 degrees) as the actual marginal value functions. (b) Randomly determine N objects and their feature values from interval [0, 1]. (c) Use actual marginal value functions to calculate the
Table 1: Average MSE for regression problems when N = 20000 (The best results are marked in bold. The number following PiLiD is the predefined number of intervals for piecewise linear functions.).
Table 2: Averaged MSE given proposed initialization and Gaussian initialization (are significantly different at 10% and 5% levels.).
marginal values of N objects and randomly add some feature interactions. (d) For regression problem, y is the summation of m marginal values and selected interactions; For binary classification problem, we use a logistic function to assign 0 and 1 labels. In this section, we present the results of regression problems due to length limit. The classification problem has similar results, which are presented in the supplementary materials.
The MLP used in PiLiD has a 100-200-400-400-200-100-1 structure. We use ADAM with 0.005 learning rate as the standard optimizer in Algorithm 1. For each problem setting, we run 20 trials. 80% is selected as the training set and 20% as the testing set. We present experimental results of regression problems with N = 20000 in Table 1. The full simulation results can refer to supplementary materials.
When the synthetic data size is small, both MLP and PiLiD perform worse than classic statistical learning models (see results in supplementary materials when N = 5000), such as SVM and Random Forest. When the data size is larger, more data are fed into the neural network, as a result, both MLP and PiLiD perform better than statistical learning models (Table 1). Encouragingly, we find that PiLiD outperforms MLP in most tasks given the same neural network setting and the number of iterations. At last, as the increase in the number of predefined intervals , the piecewise linear component becomes more complex, and in turn increases the model accuracy. This indicates that the piecewise linear component could actually improve the performance if the learned interpretability is close to the actual one. Nevertheless, we suggest setting
to avoid the overfitting problem and additional computational cost.
Summing up, the proposed PiLiD performs better because
Figure 2: An example of the learned marginal value functions given different initializations when curves are averaged over 20 trials. The solid blue curves are the actual functions and the dashed black curves are learned functions.
(a) the piecewise linear form increases model expressiveness; and (b) the proposed initialization in Algorithm 1 and the joint training process help obtain optimal solutions (see results in Table 2). To demonstrate that the proposed initialization helps obtain rational interpretations, we provide an example of learned marginal value functions by two initializations in Figure 2. Obviously, although Gaussian initialization can sometimes obtain results as accurate as the proposed initialization, it fails to properly describe the actual marginal value functions.
4.2 Real Data
We conduct experiments using four real datasets to compare the prediction performance of the proposed PiLiD and PiLiB, and a set of baseline models including global additive explanation model (GA) [Tan et al., 2018], GAM [Lou et al.,
Table 3: Predictive performance of the proposed PiLiD and PiLiB. In both models,
Table 4: Description on datasets.
Figure 3: The visualization of learned feature shapes (Age) in dataset Skill by PiLiD.
2012], GAM [Lou et al., 2013], and NIT [Tsang et al., 2018b] models. The datasets are described in Table 42.
Table 3 presents the performance of different models. Comparing with NIT and MLP, the proposed PiLiD performs better because it benefits from the extra piecewise linear component that considers more complex feature shapes and the jointly training process. Comparing with the GAM family, although GAMs can model extremely complex feature shapes, they cannot account for possible higher-order interactions (except for GAM, which only deals with the pairwise interactions). Therefore, both PiLiD and PiLiB outperform all GAMs.
To demonstrate the interpretability of PiLiD and PiLiB, we present feature shapes of Age in dataset Skill given different . First, from Figure 3(a) and other plots, we stress the importance of providing feature shapes rather than only feature attributions that uses a single value to describe the feature contribution. All other plots (
) can capture the changing tendency of the relationship between Age and players’ skill estimation (prediction). Such tendency indicates
Figure 4: The visualization of interacting features in dataset Spambase by PiLiB.
that, at first, the aging process positively affects players’ skill because they can gain experience over time, but it changes when players are older because their response speed is affected. Second, as stated in Schwab and Karlen [2019], the explanations should be robust. From plots (b) to (e) in Figure 3, the obtained feature shapes have a similar tendency given predefined number of intervals. That is because the proposed initialization enforces the optimization to start from a more promising region, and thus makes the interpretability more robust and stable.
In Figure 4, we present an example of the learned interacting effects by PiLiB. These relationships help to understand why an email is classified as spam. For example, an email is more likely to be a spam if we observe the overuse of continuous capital letters and less use of the term receive. Interpreting and applying these extracted interactions needs further examinations, and will be our future work.
We propose a hybrid interpretable model that uses piecewise linear functions to approximate the individual feature contributions. It is flexible to adapt for other model structures. The experiments demonstrate that the model is explicit enough for users to understand and also has state-of-the-art prediction performance. This research shed new light on the joint learning for interpretability and predictability, and the feasibility of using the learned interpretability to enhance the prediction performance.
6.1 Algorithm for Training PiLiB
Given loss function , the training process of PiLiB is in two phases. At the first stage, we jointly train the linear component with parameter regularization
and deep component with only
regularization. After the maximum interaction order in the deep component is smaller or equal to the allowable K, we go to next phase. In the second phase, the trained ‘masks’ for the first layer are fixed and on this basis, we optimize all parameters in the model with standard regularization. The training process can be summarized by Algorithm 2.
6.2 Extend Simulation Results
Here we present full simulation results of regression and clas-sification problems. Tables 5 to 10 present the simulation results given different numbers of objects and features for two problems. We emphasize three interesting patterns:
• As stated in main paper, when the dataset is smaller, for example N = 5000, since the proposed PiLiD is under a deep learning scheme, it requires sufficient data to train parameters. Therefore, PiLiD and MLP’s performance on simulation data are worse than some traditional predictive models. However, when the datasets are larger (N = 10000, 15000, 20000), the PiLiD perform better.
• In classification problems, the performance of the proposed PiLiD is on a par with or comparable to MLP. The proposed PiLiD performs better than MLP given same regression problem settings. That is because the pretriaing process in Algorithm 1 helps PiLiD find better solutions. Moreover, the piecewise linear form also increases the expressiveness of the entire model, thereby being more powerful than a pure MLP.
• There is a tendency that the predictive performance increases along with the increase of predefined number of intervals. It makes sense because the larger number of intervals, the more complex the linear component. As stated in main paper, an extreme simulation is let , then the linear component becomes a GAM, which is more powerful than linear models. However, larger number of intervals will require more computational cost and data samples, as a trade-off, we suggest using
.
6.3 Real-world Datasets
The datasets used in main paper can be downloaded from the following websites:
• Bike sharing: https://archive.ics.uci.edu/ml/datasets/ Bike+Sharing+Dataset
• Bank marketing: https://archive.ics.uci.edu/ml/datasets/ Bank+Marketing
• Spambase: https://archive.ics.uci.edu/ml/datasets/ Spambase
• Skill: http://archive.ics.uci.edu/ml/datasets/SkillCraft1+ Master+Table+Dataset
[Alvarez-Melis and Jaakkola, 2018] David Alvarez-Melis and Tommi S Jaakkola. Towards robust interpretability with self-explaining neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 7786–7795. Curran Associates Inc., 2018.
[Che et al., 2016] Zhengping Che, Sanjay Purushotham, Robinder Khemani, and Yan Liu. Interpretable deep models for icu outcome prediction. In AMIA Annual Symposium Proceedings, volume 2016, page 371. American Medical Informatics Association, 2016.
[Cheng et al., 2016] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide and deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, pages 7–10. ACM, 2016.
[Guo et al., 2018] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, Xiuqiang He, and Zhenhua Dong. Deepfm: An end-to-end wide & deep learning framework for ctr prediction. arXiv preprint arXiv:1804.04950, 2018.
[Hastie and Tibshirani, 1986] Trevor Hastie and Robert Tib- shirani. Generalized additive models. Statistical Science, 1(3):297–318, 1986.
[Kraus and Feuerriegel, 2019] Mathias Kraus and Stefan Feuerriegel. Forecasting remaining useful life: Interpretable deep learning approach via variational bayesian inferences. Decision Support Systems, 125:113100, 2019.
[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436, 2015.
[Letham et al., 2015] Benjamin Letham, Cynthia Rudin, Tyler H McCormick, David Madigan, et al. Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics, 9(3):1350–1371, 2015.
[Li et al., 2019] Pan Li, Zhen Qin, Xuanhui Wang, and Donald Metzler. Combining decision trees and neural networks for learning-to-rank in personal search. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2032–2040. ACM, 2019.
[Lou et al., 2012] Yin Lou, Rich Caruana, and Johannes Gehrke. Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 150–158. ACM, 2012.
Table 5: Average MSE for regression problems when N = 5000 (The best results are marked in bold. The number following PiLiD is the predefined number of intervals for piecewise linear functions.).
Table 6: Average MSE for regression problems when N = 10000.
Table 7: Average MSE for regression problems when N = 15000.
Table 8: Average AUC for classification problems when N = 5000.
Table 9: Average AUC for classification problems when N = 10000.
[Lou et al., 2013] Yin Lou, Rich Caruana, Johannes Gehrke, and Giles Hooker. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 623–631. ACM, 2013.
[Lundberg and Lee, 2017] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, pages 4765–4774, 2017.
[Murdoch et al., 2019] W James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. Interpretable machine learning: definitions, methods, and applications. arXiv preprint arXiv:1901.04592, 2019.
[Ribeiro et al., 2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should I trust you? explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144. ACM, 2016.
[Schwab and Karlen, 2019] Patrick Schwab and Walter Karlen. Cxplain: Causal explanations for model interpretation under uncertainty. In Advances in Neural
Information Processing Systems, pages 10220–10230, 2019.
[Shrikumar et al., 2017] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3145–3153. JMLR. org, 2017.
[Sundararajan et al., 2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3319–3328. JMLR. org, 2017.
[Tan et al., 2018] Sarah Tan, Rich Caruana, Giles Hooker, Paul Koch, and Albert Gordo. Learning global additive explanations for neural nets using model distillation. arXiv preprint arXiv:1801.08640, 2018.
[Tsang et al., 2018a] Michael Tsang, Dehua Cheng, and Yan Liu. Detecting statistical interactions from neural network weights. International Conference on Learning Representations, 2018.
[Tsang et al., 2018b] Michael Tsang, Hanpeng Liu, Sanjay Purushotham, Pavankumar Murali, and Yan Liu. Neural interaction transparency (nit): disentangling learned
Table 10: Average AUC for classification problems when N = 15000.
interactions for improved interpretability. In Advances in Neural Information Processing Systems, pages 5804– 5813, 2018.
[Wang and Lin, 2019] Tong Wang and Qihang Lin. Hybrid predictive model: When an interpretable model collaborates with a black-box model. Journal of Machine Learning Research, 1:1–48, 2019.
[Yosinski et al., 2015] Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579, 2015.
[Zheng et al., 2017] Zibin Zheng, Yatao Yang, Xiangdong Niu, Hong-Ning Dai, and Yuren Zhou. Wide and deep convolutional neural networks for electricity-theft detection to secure smart grids. IEEE Transactions on Industrial Informatics, 14(4):1606–1615, 2017.