Table 1: Baseline Model structure for enhancing with convex approach
Table 3: Model having both GCN and Dense Layer
In Table 3, ’In-Nodes’ represents the input nodes to a layer, ’Out-Nodes’ represent the output nodes of a layer, ’AF’ represents the activation function, whereas drop out rate is represented by ’DO’. 5. Results This section describes the results obtained on public datasets with the proposed improvements. In addition, the achieved results will be compared to the state-of-the-art models in literature. 5.1. Datasets For performance evaluation we adopt several semi-supervised classification datasets that are commonly used by other researchers. The set of dataset comprise of Cora, Citeseer, Pubmed Sen et al. [2008], and Cora-ML Bojchevski and G¨unnemann [2018]. The setup is the same as being followed in Vashishth et al. [2019]. Our aim concerns to classify documents into one of the pre-defined classes. Datasets represent citation networks, in which each document is encoded using bag-of-words features with undirected edges between nodes. The dataset statistics is summarized in table 4. Label mismatch concerns the fraction of edges between nodes with different labels in the training data. The datasets have substantially low label mismatch rate except Cora-ML.
Table 4: Dataset statistics.
mation. 5.3. Comparison We have summarized our results by showing the best results of all the enhancements for all the datasets. Table 5 shows the accuracy of all the models mentioned in Section 4. We have been successful in getting state-of-the-art result on one dataset as well as very close to the state-of-the-art work done till now on the other three datasets as highlighted in green in Table 5. On CoraML dataset we achieved the current best accuracy of 86.9 ± 0.4 using DConfGCN model. This is the current state-of-the-art based on our knowledge as the most recent papers i.e. Dual-GCN, LGCN, and Fast-GCN did not reported their results on CoraML dataset. In case of Citeseer dataset, the best result which we achieved is 73.26%, this is more than Dual GCN and 0.3% less from LGCN. This makes our accuracy with ConvConfGCN the second best till date. However, just to highlight that LGCN Gao et al. [2018] report only the best result whereas our result are based on 100 run which are more stronger compare to reporting one highest performance. We have got the 3rd best accuracy for Pubmed dataset i.e. 79.8 ± 0.4. Finally, on Cora dataset, we achieved 82.1±1.2 accuracy with DGCN that is better than baseline GCN and ConfGCn by slight margin, but at 4th position overall in the list. One of the reason for not having the best result for Citeseer, Cora, and Pubmed could be that the best reported results in the papers Gao et al. [2018]; Monti et al. [2018]; Liang et al. [2015] are not having the mean performance over multiple runs. Another reason is that, our model can not be directly compared with model like LGCN as it uses regular convolutional kernel in their model. Rather designing new kernels to work on graph data, in LGCN the authors organized the graph data in a way that normal convolutional kernel can operate over it and learn feature from them. These enhancement and results are reported to provide baseline for future works to be done in the field of SSL for the Graphs. In table 6 execution time for PubMed dataset is shown. As the size of the features in each dataset varies, that is why the time (in seconds) per epoch varies for each dataset. GCN and its enhancements are faster compared to confGCN and its enhancements. While optimizing based on hyper-parameters, we found that the major reduction in computational cost was due to usage of the
Table 5: Performance comparison of different methods on described datasets.
Table 6: Execution time on Pubmed dataset
James Atwood and Don Towsley. Diffusion-convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1993–2001, 2016.
Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of machine learning research, 7(Nov):2399–2434, 2006.
Aleksandar Bojchevski and Stephan G¨unnemann. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. In International Conference on Learning Representations, pages 1–13, 2018.
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. Spectral networks and locally connected networks on graphs. In International Conference on
Learning Representations (ICLR2014), CBLS, April 2014, 2014.
Micha¨el Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in neural information processing systems, pages 3844– 3852, 2016.
Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. Large-scale learnable graph convolutional networks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1416–1424. ACM, 2018.
Mikael Henaff, Joan Bruna, and Yann LeCun. Deep convolutional networks on graph-structured data. CoRR, abs/1506.05163, 2015.
Thomas N. Kipf and Max Welling. Semi-supervised clas- sification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. Gated graph sequence neural networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
Meimei Liang, Futao Zhang, Gulei Jin, and Jun Zhu. Fast- gcn: a gpu accelerated tool for fast gene co-expression networks. PloS one, 10(1):e0116776, 2015.
Renjie Liao, Marc Brockschmidt, Daniel Tarlow, Alexan- der L. Gaunt, Raquel Urtasun, and Richard S. Zemel. Graph partition neural networks for semi-supervised classification. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings, 2018.
Franco Manessi and Alessandro Rozza. Learning com- binations of activation functions. In 2018 24th International Conference on Pattern Recognition (ICPR), pages 61–66. IEEE, 2018.
Diego Marcheggiani and Ivan Titov. Encoding sentences with graph convolutional networks for semantic role labeling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1506–1515, 2017.
Federico Monti, Oleksandr Shchur, Aleksandar Bo- jchevski, Or Litany, Stephan G¨unnemann, and Michael M Bronstein. Dual-primal graph convolutional networks. arXiv preprint arXiv:1806.00770, 2018.
Tore Opsahl. Triadic closure in two-mode networks: Re- defining the global and local clustering coefficients. Social Networks, 35(2):159–167, 2013.
Matan Orbach and Koby Crammer. Graph-based trans- duction with confidence. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 323–338. Springer, 2012.
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deep- walk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 701–710. ACM, 2014.
Alessandro Rozza, Mario Manzo, and Alfredo Petrosino. A novel graph-based fisher kernel method for semi-supervised learning. In 2014 22nd International Conference on Pattern Recognition, pages 3786–3791. IEEE, 2014.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80, 2008.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29 (3):93–93, 2008.
Alexander Strang, Oliver Haynes, Nathan D Cahill, and Darren A Narayan. Generalized relationships between characteristic path length, efficiency, clustering coeffi-cients, and density. Social Network Analysis and Mining, 8(1):14, 2018.
Shikhar Vashishth, Prateek Yadav, Manik Bhandari, and Partha Talukdar. Confidence-based graph convolutional networks for semi-supervised learning. In The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16-18 April 2019, Naha, Okinawa, Japan, pages 1792–1801, 2019.
Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li`o, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, 2018.
Jason Weston, Fr´ed´eric Ratle, Hossein Mobahi, and Ro- nan Collobert. Deep learning via semi-supervised embedding. In Neural Networks: Tricks of the Trade, pages 639–655. Springer, 2012.
Prateek Yadav, Madhav Nimishakavi, Naganand Yadati, Shikhar Vashishth, Arun Rajkumar, and Partha Talukdar. Lov´asz convolutional networks. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1978–1987, 2019.
Zhilin Yang, William W Cohen, and Ruslan Salakhutdi- nov. Revisiting semi-supervised learning with graph embeddings. In Proceedings of the 33rd International Conference on International Conference on Machine Learning-Volume 48, pages 40–48. JMLR. org, 2016.
Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03), pages 912–919, 2003.