Reading List for Graph Neural Networks

1. Overview

• Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, Philip S. Yu: A Comprehensive Survey on Graph Neural Networks. IEEE Trans. Neural Networks Learn. Syst. 32(1): 4-24 (2019).

• Josephine M. Thomas, Alice Moallemy-Oureh, Silvia Beddar-Wiesing, Clara Holzhüter:Graph Neural Networks Designed for Different Graph Types: A Survey. CoRR abs/2204.03080 (2022)

• John Boaz Lee, Ryan A. Rossi, Sungchul Kim, Nesreen K. Ahmed, Eunyee Koh: Attention Models in Graphs: A Survey. ACM Trans. Knowl. Discov. Data 13(6): 62:1-62:25 (2019)

2. Recurrent GNNs

• Li Y, Tarlow D, Brockschmidt M, Zemel R. 2015c. Gated graph sequence neural networks.
  arXiv preprint arXiv:1511.05493

• Gilmer, J.; Schoenholz, S.S.; Riley, P.F.; Vinyals, O.; Dahl, G.E. Neural Message Passing for Quantum Chemistry. In Proceedings of the 34th International Conference on Machine Learning (ICML 2017); PMLR: Sydney, NSW, Australia, 2017; Vol. 70, Proceedings of Machine Learning Research, pp. 1263–1272.

• Aynaz Taheri, Kevin Gimpel, and Tanya Berger-Wolf. 2019. Learning to represent the evolution of dynamic graphs with recurrent models. In Proceedings of the World Wide Web Conference. 301–307

3. Graph autoencoders

• Wang, D.; Cui, P.; Zhu, W. Structural Deep Network Embedding. In Proceedings of the 22nd International Conference on Knowledge Discovery and Data Mining (SIGKDD 2016); Association for Computing Machinery: New York, NY, USA, 2016; KDD ’16, p. 1225–1234. https://doi.org/10.1145/2939672.2939753.

• Tu, K.; Cui, P.; Wang, X.; Wang, F.; Zhu, W. Structural Deep Embedding for Hyper-Networks. In Proceedings of the 32nd Conference on Artificial Intelligence (AAAI 2018); AAAI Press: New Orleans, Louisiana, USA, 2018; pp. 426–433.

• Khoshraftar, S.; Mahdavi, S.; An, A.; Hu, Y.; Liu, J. Dynamic Graph Embedding via LSTM History Tracking. In Proceedings of the International Conference on Data Science and Advanced Analytics (DSAA 2019); IEEE: Washington, DC, USA, 2019; pp. 119–127. https://doi.org/10.1109/DSAA.2019.00026.

4. Spectral GNNs

• Bruna, J.; Zaremba, W.; Szlam, A.; LeCun, Y. Spectral Networks and Locally Connected Networks on Graphs. In Proceedings of the 2nd International Conference on Learning Representations (ICLR 2014).

• Defferrard, M.; Bresson, X.; Vandergheynst, P. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NeurIPS 2016); , 2016; pp. 3837–3845.

• Kipf, T.N.; Welling, M. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR 2017); OpenReview.net: Toulon, France, 2017

5. Spatial GNNs 1

• Hamilton, W.L.; Ying, Z.; Leskovec, J. Inductive Representation Learning on Large Graphs. In Proceedings of the 30th Annual Conference on Neural Information Processing Systems (NIPS 2017); , 2017; pp. 1024–1034

• Chen, J.; Ma, T.; Xiao, C. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. In Proceedings of the 6th International Conference on Learning Representations (ICLR 2018); OpenReview.net: Vancouver, BC, Canada, 2018

• Chiang, W.; Liu, X.; Si, S.; Li, Y.; Bengio, S.; Hsieh, C. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. In Proceedings of the 25th International Conference on Knowledge Discovery & Data Mining (KDD 2019); ACM: Anchorage, Alaska, USA, 2019; pp. 257–266. https://doi.org/10.1145/3292500.3330925.

6. Spatial GNNs 2

• Li, G.; Xiong, C.; Thabet, A.K.; Ghanem, B. DeeperGCN: All You Need to Train Deeper GCNs. CoRR 2020, abs/2006.07739 [2006.07739].

• Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, Viktor K. Prasanna: GraphSAINT: Graph Sampling Based Inductive Learning Method. ICLR 2020

• Chen, M.; Wei, Z.; Huang, Z.; Ding, B.; Li, Y. Simple and Deep Graph Convolutional Networks. In Proceedings of the 37th International Conference on Machine Learning (ICML 2020); PMLR: Virtual Event, 2020; Vol. 119, Proceedings of Machine Learning Research, pp. 1725–1735

7. Attentive GNNs 1

• Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio: Graph Attention Networks. CoRR abs/1710.10903 (2017)

• Yang Ye, Shihao Ji: Sparse Graph Attention Networks. IEEE Trans. Knowl. Data Eng. 35(1): 905-916 (2023)

• Haonan, L.; Huang, S.H.; Ye, T.; Xiuyan, G. Graph Star Net for Generalized Multi-Task Learning. arXiv e-prints 2019, p.arXiv:1906.12330, [arXiv:cs.SI/1906.12330].

8. Attentive GNNs 2

• Kim, D.; Oh, A. How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision. In Proceedings of the 9th International Conference on Learning Representations (ICLR 2021); OpenReview.net: Virtual Event, Austria, 2021

• Wang, X.; Ji, H.; Shi, C.; Wang, B.; Ye, Y.; Cui, P.; Yu, P.S. Heterogeneous Graph Attention Network. In Proceedings of the World Wide Web Conference (WWW 2019); ACM: San Francisco, CA, USA, 2019; pp. 2022–2032. https://doi.org/10.1145/3308558.3313562.

• Wang, G.; Ying, R.; Huang, J.; Leskovec, J. Improving Graph Attention Networks with Large Margin-based Constraints. CoRR 2254 2019, abs/1910.11945, [1910.11945].

9. Subgraph aggregation

• Leonardo Cotta, Christopher Morris, and Bruno Ribeiro. Reconstruction for powerful graph representations. Advances in Neural Information Processing Systems, 34, 2021.

• Lingxiao Zhao, Wei Jin, Leman Akoglu, and Neil Shah. From stars to subgraphs: Uplifting any gnn with local structure awareness. arXiv preprint arXiv:2110.03753, 2021.

• Beatrice Bevilacqua, Fabrizio Frasca, Derek Lim, Balasubramaniam Srinivasan, Chen Cai, Gopinath Balamurugan, Michael M Bronstein, and Haggai Maron. Equivariant subgraph aggregation networks. arXiv preprint arXiv:2110.02910, 2021.

10. GNNs limitations and solutions

• Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, Martin Grohe: Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks. AAAI 2019: 4602-4609

• Hualei Yu, Jinliang Yuan, Yirong Yao, Chongjun Wang: Not all edges are peers: Accurate structure-aware graph pooling networks. Neural Networks 156: 58-66 (2022)

• Juan Shu, Bowei Xi, Yu Li, Fan Wu, Charles A. Kamhoua, Jianzhu Ma: Understanding Dropout for Graph Neural Networks. WWW (Companion Volume) 2022: 1128-1138

11. GNNs with Gaussian distribution

• Kipf, T.N.; Welling, M. Variational Graph Auto-Encoders. CoRR 2016, abs/1611.07308, [1611.07308].

• Santos, L.D.; Piwowarski, B.; Gallinari, P. Multilabel Classification on Heterogeneous Graphs with Gaussian Embeddings. In Proceedings of the Machine Learning and Knowledge Discovery in Databases European Conference (ECML PKDD 2016); Springer: Riva del Garda, Italy, 2016; Vol. 9852, Lecture Notes in Computer Science, pp. 606–622. https://doi.org/10.1007/978-3-319-46227-138

• Zhu, D.; Cui, P.; Wang, D.; Zhu, W. Deep Variational Network Embedding in Wasserstein Space. In Proceedings of the 24th International Conference on Knowledge Discovery & Data Mining (KDD 2018); Guo, Y.; Farooq, F., Eds.; ACM: London, UK, 2018; pp. 2827–2836. https://doi.org/10.1145/3219819.3220052

12. Transformer with GNNs

• Shi, Y.; Huang, Z.; Feng, S.; Zhong, H.; Wang, W.; Sun, Y. Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification. In Proceedings of the Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI 2021); Zhou, Z., Ed.; ijcai.org: Virtual Event / Montreal, Canada, 2021; pp. 1548–1554. https://doi.org/10.24963/ijcai.2021/214

• Lin, K.; Wang, L.; Liu, Z. Mesh Graphormer. In Proceedings of the International Conference on Computer Vision (ICCV 2021); IEEE: Montreal, QC, Canada, 2021; pp. 12919–12928. https://doi.org/10.1109/ICCV48922.2021.01270

• Nguyen, D.Q.; Nguyen, T.D.; Phung, D. Universal Graph Transformer Self-Attention Networks. In Proceedings of the Companion Proceedings of the Web Conference 2022; Association for Computing Machinery: New York, NY, USA, 2022; WWW ’22, p. 193–196. https://doi.org/10.1145/3487553.3524258

13. Transformer with global self-attention

• Dexiong Chen, Leslie O'Bray, Karsten M. Borgwardt: Structure-Aware Transformer for Graph Representation Learning. ICML 2022: 3469-3489

• Jinwoo Kim, Tien Dat Nguyen, Seonwoo Min, Sungjun Cho, Moontae Lee, Honglak Lee, Seunghoon Hong: Pure Transformers are Powerful Graph Learners. CoRR abs/2207.02505 (2022)

• Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, Tie-Yan Liu: Do Transformers Really Perform Badly for Graph Representation? NeurIPS 2021: 28877-28888