1 China West Normal University, Nanchong 637000, China; 2 The Internet of Things Perception and Big Data Analysis Key Laboratory of Nanchong City, Nanchong 637000, China; 3 Institute of Artificial Intelligence, China West Normal University, Nanchong 637000, China
Abstract Graph neural networks (GNNs) have demonstrated excellent performance in graph representation learning. However, as the volume of graph data grows, issues related to cost and efficiency become increasingly prominent. Graph distillation methods address this challenge by extracting a smaller, reduced graph, ensuring that GNNs trained on both the original and reduced graphs show similar performance. Existing methods, however, primarily optimize the feature matrix of the reduced graph and rely on correlation information from GNNs, while neglecting the original graph's structure and redundant nodes. This often results in a loss of critical information within the reduced graph. To overcome this limitation, we propose a graph distillation method guided by network symmetry. Specifically, we identify symmetric nodes with equivalent neighborhood structures and merge them into "super nodes", thereby simplifying the network structure, reducing redundant parameter optimization and enhancing training efficiency. At the same time, instead of relying on the original node features, we employ gradient descent to match optimal features that align with the original features, thus improving downstream task performance. Theoretically, our method guarantees that the reduced graph retains the key information present in the original graph. Extensive experiments demonstrate that our approach achieves significant improvements in graph distillation, exhibiting strong generalization capability and outperforming existing graph reduction methods.
Fund: Project supported by the National Natural Science Foundation of China (Grant No. 62176217), the Program from the Sichuan Provincial Science and Technology, China (Grant No. 2018RZ0081), and the Fundamental Research Funds of China West Normal University (Grant No. 17E063).
Corresponding Authors:
Jia-Lin He
E-mail: hejialin32@126.com
Cite this article:
Feng Lin(林峰) and Jia-Lin He(何嘉林) Graph distillation with network symmetry 2025 Chin. Phys. B 34 040204
[1] Mitchell J C 1974 Annual Review of Anthropology 3 279 [2] Lü L, Medo M, Yeung C H, Zhang Y C, Zhang Z K and Zhou T 2012 Physics Reports 519 1 [3] Cui D X, He J L, Xiao Z F and RenWP 2023 Chin. Phys. B 32 098904 [4] Zhou J, Cui G, Hu S, Zhang Z, Yang C, Liu Z, Wang L, Li C and Sun M 2020 AI open 1 57 [5] Lin B and Wang Y X 2024 Chin. Phys. B 33 084401 [6] Kong J G, Li Q X, Li J, Lin Y and Zhu J J 2022 Chin. Phys. Lett. 39 067503 [7] Kipf T N and Welling M 2016 arXiv:1609.02907[cs.LG] [8] Veličković P, Cucurull G, Casanova A, Romero A, Lio P and Bengio Y 2017 arXiv:1710.10903[stat.ML] [9] Hamilton W, Ying Z and Leskovec J 2017 arXiv:1706.02216cs.SI] [10] Gao X, Yu J, Jiang W, Chen T, Zhang W and Yin H 2024 arxiv:2401.11720[cs.LG] [11] Jin W, Zhao L, Zhang S, Liu Y, Tang J and Shah N 2021 arxiv:2110.07580[cs.LG] [12] Liu M, Li S, Chen X and Song L 2022 arxiv:2206.13697[cs.LG] [13] Zheng X, Zhang M, Chen C, Nguyen Q V H, Zhu X and Pan S 2023 arXiv:2306.02664cs.LG] [14] Sánchez-García R J 2020 Communications Physics 3 87 [15] McKay B D and Piperno A 2014 Journal of Symbolic Computation 60 94 [16] Gilmer J, Schoenholz S S, Riley P F, Vinyals O and Dahl G E 2017 arXiv:1704.01212cs.LG] [17] You J, Gomes-Selman J M, Ying R and Leskovec J 2021 Proceedings of the AAAI Conference on Artificial Intelligence 35 10737 [18] Xiao Y, Macarthur B D, Wang H and Xiong M 2008 Phys. Rev. E 78 046102 [19] Slansky R 1981 Physics Reports 79 1 [20] Bradley A P 1997 Pattern Recognition 30 1145 [21] Rozemberczki B, Allen C and Sarkar R 2021 Journal of Complex Networks 9 cnab014 [22] Yun S, Jeong M, Kim R, Kang J and Kim H J 2019 arXiv:1911.06455[cs.LG] [23] Welling M 2009 ICML ’09, June 14-18, Montreal Quebec, Canada, p.1121 [24] Sener O and Savarese S 2017 arXiv:1708.00489[stat.ML] [25] Huang Z, Zhang S, Xi C, Liu T and Zhou M 2021 KDD ’21, August 14-18, Singapore, p. 675 [26] Lim D, Hohne F, Li X, Huang S L, Gupta V, Bhalerao O and Lim S 2021 arXiv:2110.14446cs.LG] [27] Cen Y, Hou Z,Wang Y, et al. 2023 WWW’23, April 30-May 4, Austin TX, USA, p. 747 [28] Wu F, Souza A, Zhang T, Fifty C, Yu T and Weinberger K 2019 arXiv:1902.07153cs.LG] [29] Hassani K and Khasahmadi A H 2020 arXiv:2006.05582cs.LG] [30] Feng W, Zhang J, Dong Y, Han Y, Luan H, Xu Q, Yang Q, Kharlamov E and Tang J 2020 NeurIPS 2020, December 6-12, Online, p. 22092 [31] Abu-El-Haija S, Perozzi B, Kapoor A, Alipourfard N, Lerman K, Harutyunyan H, Steeg G V and Galstyan A 2019 arXiv:1905.00067cs.LG] [32] Tang S, Li B and Yu H 2024 Communications in Mathematics and Statistics 2024 1 [33] He M, Wei Z and Xu H 2021 NeurIPS 2021, December 6-14, Online, p. 14239 [34] Xu B, Shen H, Cao Q, Qiu Y and Cheng X 2019 arXiv:1904.07785[cs.LG] [35] Danel T, Spurek P, Tabor J, Smieja M, Struski L, Slowik A and Maziarka L 2020 ICONIP 2020, November 23-27, Bangkok, Thailand, p. 668 [36] Wang T, Zhu J Y, Torralba A and Efros A A 2018 arXiv:1811.10959[cs.LG] [37] Zhao B, Mopuri K R and Bilen H 2020 arXiv:2006.05929[cs.CV] [38] Loukas A 2019 arXiv:1808.10650cs.DS] [39] Loukas A and Vandergheynst P 2018 arXiv:1802.07510cs.LG]
Altmetric calculates a score based on the online attention an article receives. Each coloured thread in the circle represents a different type of online attention. The number in the centre is the Altmetric score. Social media and mainstream news media are the main sources that calculate the score. Reference managers such as Mendeley are also tracked but do not contribute to the score. Older articles often score higher because they have had more time to get noticed. To account for this, Altmetric has included the context data for other articles of a similar age.
