Please wait a minute...
Chin. Phys. B, 2021, Vol. 30(4): 048702    DOI: 10.1088/1674-1056/abd68e
Special Issue: SPECIAL TOPIC — Machine learning in statistical physics
SPECIAL TOPIC—Machine learning in statistical physics Prev   Next  

Relationship between manifold smoothness and adversarial vulnerability in deep learning with local errors

Zijian Jiang(蒋子健), Jianwen Zhou(周健文), and Haiping Huang(黄海平)
1 PMI Laboratory, School of Physics, Sun Yat-sen University, Guangzhou 510275, China
Abstract  Artificial neural networks can achieve impressive performances, and even outperform humans in some specific tasks. Nevertheless, unlike biological brains, the artificial neural networks suffer from tiny perturbations in sensory input, under various kinds of adversarial attacks. It is therefore necessary to study the origin of the adversarial vulnerability. Here, we establish a fundamental relationship between geometry of hidden representations (manifold perspective) and the generalization capability of the deep networks. For this purpose, we choose a deep neural network trained by local errors, and then analyze emergent properties of the trained networks through the manifold dimensionality, manifold smoothness, and the generalization capability. To explore effects of adversarial examples, we consider independent Gaussian noise attacks and fast-gradient-sign-method (FGSM) attacks. Our study reveals that a high generalization accuracy requires a relatively fast power-law decay of the eigen-spectrum of hidden representations. Under Gaussian attacks, the relationship between generalization accuracy and power-law exponent is monotonic, while a non-monotonic behavior is observed for FGSM attacks. Our empirical study provides a route towards a final mechanistic interpretation of adversarial vulnerability under adversarial attacks.
Keywords:  neural networks      learning  
Received:  09 July 2020      Revised:  09 December 2020      Accepted manuscript online:  24 December 2020
PACS:  87.18.Sn (Neural networks and synaptic communication) (Learning and memory)  
Fund: Project supported by the National Key R&D Program of China (Grant No. 2019YFA0706302), the start-up budget 74130-18831109 of the 100-talent-program of Sun Yat-sen University, and the National Natural Science Foundation of China (Grant No. 11805284).
Corresponding Authors:  Corresponding author. E-mail:   

Cite this article: 

Zijian Jiang(蒋子健), Jianwen Zhou(周健文), and Haiping Huang(黄海平) Relationship between manifold smoothness and adversarial vulnerability in deep learning with local errors 2021 Chin. Phys. B 30 048702

1 Goodfellow I, Bengio Y and Courville A2016 Deep Learning (Cambridge, MA: MIT Press)
2 Carlini N and Wagner D 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57
3 Su J W, Vargas D V and Sakurai K 2019 IEEE Transactions on Evolutionary Computation 23 828
4 Zhou Z L and Firestone C 2019 Nat. Commun. 10 1334
5 Stringer C, Pachitariu M, Steinmetz N, Carandini M and Harris K D 2019 Nature 571 361
6 Mostafa H, Ramesh V and Cauwenberghs G 2018 Frontiers in Neuroscience 12 608
7 Lillicrap T P, Santoro A, Marris L, Akerman C J and Hinton G 2020 Nature Reviews Neuroscience 21 335
8 Yamins D L K and DiCarlo J 2016 Nat. Neurosci. 19 356
9 Lecun Y, Bottou L, Bengio Y and Haffner P 1998 Proc. IEEE 86 2278
10 Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I and Fergus R 2014 International Conference on Learning Representations (ICLR)
11 Goodfellow I, Shlens J and Szegedy C 2015 International Conference on Learning Representations (ICLR)
12 Huang H P 2018 Phys. Rev. E 98 062313
13 Zhou J W and Huang H P 2021 Phys. Rev. E 103 012315
[1] Serverless distributed learning for smart grid analytics
Gang Huang(黄刚), Chao Wu(吴超), Yifan Hu(胡一帆), and Chuangxin Guo(郭创新). Chin. Phys. B, 2021, 30(8): 088802.
[2] Artificial synaptic behavior of the SBT-memristor
Gang Dou(窦刚), Ming-Long Dou(窦明龙), Ren-Yuan Liu(刘任远), and Mei Guo(郭梅). Chin. Phys. B, 2021, 30(7): 078401.
[3] Soliton, breather, and rogue wave solutions for solving the nonlinear Schrödinger equation using a deep learning method with physical constraints
Jun-Cai Pu(蒲俊才), Jun Li(李军), and Yong Chen(陈勇). Chin. Phys. B, 2021, 30(6): 060202.
[4] High speed ghost imaging based on a heuristic algorithm and deep learning
Yi-Yi Huang(黄祎祎), Chen Ou-Yang(欧阳琛), Ke Fang(方可), Yu-Feng Dong(董玉峰), Jie Zhang(张杰), Li-Ming Chen(陈黎明), and Ling-An Wu(吴令安). Chin. Phys. B, 2021, 30(6): 064202.
[5] $\mathcal{H}_{\infty }$ state estimation for Markov jump neural networks with transition probabilities subject to the persistent dwell-time switching rule
Hao Shen(沈浩), Jia-Cheng Wu(吴佳成), Jian-Wei Xia(夏建伟), and Zhen Wang(王震). Chin. Phys. B, 2021, 30(6): 060203.
[6] Efficient sampling for decision making in materials discovery
Yuan Tian(田原), Turab Lookman, and Dezhen Xue(薛德祯). Chin. Phys. B, 2021, 30(5): 050705.
[7] Handwritten digit recognition based on ghost imaging with deep learning
Xing He(何行), Sheng-Mei Zhao(赵生妹), and Le Wang(王乐). Chin. Phys. B, 2021, 30(5): 054201.
[8] Quantitative structure-plasticity relationship in metallic glass: A machine learning study
Yicheng Wu(吴义成), Bin Xu(徐斌), Yitao Sun(孙奕韬), and Pengfei Guan(管鹏飞). Chin. Phys. B, 2021, 30(5): 057103.
[9] Control of chaos in Frenkel-Kontorova model using reinforcement learning
You-Ming Lei(雷佑铭) and Yan-Yan Han(韩彦彦). Chin. Phys. B, 2021, 30(5): 050503.
[10] Accurate Deep Potential model for the Al-Cu-Mg alloy in the full concentration space
Wanrun Jiang(姜万润), Yuzhi Zhang(张与之), Linfeng Zhang(张林峰), and Han Wang(王涵). Chin. Phys. B, 2021, 30(5): 050706.
[11] Exploring individuals' effective preventive measures against epidemics through reinforcement learning
Ya-Peng Cui(崔亚鹏), Shun-Jiang Ni (倪顺江), and Shi-Fei Shen(申世飞). Chin. Phys. B, 2021, 30(4): 048901.
[12] Quantum annealing for semi-supervised learning
Yu-Lin Zheng(郑玉鳞), Wen Zhang(张文), Cheng Zhou(周诚), and Wei Geng(耿巍). Chin. Phys. B, 2021, 30(4): 040306.
[13] Restricted Boltzmann machine: Recent advances and mean-field theory*

AD was supported by the Comunidad de Madrid and the Complutense University of Madrid (Spain) through the Atracción de Talento program (Ref. 2019-T1/TIC-13298).

Aurélien Decelle, Cyril Furtlehner. Chin. Phys. B, 2021, 30(4): 00.
[14] Constructing reduced model for complex physical systems via interpolation and neural networks
Xuefang Lai(赖学方), Xiaolong Wang(王晓龙, and Yufeng Nie(聂玉峰). Chin. Phys. B, 2021, 30(3): 030204.
[15] Effective suppression of beta oscillation in Parkinsonian state via a noisy direct delayed feedback control scheme
Hai-Tao Yu(于海涛), Zi-Han Meng(孟紫寒), Chen Liu(刘晨), Jiang Wang(王江), and Jing Liu(刘静). Chin. Phys. B, 2021, 30(3): 038703.
[2] Chen Xiang-Wei. Closed orbits and limit cycles of second-order autonomous Birkhoff systems[J]. Chin. Phys., 2003, 12(6): 586 -589 .
[3] R. Z. Levitin, Guo Guang-Hua, Zhang Hai-Bei. Magnetic properties and magnetic phase diagrams of intermetallic compound GdMn2Ge2[J]. Chin. Phys., 2003, 12(6): 655 -660 .
[4] Liu Bing-Chen, Ni Guo-Quan, Xu Zhi-Zhan, Li Shao-Hui. Investigation of the time characteristics of a pulsed flow of large rare gas clusters[J]. Chin. Phys., 2003, 12(8): 856 -860 .
[5] Qiang Wen-Chao. Bound states of the Klein-Gordon equation for ring-shaped Kratzer-type potential[J]. Chin. Phys., 2004, 13(5): 575 -578 .
[6] Yang Chun-Ling, Wang Yu-Ye, Zhao Dong-Yang, Zhao Guo-Liang. The measuring of spectral emissivity of object using chaotic optimal algorithm[J]. Chin. Phys., 2005, 14(10): 2041 -2045 .
[7] Zhou Qian, Wan Bao-Nian, Wu Zhen-Wei, Huang Juan. The carbon impurity particle transport in ohmic discharges on the HT-7 tokamak[J]. Chin. Phys., 2005, 14(12): 2539 -2545 .
[8] Zhang Hong-Bin, Chen Li-Qun, Liu Rong-Wan. The discrete variational principle in Hamiltonian formalism and first integrals[J]. Chin. Phys., 2005, 14(6): 1063 -1068 .
[9] Song Yun-Zhong, Zhao Guang-Zhou, Qi Dong-Lian. Passive control of chaotic system with multiple strange attractors[J]. Chin. Phys., 2006, 15(10): 2266 -2270 .
[10] Muhammad Abbas Bari, Zhong Jia-Yong, Chen Min, Zhao Jing, Zhang Jie. Calculation of plasma characteristics of the sun[J]. Chin. Phys., 2006, 15(11): 2578 -2582 .