Pulse coding off-chip learning algorithm for memristive artificial neural network

doi:10.1088/1674-1056/ac4f4e

中国物理B ›› 2022, Vol. 31 ›› Issue (7): 78702-078702.doi: 10.1088/1674-1056/ac4f4e

Pulse coding off-chip learning algorithm for memristive artificial neural network

Ming-Jian Guo(郭明健), Shu-Kai Duan(段书凯), and Li-Dan Wang(王丽丹)^†

College of Artificial Intelligence, Southwest University, Chongqing 400715, China

收稿日期:2021-12-08 修回日期:2022-01-18 接受日期:2022-01-27 出版日期:2022-06-09 发布日期:2022-07-19
通讯作者: Li-Dan Wang E-mail:ldwang@swu.edu.cn
基金资助:
Project supported by the National Natural Science Foundation of China (Grant Nos. 62076208, 62076207, and U20A20227) and the National Key Research and Development Program of China (Grant No. 2018YFB1306600).

Pulse coding off-chip learning algorithm for memristive artificial neural network

Ming-Jian Guo(郭明健), Shu-Kai Duan(段书凯), and Li-Dan Wang(王丽丹)^†

College of Artificial Intelligence, Southwest University, Chongqing 400715, China

Received:2021-12-08 Revised:2022-01-18 Accepted:2022-01-27 Online:2022-06-09 Published:2022-07-19
Contact: Li-Dan Wang E-mail:ldwang@swu.edu.cn
Supported by:
Project supported by the National Natural Science Foundation of China (Grant Nos. 62076208, 62076207, and U20A20227) and the National Key Research and Development Program of China (Grant No. 2018YFB1306600).

摘要/Abstract

摘要： Memristive neural network has attracted tremendous attention since the memristor array can perform parallel multiply-accumulate calculation (MAC) operations and memory-computation operations as compared with digital CMOS hardware systems. However, owing to the variability of the memristor, the implementation of high-precision neural network in memristive computation units is still difficult. Existing learning algorithms for memristive artificial neural network (ANN) is unable to achieve the performance comparable to high-precision by using CMOS-based system. Here, we propose an algorithm based on off-chip learning for memristive ANN in low precision. Training the ANN in the high-precision in digital CPUs and then quantifying the weight of the network to low precision, the quantified weights are mapped to the memristor arrays based on VTEAM model through using the pulse coding weight-mapping rule. In this work, we execute the inference of trained 5-layers convolution neural network on the memristor arrays and achieve an accuracy close to the inference in the case of high precision (64-bit). Compared with other algorithms-based off-chip learning, the algorithm proposed in the present study can easily implement the mapping process and less influence of the device variability. Our result provides an effective approach to implementing the ANN on the memristive hardware platform.

关键词: off-chip learning, mapping, memristor array, artificial neural network

Abstract: Memristive neural network has attracted tremendous attention since the memristor array can perform parallel multiply-accumulate calculation (MAC) operations and memory-computation operations as compared with digital CMOS hardware systems. However, owing to the variability of the memristor, the implementation of high-precision neural network in memristive computation units is still difficult. Existing learning algorithms for memristive artificial neural network (ANN) is unable to achieve the performance comparable to high-precision by using CMOS-based system. Here, we propose an algorithm based on off-chip learning for memristive ANN in low precision. Training the ANN in the high-precision in digital CPUs and then quantifying the weight of the network to low precision, the quantified weights are mapped to the memristor arrays based on VTEAM model through using the pulse coding weight-mapping rule. In this work, we execute the inference of trained 5-layers convolution neural network on the memristor arrays and achieve an accuracy close to the inference in the case of high precision (64-bit). Compared with other algorithms-based off-chip learning, the algorithm proposed in the present study can easily implement the mapping process and less influence of the device variability. Our result provides an effective approach to implementing the ANN on the memristive hardware platform.

Key words: off-chip learning, mapping, memristor array, artificial neural network

中图分类号: (Neural networks and synaptic communication)

87.18.Sn

07.05.Rm (Data presentation and visualization: algorithms and implementation) 07.05.Tp (Computer modeling and simulation) 07.05.Mh (Neural networks, fuzzy logic, artificial intelligence)

Ming-Jian Guo(郭明健), Shu-Kai Duan(段书凯), and Li-Dan Wang(王丽丹). Pulse coding off-chip learning algorithm for memristive artificial neural network[J]. 中国物理B, 2022, 31(7): 78702-078702.

Ming-Jian Guo(郭明健), Shu-Kai Duan(段书凯), and Li-Dan Wang(王丽丹). Pulse coding off-chip learning algorithm for memristive artificial neural network[J]. Chin. Phys. B, 2022, 31(7): 78702-078702.

参考文献

[1] Uijlings J R R, van de Sande K E A, Gevers T and Smeulders A W M 2013 International Journal of Computer Vision 104 154
[2] Richardson F, Reynolds D and Dehak N 2015 IEEE Signal Processing Letters 22 1671
[3] LeCun Y, Bengio Y and Hinton G 2015 Nature 521 436
[4] Li C, Hu M, Li Y, Jiang H, Ge N, Montgomery E, Zhang J, Song W, Davila N, Graves C E, Li Z, Strachan J P, Lin P, Wang Z, Barnell M, Wu Q, Williams R S, Yang J J and Xia Q 2018 Nat. Electron. 1 52
[5] Coates A, Huval B, Wang T, Wu D J, Ng A Y and Catanzaro B 2013 30th International Conference on Machine Learning, (ICML 2013), June 16-21, 2013, Atlanta, USA, p. 2374
[6] Jouppi N P, Young C and Patil N 2017 44th Annual International Symposium on Computer Architecture (ISCA 2017), June 24-28, 2017, Toronto, Canada, p. 1
[7] Chen Y H, Krishna T, Emer J S and Sze V 2017 IEEE Journal of Solid-State Circuits 52 127
[8] Zidan M A, Strachan J P and Lu W D 2018 Nat. Electron. 1 22
[9] Boschker J E and Calarco R 2017 Adv. Phys. X 2 675
[10] Boybat I, Le Gallo M, Nandakumar S R, Moraitis T, Parnell T, Tuma T, Rajendran B, Leblebici Y, Sebastian A and Eleftheriou E 2018 Nat. Commun. 9 2514
[11] Indiveri G, Linares-Barranco B, Legenstein R, Deligeorgis G and Prodromakis T 2013 Nanotechnology 24 384010
[12] Chua L O 1971 IEEE Transactions on Circuit Theory CT18 507
[13] Shang L, Duan S, Wang L and Huang T 2018 IEEE Transactions on Very Large Scale Integration (Vlsi) Systems 26 2830
[14] Li Y, Li J, Li J, Duan S, Wang L and Guo M 2021 Neurocomputing 454 382
[15] Strukov D B, Snider G S, Stewart D R and Williams R S 2008 Nature 453 80
[16] Yu F, Zhang Z, Shen H, Huang Y, Cai S and Du S 2021 Chin. Phys. B 1088 152
[17] Xu Q, Ju Z, Ding S, Feng C, Chen M and Bao B 2021 Cognitive Neurodynamics 1007 64
[18] Le Gallo M, Sebastian A, Mathis R, Manica M, Giefers H, Tuma T, Bekas C, Curioni A and Eleftheriou E 2018 Nat. Electron. 1 246
[19] Prezioso M, Merrikh-Bayat F, Hoskins B D, Adam G C, Likharev K K and Strukov D B 2015 Nature 521 61
[20] Chen J, Wang L and Duan S 2021 Neurocomputing 461 129
[21] Li C, Belkin D, Li Y, Yan P, Hu M, Ge N, Jiang H, Montgomery E, Lin P and Wang Z 2018 Nat. Commun. 9 1
[22] Yao P, Wu H, Gao B, Eryilmaz S B, Huang X, Zhang W, Zhang Q, Deng N, Shi L and Wong H S P 2017 Nat. Commun. 8 1
[23] Gao B, Bi Y, Chen H Y, Liu R, Huang P, Chen B, Liu L, Liu X, Yu S, Wong H S P and Kang J 2014 Acs Nano 8 6998
[24] Jo S H, Chang T, Ebong I, Bhadviya B B, Mazumder P and Lu W 2010 Nano Lett. 10 1297
[25] Hu M, Li H, Wu Q, Rose G S and Chen Y 2012 The International Joint Conference on Neural Networks (IJCNN), 2012, p. 1
[26] Li B, Shan Y, Hu M, Wang Y, Chen Y and Yang H 2013 International Symposium on Low Power Electronics and Design (ISLPED), 2013, p. 242
[27] Zhang Q, Wu H, Yao P, Zhang W, Gao B, Deng N and Qian H 2018 Neural Networks 108 217
[28] Lim S, Bae J H, Eum J H, Lee S, Kim C H, Kwon D, Park B G and Lee J H 2019 Neural Computing & Applications 31 8101
[29] Merrikh-Bayat F, Guo X, Klachko M, Prezioso M, Likharev K K and Strukov D B 2018 IEEE Transactions on Neural Networks and Learning Systems 29 4782
[30] Hikawa H, Tamaki M and Ito H 2018 IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences E101A 499
[31] Wong H S P, Lee H Y, Yu S, Chen Y S, Wu Y, Chen P S, Lee B, Chen F T and Tsai M J 2012 Proc. IEEE 100 1951
[32] Zamanidoost E, Klachko M, Strukov D and Kataeva I 2015 Proceedings of the IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH'15) 2015, p. 139
[33] Chi P, Li S, Xu C, Zhang T, Zhao J, Liu Y, Wang Y and Xie Y 2016 "PRIME:A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory", in:2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), p. 27
[34] Wang Z, Wu H, Burr G W, Hwang C S, Wang K L, Xia Q and Yang J J 2020 Nat. Rev. Mater. 5 173
[35] Kvatinsky S, Ramadan M, Friedman E G and Kolodny A 2015 IEEE Transactions on Circuits and Systems II-Express Briefs 62 786

Pulse coding off-chip learning algorithm for memristive artificial neural network

Pulse coding off-chip learning algorithm for memristive artificial neural network

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	Chunlei Fan(范春雷) and Qun Ding(丁群). A novel algorithm to analyze the dynamics of digital chaotic maps in finite-precision domain[J]. 中国物理B, 2023, 32(1): 10501-010501.
[2]	Xiaopeng Yan(闫晓鹏), Xingyuan Wang(王兴元), and Yongjin Xian(咸永锦). Synchronously scrambled diffuse image encryption method based on a new cosine chaotic map[J]. 中国物理B, 2022, 31(8): 80504-080504.
[3]	Hao Luo(罗浩), Yi-Jun Wang(王一军), Wei Ye(叶炜), Hai Zhong(钟海), Yi-Yu Mao(毛宜钰), and Ying Guo(郭迎). Parameter estimation of continuous variable quantum key distribution system via artificial neural networks[J]. 中国物理B, 2022, 31(2): 20306-020306.
[4]	Jian-Long Wang(王剑龙), Xiao-Lei Leng(冷小磊), and Xian-Bin Liu(刘先斌). Stationary response of colored noise excited vibro-impact system[J]. 中国物理B, 2021, 30(6): 60501-060501.
[5]	韩磊, 齐淑霞, 刘圣, 李鹏, 程华超, 赵建林. Hybrid vector beams with non-uniform orbital angular momentum density induced by designed azimuthal polarization gradient[J]. 中国物理B, 2020, 29(9): 94203-094203.
[6]	付新. Uncovering the internal structure of five-fold twinned nanowires through 3D electron diffraction mapping[J]. 中国物理B, 2020, 29(6): 68101-068101.
[7]	王勇, 崔金超, 陈菊, 郭永新. Quasi-canonicalization for linear homogeneous nonholonomic systems[J]. 中国物理B, 2020, 29(6): 64501-064501.
[8]	汪志刚, 廖涛, 王亚南. Modeling electric field of power metal-oxide-semiconductor field-effect transistor with dielectric trench based on Schwarz-Christoffel transformation[J]. 中国物理B, 2019, 28(5): 58503-058503.
[9]	陈琳, 郑志伟, 包立君, 方金生, 杨天和, 蔡淑惠, 蔡聪波. Weighted total variation using split Bregman fast quantitative susceptibility mapping reconstruction method[J]. 中国物理B, 2018, 27(8): 88701-088701.
[10]	蔡爱龙, 李磊, 王林元, 闫镔, 郑治中, 张瀚铭, 胡国恩. Image reconstruction for cone-beam computed tomography using total p-variation plus Kullback–Leibler data divergence[J]. 中国物理B, 2017, 26(7): 78701-078701.
[11]	郑芳林, 刘程晟, 任佳琪, 石艳玲, 孙亚宾, 李小进. Analytical capacitance model for 14 nm FinFET considering dual-k spacer[J]. 中国物理B, 2017, 26(7): 77303-077303.
[12]	杨亿斌, 柳铭岗, 陈伟杰, 韩小标, 陈杰, 林秀其, 林佳利, 罗慧, 廖强, 臧文杰, 陈崟松, 邱运灵, 吴志盛, 刘扬, 张佰君. In-situ wafer bowing measurements of GaN grown on Si (111) substrate by reflectivity mapping in metal organic chemical vapor deposition system[J]. 中国物理B, 2015, 24(9): 96103-096103.
[13]	刘爽, 王兆龙, 赵双双, 李海滨, 李建雄. Grazing bifurcation analysis of a relative rotation system with backlash non-smooth characteristic[J]. 中国物理B, 2015, 24(7): 74501-074501.
[14]	包伯成, 胡丰伟, 刘中, 许建平. Mapping equivalent approach to analysis and realization of memristor-based dynamical circuit[J]. , 2014, 23(7): 70503-070503.
[15]	马松华, 徐根海, 朱海平. Complex solutions and novel complex wave localized excitations for the (2+1)-dimensional Boiti-Leon-Pempinelli system[J]. 中国物理B, 2014, 23(5): 50511-050511.