| INTERDISCIPLINARY PHYSICS AND RELATED AREAS OF SCIENCE AND TECHNOLOGY |
Prev
Next
|
|
|
Pulse coding off-chip learning algorithm for memristive artificial neural network |
| Ming-Jian Guo(郭明健), Shu-Kai Duan(段书凯), and Li-Dan Wang(王丽丹)† |
| College of Artificial Intelligence, Southwest University, Chongqing 400715, China |
|
|
|
|
Abstract Memristive neural network has attracted tremendous attention since the memristor array can perform parallel multiply-accumulate calculation (MAC) operations and memory-computation operations as compared with digital CMOS hardware systems. However, owing to the variability of the memristor, the implementation of high-precision neural network in memristive computation units is still difficult. Existing learning algorithms for memristive artificial neural network (ANN) is unable to achieve the performance comparable to high-precision by using CMOS-based system. Here, we propose an algorithm based on off-chip learning for memristive ANN in low precision. Training the ANN in the high-precision in digital CPUs and then quantifying the weight of the network to low precision, the quantified weights are mapped to the memristor arrays based on VTEAM model through using the pulse coding weight-mapping rule. In this work, we execute the inference of trained 5-layers convolution neural network on the memristor arrays and achieve an accuracy close to the inference in the case of high precision (64-bit). Compared with other algorithms-based off-chip learning, the algorithm proposed in the present study can easily implement the mapping process and less influence of the device variability. Our result provides an effective approach to implementing the ANN on the memristive hardware platform.
|
Received: 08 December 2021
Revised: 18 January 2022
Accepted manuscript online: 27 January 2022
|
|
PACS:
|
87.18.Sn
|
(Neural networks and synaptic communication)
|
| |
07.05.Rm
|
(Data presentation and visualization: algorithms and implementation)
|
| |
07.05.Tp
|
(Computer modeling and simulation)
|
| |
07.05.Mh
|
(Neural networks, fuzzy logic, artificial intelligence)
|
|
| Fund: Project supported by the National Natural Science Foundation of China (Grant Nos. 62076208, 62076207, and U20A20227) and the National Key Research and Development Program of China (Grant No. 2018YFB1306600). |
Corresponding Authors:
Li-Dan Wang
E-mail: ldwang@swu.edu.cn
|
Cite this article:
Ming-Jian Guo(郭明健), Shu-Kai Duan(段书凯), and Li-Dan Wang(王丽丹) Pulse coding off-chip learning algorithm for memristive artificial neural network 2022 Chin. Phys. B 31 078702
|
[1] Uijlings J R R, van de Sande K E A, Gevers T and Smeulders A W M 2013 International Journal of Computer Vision 104 154
[2] Richardson F, Reynolds D and Dehak N 2015 IEEE Signal Processing Letters 22 1671
[3] LeCun Y, Bengio Y and Hinton G 2015 Nature 521 436
[4] Li C, Hu M, Li Y, Jiang H, Ge N, Montgomery E, Zhang J, Song W, Davila N, Graves C E, Li Z, Strachan J P, Lin P, Wang Z, Barnell M, Wu Q, Williams R S, Yang J J and Xia Q 2018 Nat. Electron. 1 52
[5] Coates A, Huval B, Wang T, Wu D J, Ng A Y and Catanzaro B 2013 30th International Conference on Machine Learning, (ICML 2013), June 16-21, 2013, Atlanta, USA, p. 2374
[6] Jouppi N P, Young C and Patil N 2017 44th Annual International Symposium on Computer Architecture (ISCA 2017), June 24-28, 2017, Toronto, Canada, p. 1
[7] Chen Y H, Krishna T, Emer J S and Sze V 2017 IEEE Journal of Solid-State Circuits 52 127
[8] Zidan M A, Strachan J P and Lu W D 2018 Nat. Electron. 1 22
[9] Boschker J E and Calarco R 2017 Adv. Phys. X 2 675
[10] Boybat I, Le Gallo M, Nandakumar S R, Moraitis T, Parnell T, Tuma T, Rajendran B, Leblebici Y, Sebastian A and Eleftheriou E 2018 Nat. Commun. 9 2514
[11] Indiveri G, Linares-Barranco B, Legenstein R, Deligeorgis G and Prodromakis T 2013 Nanotechnology 24 384010
[12] Chua L O 1971 IEEE Transactions on Circuit Theory CT18 507
[13] Shang L, Duan S, Wang L and Huang T 2018 IEEE Transactions on Very Large Scale Integration (Vlsi) Systems 26 2830
[14] Li Y, Li J, Li J, Duan S, Wang L and Guo M 2021 Neurocomputing 454 382
[15] Strukov D B, Snider G S, Stewart D R and Williams R S 2008 Nature 453 80
[16] Yu F, Zhang Z, Shen H, Huang Y, Cai S and Du S 2021 Chin. Phys. B 1088 152
[17] Xu Q, Ju Z, Ding S, Feng C, Chen M and Bao B 2021 Cognitive Neurodynamics 1007 64
[18] Le Gallo M, Sebastian A, Mathis R, Manica M, Giefers H, Tuma T, Bekas C, Curioni A and Eleftheriou E 2018 Nat. Electron. 1 246
[19] Prezioso M, Merrikh-Bayat F, Hoskins B D, Adam G C, Likharev K K and Strukov D B 2015 Nature 521 61
[20] Chen J, Wang L and Duan S 2021 Neurocomputing 461 129
[21] Li C, Belkin D, Li Y, Yan P, Hu M, Ge N, Jiang H, Montgomery E, Lin P and Wang Z 2018 Nat. Commun. 9 1
[22] Yao P, Wu H, Gao B, Eryilmaz S B, Huang X, Zhang W, Zhang Q, Deng N, Shi L and Wong H S P 2017 Nat. Commun. 8 1
[23] Gao B, Bi Y, Chen H Y, Liu R, Huang P, Chen B, Liu L, Liu X, Yu S, Wong H S P and Kang J 2014 Acs Nano 8 6998
[24] Jo S H, Chang T, Ebong I, Bhadviya B B, Mazumder P and Lu W 2010 Nano Lett. 10 1297
[25] Hu M, Li H, Wu Q, Rose G S and Chen Y 2012 The International Joint Conference on Neural Networks (IJCNN), 2012, p. 1
[26] Li B, Shan Y, Hu M, Wang Y, Chen Y and Yang H 2013 International Symposium on Low Power Electronics and Design (ISLPED), 2013, p. 242
[27] Zhang Q, Wu H, Yao P, Zhang W, Gao B, Deng N and Qian H 2018 Neural Networks 108 217
[28] Lim S, Bae J H, Eum J H, Lee S, Kim C H, Kwon D, Park B G and Lee J H 2019 Neural Computing & Applications 31 8101
[29] Merrikh-Bayat F, Guo X, Klachko M, Prezioso M, Likharev K K and Strukov D B 2018 IEEE Transactions on Neural Networks and Learning Systems 29 4782
[30] Hikawa H, Tamaki M and Ito H 2018 IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences E101A 499
[31] Wong H S P, Lee H Y, Yu S, Chen Y S, Wu Y, Chen P S, Lee B, Chen F T and Tsai M J 2012 Proc. IEEE 100 1951
[32] Zamanidoost E, Klachko M, Strukov D and Kataeva I 2015 Proceedings of the IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH'15) 2015, p. 139
[33] Chi P, Li S, Xu C, Zhang T, Zhao J, Liu Y, Wang Y and Xie Y 2016 "PRIME:A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory", in:2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), p. 27
[34] Wang Z, Wu H, Burr G W, Hwang C S, Wang K L, Xia Q and Yang J J 2020 Nat. Rev. Mater. 5 173
[35] Kvatinsky S, Ramadan M, Friedman E G and Kolodny A 2015 IEEE Transactions on Circuits and Systems II-Express Briefs 62 786 |
| No Suggested Reading articles found! |
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
Altmetric
|
|
blogs
Facebook pages
Wikipedia page
Google+ users
|
Online attention
Altmetric calculates a score based on the online attention an article receives. Each coloured thread in the circle represents a different type of online attention. The number in the centre is the Altmetric score. Social media and mainstream news media are the main sources that calculate the score. Reference managers such as Mendeley are also tracked but do not contribute to the score. Older articles often score higher because they have had more time to get noticed. To account for this, Altmetric has included the context data for other articles of a similar age.
View more on Altmetrics
|
|
|
