Please wait a minute...
Chin. Phys. B, 2021, Vol. 30(5): 050503    DOI: 10.1088/1674-1056/abd74f
GENERAL Prev   Next  

Control of chaos in Frenkel-Kontorova model using reinforcement learning

You-Ming Lei(雷佑铭)1,2,† and Yan-Yan Han(韩彦彦)1
1 School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710072, China;
2 Ministry of Industry and Information Technology(MIIT) Key Laboratory of Dynamics and Control of Complex Systems, Northwestern Polytechnical University, Xi'an 710072, China
Abstract  It is shown that we can control spatiotemporal chaos in the Frenkel-Kontorova (FK) model by a model-free control method based on reinforcement learning. The method uses Q-learning to find optimal control strategies based on the reward feedback from the environment that maximizes its performance. The optimal control strategies are recorded in a Q-table and then employed to implement controllers. The advantage of the method is that it does not require an explicit knowledge of the system, target states, and unstable periodic orbits. All that we need is the parameters that we are trying to control and an unknown simulation model that represents the interactive environment. To control the FK model, we employ the perturbation policy on two different kinds of parameters, i.e., the pendulum lengths and the phase angles. We show that both of the two perturbation techniques, i.e., changing the lengths and changing their phase angles, can suppress chaos in the system and make it create the periodic patterns. The form of patterns depends on the initial values of the angular displacements and velocities. In particular, we show that the pinning control strategy, which only changes a small number of lengths or phase angles, can be put into effect.
Keywords:  chaos control      Frenkel-Kontorova model      reinforcement learning  
Received:  01 October 2020      Revised:  08 December 2020      Accepted manuscript online:  30 December 2020
PACS:  05.45.Gg (Control of chaos, applications of chaos)  
  05.45.Xt (Synchronization; coupled oscillators)  
Fund: Project supported by the National Natural Science Foundation of China (Grant Nos. 12072262 and 11672231).
Corresponding Authors:  You-Ming Lei     E-mail:

Cite this article: 

You-Ming Lei(雷佑铭) and Yan-Yan Han(韩彦彦) Control of chaos in Frenkel-Kontorova model using reinforcement learning 2021 Chin. Phys. B 30 050503

[1] Frenkel Y I and Kontorova T 1938 Zh. Eksp. Teor. Fiz. 8 1340
[2] Braun O M and Kivshar Y S 2004 The Frenkel-Kontorova Model: Concepts, Methods, and Applications (Berlin: Springer-Verlag)
[3] Zhu J H and Sun Y M 2020 Chaos, Solitons, and Fractals 135 109745
[4] Gninzanlong C L, Ndjomatchoua F T and Tchawoua C 2019 Phys. Rev. E 99 052210
[5] Pang M B and Huang Y M 2018 Chin. Phys. B 27 118902
[6] Boroujeni F R, Vasegh N and Sedigh A K 2014 Int. J. Bifur. Chaos 24 1450140
[7] Nam S G, Nguyen V H and Song H J 2014 Chin. Phys. Lett. 31 060502
[8] Ott E, Grebogi C and Yorke J A 1990 Phys. Rev. Lett. 64 1196
[9] Pyragas K 1992 Phys. Rev. A 170 421
[10] Pyragas V and Pyragas K 2018 Phys. Lett. A 382 574
[11] Braiman Y, Barhen J and Protopopescu V 2003 Phys. Rev. Lett. 90 094301
[12] Lei Y M, Zheng F and Shao X 2017 Int. J. Bifur. Chaos 27 1750052
[13] Yadav V K, Shukla V K and Das S 2019 Chaos, Solitons, and Fractals 124 36
[14] Zhou J C and Song H J 2013 Chin. Phys. Lett. 30 020501
[15] Gao F, Hu D N, Tong H Q and Wang C M 2018 Acta Phys. Sin. 67 150501 (in Chinese)
[16] Sharma A and Sinha S C 2020 Nonlinear Dyn. 99 559
[17] Wang S F, Li X C, Xia F and Xie Z S 2014 Appl. Math. Comput. 247 487
[18] Mohsen A, Ahmed R, Abdelwaheb R, Sundarapandian V and Ahmad T A 2021 Backstepping Control of Nonlinear Dynamical Systems (Academic: Sundarapandian Vaidyanathan and Ahmad Taher Azar) pp. 291-345
[19] Schöll E and Schuster H G 2008 Handbook of Chaos Control, 2nd edn. (Weinheim, Germany: Wiley-VCH)
[20] Gadaleta S and Dangelmayr G 1999 Chaos 9 775
[21] Gadaleta S and Dangelmayr G 2000 Proceedings of the 2nd International Conference on Control of Oscillations and Chaos, July 5-7, 2000, Saint Petersburg, Russia, pp. 109-112
[22] Wei Q L, Song R Z, Sun Q Y and Xiao W D 2015 Chin. Phys. B 24 090504
[23] Wiering M and Otterlo M V 2012 Reinforcement Learning: State-of-the-Art (Berlin: Springer), pp. 3-42
[24] Braiman Y, Lindner J F and Ditto W L 1995 Nature 378 465
[25] Brandt S F, Dellen B K and Wessel R 2006 Phys. Rev. Lett. 96 034104
[26] Watkins C J and Dayan P 1992 Machine Learning 8 279
[27] Singh S P, Barto A G, Grupen R and Connolly C 1993 Advances in Neural Information Processing Systems 6 (Morgan Kaufmann: Cowan, Tesauro and Alspector), pp. 655-662
[28] Robbins H and Monro S 1951 Ann. Math. Statistics 22 400
[29] Sutton R S and Barto A G 1998 Reinforcement Learning: an Introduction (Cambridge: The MIT Press)
[30] Zhou S and Wang X Y 2018 Chaos 28 123118
[31] Rodriguez A and Laio A 2014 Science 344 1492
[1] Nano-friction phenomenon of Frenkel—Kontorova model under Gaussian colored noise
Yi-Wei Li(李毅伟), Peng-Fei Xu(许鹏飞), and Yong-Ge Yang(杨勇歌). Chin. Phys. B, 2022, 31(5): 050501.
[2] Exploring individuals' effective preventive measures against epidemics through reinforcement learning
Ya-Peng Cui(崔亚鹏), Shun-Jiang Ni (倪顺江), and Shi-Fei Shen(申世飞). Chin. Phys. B, 2021, 30(4): 048901.
[3] Optimal control strategy for COVID-19 concerning both life and economy based on deep reinforcement learning
Wei Deng(邓为), Guoyuan Qi(齐国元), and Xinchen Yu(蔚昕晨). Chin. Phys. B, 2021, 30(12): 120203.
[4] Chaotic analysis of Atangana-Baleanu derivative fractional order Willis aneurysm system
Fei Gao(高飞), Wen-Qin Li(李文琴), Heng-Qing Tong(童恒庆), Xi-Ling Li(李喜玲). Chin. Phys. B, 2019, 28(9): 090501.
[5] Coordinated chaos control of urban expressway based on synchronization of complex networks
Ming-bao Pang(庞明宝), Yu-man Huang(黄玉满). Chin. Phys. B, 2018, 27(11): 118902.
[6] Parrondo's paradox for chaos control and anticontrol of fractional-order systems
Marius-F Danca, Wallace K S Tang. Chin. Phys. B, 2016, 25(1): 010505.
[7] Control of fractional chaotic and hyperchaotic systems based on a fractional order controller
Li Tian-Zeng (李天增), Wang Yu (王瑜), Luo Mao-Kang (罗懋康). Chin. Phys. B, 2014, 23(8): 080501.
[8] Chaos control in the nonlinear Schrödinger equation with Kerr law nonlinearity
Yin Jiu-Li (殷久利), Zhao Liu-Wei (赵刘威), Tian Li-Xin (田立新). Chin. Phys. B, 2014, 23(2): 020204.
[9] Complex dynamical behavior and chaos control for fractional-order Lorenz-like system
Li Rui-Hong (李瑞红), Chen Wei-Sheng (陈为胜). Chin. Phys. B, 2013, 22(4): 040503.
[10] Chaos detection and control in a typical power system
Hossein Gholizadeh, Amir Hassannia, Azita Azarfar. Chin. Phys. B, 2013, 22(1): 010503.
[11] Control of fractional chaotic system based on fractional-order resistor–capacitor filter
Zhang Lu (张路), Deng Ke (邓科), Luo Mao-Kang (罗懋康). Chin. Phys. B, 2012, 21(9): 090505.
[12] Cascade adaptive control of uncertain unified chaotic systems
Wei Wei(魏伟), Li Dong-Hai(李东海), and Wang Jing(王京). Chin. Phys. B, 2011, 20(4): 040510.
[13] Controlling chaos in power system based on finite-time stability theory
Zhao Hui(赵辉), Ma Ya-Jun(马亚军), Liu Si-Jia(刘思佳), Gao Shi-Gen(高士根), and Zhong Dan(钟丹) . Chin. Phys. B, 2011, 20(12): 120501.
[14] No-chattering sliding mode control in a class of fractional-order chaotic systems
Chen Di-Yi(陈帝伊), Liu Yu-Xiao(刘玉晓), Ma Xiao-Yi(马孝义), and Zhang Run-Fan(张润凡) . Chin. Phys. B, 2011, 20(12): 120506.
[15] Adaptive stabilization of an incommensurate fractional order chaotic system via a single state controller
Zhang Ruo-Xun(张若洵) and Yang Shi-Ping (杨世平) . Chin. Phys. B, 2011, 20(11): 110506.
No Suggested Reading articles found!