|
|
|
Control of chaos in Frenkel-Kontorova model using reinforcement learning |
| You-Ming Lei(雷佑铭)1,2,† and Yan-Yan Han(韩彦彦)1 |
1 School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710072, China;
2 Ministry of Industry and Information Technology(MIIT) Key Laboratory of Dynamics and Control of Complex Systems, Northwestern Polytechnical University, Xi'an 710072, China |
|
|
|
|
Abstract It is shown that we can control spatiotemporal chaos in the Frenkel-Kontorova (FK) model by a model-free control method based on reinforcement learning. The method uses Q-learning to find optimal control strategies based on the reward feedback from the environment that maximizes its performance. The optimal control strategies are recorded in a Q-table and then employed to implement controllers. The advantage of the method is that it does not require an explicit knowledge of the system, target states, and unstable periodic orbits. All that we need is the parameters that we are trying to control and an unknown simulation model that represents the interactive environment. To control the FK model, we employ the perturbation policy on two different kinds of parameters, i.e., the pendulum lengths and the phase angles. We show that both of the two perturbation techniques, i.e., changing the lengths and changing their phase angles, can suppress chaos in the system and make it create the periodic patterns. The form of patterns depends on the initial values of the angular displacements and velocities. In particular, we show that the pinning control strategy, which only changes a small number of lengths or phase angles, can be put into effect.
|
Received: 01 October 2020
Revised: 08 December 2020
Accepted manuscript online: 30 December 2020
|
|
PACS:
|
05.45.Gg
|
(Control of chaos, applications of chaos)
|
| |
05.45.Xt
|
(Synchronization; coupled oscillators)
|
|
| Fund: Project supported by the National Natural Science Foundation of China (Grant Nos. 12072262 and 11672231). |
Corresponding Authors:
You-Ming Lei
E-mail: leiyouming@nwpu.edu.cn
|
Cite this article:
You-Ming Lei(雷佑铭) and Yan-Yan Han(韩彦彦) Control of chaos in Frenkel-Kontorova model using reinforcement learning 2021 Chin. Phys. B 30 050503
|
[1] Frenkel Y I and Kontorova T 1938 Zh. Eksp. Teor. Fiz. 8 1340
[2] Braun O M and Kivshar Y S 2004 The Frenkel-Kontorova Model: Concepts, Methods, and Applications (Berlin: Springer-Verlag)
[3] Zhu J H and Sun Y M 2020 Chaos, Solitons, and Fractals 135 109745
[4] Gninzanlong C L, Ndjomatchoua F T and Tchawoua C 2019 Phys. Rev. E 99 052210
[5] Pang M B and Huang Y M 2018 Chin. Phys. B 27 118902
[6] Boroujeni F R, Vasegh N and Sedigh A K 2014 Int. J. Bifur. Chaos 24 1450140
[7] Nam S G, Nguyen V H and Song H J 2014 Chin. Phys. Lett. 31 060502
[8] Ott E, Grebogi C and Yorke J A 1990 Phys. Rev. Lett. 64 1196
[9] Pyragas K 1992 Phys. Rev. A 170 421
[10] Pyragas V and Pyragas K 2018 Phys. Lett. A 382 574
[11] Braiman Y, Barhen J and Protopopescu V 2003 Phys. Rev. Lett. 90 094301
[12] Lei Y M, Zheng F and Shao X 2017 Int. J. Bifur. Chaos 27 1750052
[13] Yadav V K, Shukla V K and Das S 2019 Chaos, Solitons, and Fractals 124 36
[14] Zhou J C and Song H J 2013 Chin. Phys. Lett. 30 020501
[15] Gao F, Hu D N, Tong H Q and Wang C M 2018 Acta Phys. Sin. 67 150501 (in Chinese)
[16] Sharma A and Sinha S C 2020 Nonlinear Dyn. 99 559
[17] Wang S F, Li X C, Xia F and Xie Z S 2014 Appl. Math. Comput. 247 487
[18] Mohsen A, Ahmed R, Abdelwaheb R, Sundarapandian V and Ahmad T A 2021 Backstepping Control of Nonlinear Dynamical Systems (Academic: Sundarapandian Vaidyanathan and Ahmad Taher Azar) pp. 291-345
[19] Schöll E and Schuster H G 2008 Handbook of Chaos Control, 2nd edn. (Weinheim, Germany: Wiley-VCH)
[20] Gadaleta S and Dangelmayr G 1999 Chaos 9 775
[21] Gadaleta S and Dangelmayr G 2000 Proceedings of the 2nd International Conference on Control of Oscillations and Chaos, July 5-7, 2000, Saint Petersburg, Russia, pp. 109-112
[22] Wei Q L, Song R Z, Sun Q Y and Xiao W D 2015 Chin. Phys. B 24 090504
[23] Wiering M and Otterlo M V 2012 Reinforcement Learning: State-of-the-Art (Berlin: Springer), pp. 3-42
[24] Braiman Y, Lindner J F and Ditto W L 1995 Nature 378 465
[25] Brandt S F, Dellen B K and Wessel R 2006 Phys. Rev. Lett. 96 034104
[26] Watkins C J and Dayan P 1992 Machine Learning 8 279
[27] Singh S P, Barto A G, Grupen R and Connolly C 1993 Advances in Neural Information Processing Systems 6 (Morgan Kaufmann: Cowan, Tesauro and Alspector), pp. 655-662
[28] Robbins H and Monro S 1951 Ann. Math. Statistics 22 400
[29] Sutton R S and Barto A G 1998 Reinforcement Learning: an Introduction (Cambridge: The MIT Press)
[30] Zhou S and Wang X Y 2018 Chaos 28 123118
[31] Rodriguez A and Laio A 2014 Science 344 1492 |
| No Suggested Reading articles found! |
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
Altmetric
|
|
blogs
Facebook pages
Wikipedia page
Google+ users
|
Online attention
Altmetric calculates a score based on the online attention an article receives. Each coloured thread in the circle represents a different type of online attention. The number in the centre is the Altmetric score. Social media and mainstream news media are the main sources that calculate the score. Reference managers such as Mendeley are also tracked but do not contribute to the score. Older articles often score higher because they have had more time to get noticed. To account for this, Altmetric has included the context data for other articles of a similar age.
View more on Altmetrics
|
|
|
