Please wait a minute...
Chin. Phys. B, 2018, Vol. 27(1): 010203    DOI: 10.1088/1674-1056/27/1/010203
GENERAL Prev   Next  

A novel stable value iteration-based approximate dynamic programming algorithm for discrete-time nonlinear systems

Yan-Hua Qu(曲延华), An-Na Wang(王安娜), Sheng Lin(林盛)
College of Information Science and Engineering, Northeastern University, Shenyang 110819, China
Abstract  The convergence and stability of a value-iteration-based adaptive dynamic programming (ADP) algorithm are considered for discrete-time nonlinear systems accompanied by a discounted quadric performance index. More importantly than sufficing to achieve a good approximate structure, the iterative feedback control law must guarantee the closed-loop stability. Specifically, it is firstly proved that the iterative value function sequence will precisely converge to the optimum. Secondly, the necessary and sufficient condition of the optimal value function serving as a Lyapunov function is investigated. We prove that for the case of infinite horizon, there exists a finite horizon length of which the iterative feedback control law will provide stability, and this increases the practicability of the proposed value iteration algorithm. Neural networks (NNs) are employed to approximate the value functions and the optimal feedback control laws, and the approach allows the implementation of the algorithm without knowing the internal dynamics of the system. Finally, a simulation example is employed to demonstrate the effectiveness of the developed optimal control method.
Keywords:  adaptive dynamic programming (ADP)      convergence      stability      discounted quadric performance index  
Received:  04 July 2017      Revised:  11 October 2017      Published:  05 January 2018
PACS:  02.60.Gf (Algorithms for functional approximation)  
  02.30.Jr (Partial differential equations)  
  02.30.Yy (Control theory)  
Corresponding Authors:  Yan-Hua Qu     E-mail:  quyanhuawang@sina.com

Cite this article: 

Yan-Hua Qu(曲延华), An-Na Wang(王安娜), Sheng Lin(林盛) A novel stable value iteration-based approximate dynamic programming algorithm for discrete-time nonlinear systems 2018 Chin. Phys. B 27 010203

[1] Bellman R E 1957 Dynamic Programming (Princeton: Princeton University Press)
[2] Zhang H G, Liu D R, Luo Y H and Wang D 2013 Adaptive Dynamic Programming for Control-algorithms and Stability (London: SpringerVerlag)
[3] Werbos P J 1992 Approximate Dynamic Programming for Real-time Control and Neural Modeling, Handbook of Intelligent Control (New York: Van Nostrand Reinhold)
[4] Bertsekas D P and Tsitsiklis J N 1995 Proceedings of the 34th IEEE Conference on IEEE, 1995, pp. 560-564
[5] Prokhorov D V and Wunsch D C 1997 IEEE Trans. Neural Netw. 8 997
[6] Wang F Y, Zhang H G and Liu D R 2009 IEEE Computational Intelligence Magazine 4 39
[7] Doya K 2000 Neural Computation 12 219
[8] Liang H J, Li H Y, Yu Z D, Li P and Wang W 2017 IET Control Theory & Applications 11 1928
[9] Hanselmann T, Noakes L and Zaknich A 2007 IEEE Trans on Neural Networks 18 631
[10] Wei Q L, Shi G, Song R Z and Liu Y 2017 IEEE Trans. Indus. Elect. 64 5468
[11] Howard R 1960 Dynamic Programming and Markov Processes (Cambridge: MIT Press)
[12] Song R Z, Xiao W D, Sun C Y and Wei Q L 2015 Chin. Phys. B 22090502
[13] Wei Q L, Lewis F L, Shi G and Song R Z 2017 IEEE Trans. Indus. Elect. 64 9527
[14] Leake R J and Liu R W 1967 J. SIAM Control 5 54
[15] Song R Z and Wei Q L 2017 Chin. Phys. B 26 030505
[16] Wei Q L, Song R Z, S Q Y and Xiao W D 2015 Chin. Phys. B 24090504
[17] Beard R, Saridis G and Wen J 1997 Automatica 33 2158
[18] Wei Q L, Liu D R and Xu Y C 2015 Chin. Phys. B 24 030502
[19] Song R Z, Xiao W D and Wei Q L 2014 Chin. Phys. B 23 050504
[20] Wei Q L, Liu D R, Lewis F L, Liu Y and Zhang J 2017 IEEE Trans. Indus. Elect. 64 4110
[21] Zhang Y, Zhang Z, Qian H and Hu G 2017 Chin. Phys. B 26 100508
[22] Wei Q L, Liu D R, and Lin H Q 2016 IEEE Trans. Cybernetics 46 840
[23] Hong Y Y, Qiang Z and Qi J Z 2017 Chin. Phys. B 26 100506
[24] Wei Q L, Lewis F L, Sun Q Y, Yan P F and Song R Z 2017 IEEE Trans. Cybernetics 47 1224
[25] Abu-Khalaf M and Lewis F L 2005 Automatica 41 779
[26] Zhang H G, Wei Q L and Liu D R 2011 Automatica 47 207
[27] Vrabie D and Lewis F L 2009 Proceedings of International Joint Conference on Neural Networks, Atlanta, pp. 3224-3231
[28] Lewis F L and Vamvoudakis K G 2011 IEEE Trans. Syst., Man, Cybern. B 41 14
[29] Song R Z, Lewis F L and Wei Q L Vrabie D and Lewis F L 2017 IEEE Trans. Neural Netw. Learn. Syst. 28 704
[30] Zhang H G, Luo Y H and Liu D R 2009 IEEE Trans. Neural Netw. 201490
[31] Zhang H G, Wei Q L and Luo Y H 2008 IEEE Trans. Syst., Man, Cybern. B 38 937
[32] Al-Tamimi A, Lewis F L and Abu-Khalaf M 2008 IEEE Trans. Syst., Man, Cybern. B 38 943
[33] Li H L, Liu D R and Wang D 2012 Proceedings of the 31st Chinese Control Conference, July 25-27, 2012, Hefei, China, p. 2932
[34] Wei Q L and Liu D R Vrabie D and Lewis F L 2012 WCCI 2012 IEEE World Congress on Computational Intelligence, June, 10-15, 2012, Brisbane, Australia
[35] Wang D, Liu D R, Wei Q L, Zhao D B and Jin N 2012 Automatica 481825
[36] Primbs J A and Nevistic V 2000 Automatica 36 965
[37] Lewis F L and Vrabie D 2009 Proceedings of the 7th Asian Control Conference, Hong Kong, China, August 27-29, 2009
[38] Rantzer A IEE Proc. Control Theory Appl. 153 567
[39] Lincoln B and Rantzer A Vrabie D and Lewis F L 2006 IEEE Trans. Autom. Con. 51 1249
[1] Stability and optoelectronic property of low-dimensional organic tin bromide perovskites
J H Lei(雷军辉), Q Tang(汤琼), J He(何军), and M Q Cai(蔡孟秋). Chin. Phys. B, 2021, 30(3): 038102.
[2] A meshless algorithm with the improved moving least square approximation for nonlinear improved Boussinesq equation
Yu Tan(谭渝) and Xiao-Lin Li(李小林). Chin. Phys. B, 2021, 30(1): 010201.
[3] Surface active agents stabilize nanodroplets and enhance haze formation
Yunqing Ma(马韵箐), Changsheng Chen(陈昌盛), and Xianren Zhang(张现仁). Chin. Phys. B, 2021, 30(1): 010504.
[4] Propagation dynamics of relativistic electromagnetic solitary wave as well as modulational instability in plasmas
Rong-An Tang(唐荣安), Tiao-Fang Liu(刘调芳), Xue-Ren Hong(洪学仁), Ji-Ming Gao(高吉明), Rui-Jin Cheng(程瑞锦), You-Lian Zheng(郑有莲), and Ju-Kui Xue(薛具奎). Chin. Phys. B, 2021, 30(1): 015201.
[5] Vanadium based XVO3 (X=Na, K, Rb) as promising thermoelectric materials: First-principle DFT calculations
N A Noor, Nosheen Mushahid, Aslam Khan, Nessrin A. Kattan, Asif Mahmood, Shahid M. Ramay. Chin. Phys. B, 2020, 29(9): 097101.
[6] Study of optical clocks based on ultracold 171Yb atoms
Di Ai(艾迪), Hao Qiao(谯皓), Shuang Zhang(张爽), Li-Meng Luo(骆莉梦), Chang-Yue Sun(孙常越), Sheng Zhang(张胜), Cheng-Quan Peng(彭成权), Qi-Chao Qi(齐启超), Tao-Yun Jin(金涛韫), Min Zhou(周敏), Xin-Ye Xu(徐信业). Chin. Phys. B, 2020, 29(9): 090601.
[7] Experimental investigation on the properties of liquid film breakup induced by shock waves
Xianzhao Song(宋先钊), Bin Li(李斌), Lifeng Xie(解立峰). Chin. Phys. B, 2020, 29(8): 086201.
[8] Thermal stability of magnetron sputtering Ge-Ga-S films
Lei Niu(牛磊), Yimin Chen(陈益敏), Xiang Shen(沈祥), Tiefeng Xu(徐铁峰). Chin. Phys. B, 2020, 29(8): 087803.
[9] Tilt adjustment for a portable absolute atomic gravimeter
Hong-Tai Xie(谢宏泰), Bin Chen(陈斌), Jin-Bao Long(龙金宝), Chun Xue(薛春), Luo-Kan Chen(陈泺侃), Shuai Chen(陈帅). Chin. Phys. B, 2020, 29(7): 073701.
[10] Progress on the 40Ca+ ion optical clock
Baolin Zhang(张宝林), Yao Huang(黄垚), Huaqing Zhang(张华青), Yanmei Hao(郝艳梅), Mengyan Zeng(曾孟彦), Hua Guan(管桦), Kelin Gao(高克林). Chin. Phys. B, 2020, 29(7): 074209.
[11] A transportable optical lattice clock at the National Time Service Center
De-Huan Kong(孔德欢), Zhi-Hui Wang(王志辉), Feng Guo(郭峰), Qiang Zhang(张强), Xiao-Tong Lu(卢晓同), Ye-Bing Wang(王叶兵), Hong Chang(常宏). Chin. Phys. B, 2020, 29(7): 070602.
[12] A novel class of two-dimensional chaotic maps with infinitely many coexisting attractors
Li-Ping Zhang(张丽萍), Yang Liu(刘洋), Zhou-Chao Wei(魏周超), Hai-Bo Jiang(姜海波), Qin-Sheng Bi(毕勤胜). Chin. Phys. B, 2020, 29(6): 060501.
[13] Solid angle car following model
Dongfang Ma(马东方), Yueyi Han(韩月一), Sheng Jin(金盛). Chin. Phys. B, 2020, 29(6): 060504.
[14] Energy transfer, luminescence properties, and thermal stability of color tunable barium pyrophosphate phosphors
Meng-Jiao Xu(徐梦姣), Su-Xia Li(李素霞), Chen-Chen Ji(季辰辰), Wan-Xia Luo(雒晚霞), Lu-Xiang Wang(王鲁香). Chin. Phys. B, 2020, 29(6): 063301.
[15] Multistability and coexisting transient chaos in a simple memcapacitive system
Fu-Ping Wang(王富平), Fa-Qiang Wang(王发强). Chin. Phys. B, 2020, 29(5): 058502.
No Suggested Reading articles found!