Optimal control strategy for COVID-19 concerning both life and economy based on deep reinforcement learning
Wei Deng(邓为)1, Guoyuan Qi(齐国元)1,†, and Xinchen Yu(蔚昕晨)2
1 Tianjin Key Laboratory of Advanced Technology in Electrical Engineering and Energy, School of Control Science and Engineering, Tiangong University, Tianjin 300387, China; 2 School of Mechanical Engineering, Tiangong University, Tianjin 300387, China
Abstract At present, the global COVID-19 is still severe. More and more countries have experienced second or even third outbreaks. The epidemic is far from over until the vaccine is successfully developed and put on the market on a large scale. Inappropriate epidemic control strategies may bring catastrophic consequences. It is essential to maximize the epidemic restraining and to mitigate economic damage. However, the study on the optimal control strategy concerning both sides is rare, and no optimal model has been built. In this paper, the Susceptible-Infectious-Hospitalized-Recovered (SIHR) compartment model is expanded to simulate the epidemic's spread concerning isolation rate. An economic model affected by epidemic isolation measures is established. The effective reproduction number and the eigenvalues at the equilibrium point are introduced as the indicators of controllability and stability of the model and verified the effectiveness of the SIHR model. Based on the Deep Q Network (DQN), one of the deep reinforcement learning (RL) methods, the blocking policy is studied to maximize the economic output under the premise of controlling the number of infections in different stages. The epidemic control strategies given by deep RL under different learning strategies are compared for different reward coefficients. The study demonstrates that optimal policies may differ in various countries depending on disease spread and anti-economic risk ability. The results show that the more economical strategy, the less economic loss in the short term, which can save economically fragile countries from economic crises. In the second or third outbreak stage, the earlier the government adopts the control strategy, the smaller the economic loss. We recommend the method of deep RL to specify a policy which can control the epidemic while making quarantine economically viable.
Fund: Project supported by the National Natural Science Foundation of China (Grant No. 61873186) and the Tianjin Natural Science Foundation, China (Grant No. 17JCZDJC38300).
Wei Deng(邓为), Guoyuan Qi(齐国元), and Xinchen Yu(蔚昕晨) Optimal control strategy for COVID-19 concerning both life and economy based on deep reinforcement learning 2021 Chin. Phys. B 30 120203
[1] World Health Organization:"Coronavirus disease (COVID-2019) situation reports," https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports/. (accessed on April 13, 2021) [2] Chan J F, Yuan S F, Kok K H, To K K, Chu H, Yang J, Xing F, Liu J, Yip C C, Poon R W, et al. 2020 Lancet395 514 [3] Tong Z, Tang A, Li K, Li P, Wang H, Yi J, Zhang Y and Yan J 2020 Emerging Infectious Diseases26 1052 [4] Centers for disease control and prevention:2019 novel coronavirus, https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html 2021 (accessed on April 13, 2021) [5] Chetty R, Stepner M, Abraham S, Lin S, Scuderi B, Turner N, Bergeron A and Cutler D 2016 JAMA315 1750 [6] Cutler D M and Huang W "A. Lleras-Muney, Economic conditions and mortality:evidence from 200 Years of Data," NBER Working Papers [7] Enserink M and Kupferschmidt K 2020 Science367 1414 [8] Fang Y, Nie Y, and Penny M 2020 J. Med. Virol92 645 [9] Mandal M, Jana S, Nandi S K, Khatua A, Adak S and Kar T K 2020 Chaos, Solitons & Fractals136 109889 [10] Huang J and Qi G 2020 Nonlinear Dyn.101 1889 [11] Yu X, Qi G and Hu J 2021 Nonlinear Dyn.106 1149 [12] Wang Z, Xia C, Chen Z and Chen G 2020 IEEE Transactions on Cybernetics51 1454 [13] Huang J, Wang J and Xia C 2019 Chaos, Solitons & Fractals130 109425 [14] Rong X, Yang L, Chu H and Fan M 2020 Math. Biosci. Eng.17 2725 [15] Cui Y, Ni S and Shen S 2021 Chin. Phys. B30 048901 [16] Tong Y, King C and Hu Y 2021 Chin. Phys. B30 098903 [17] Arvind V, Kim J S, Cho B H, Mehmood A, Geng E and Samuel K 2021 Journal of Critical Care62 25 [18] Vaid S, Cakan C and Bhandari M 2020 JBJS102 e70 [19] Rustam F, Reshi A A, Mehmood A, Ullah S, On B W, Aslam W and Choi G S 2020 IEEE Access8 101489 [20] Goldsztejn U, Schwartzman D, and Nehorai A 2020 Plos One15 e0244174 [21] Berger D W, Herkenhoff K and Mongey S 2020 University of Chicago, Becker Friedman Institute for Economics Working Paper No. 2020-25 [22] Atkeson A and Andrew G 2020 Federal Reserve Bank of Minneapolis1 25 [23] Wu J, Xu X, Zhang P and Liu C 2011 Future Generation Computer Systems27 430 [24] Jamil A, Ganguly K and Nower N 2021 IET Intelligent Transport Systems14 2030 [25] Fotuhi F, Huynh N, Vidal J M, Jose M and Xie Y 2013 Research in Transportation Economics42 3 [26] Mnil V, Kavukcuoglu K, Silver D, Rusu A A, Veness J, Bellemare M G, Graves A, Riedmiller M, Fidjeland A K, Ostrovski G, et al. 2015 Nature518 529 [27] Sharma A, Anand S and Kaul S K 2020 Image and Vision Computing103 104022 [28] Chen M and Chan C 2021 Proc. Inst. Mech. Eng. Part D-J. Automob235 541 [29] Mousavi S S, Schukat M and Howley E 2016 In Proceedings of the SAI Intelligent Systems Conference, London, UK, 21-22 September 2016, p. 426 [30] Kermack W O and McKendrick A G 1927 Proc. R. Soc. Lond. A115 700 [31] Hu J, Qi G, Yu X and Xu L 2021 Dyn.106 1411 [32] Covid-19 vaccination in Italy https://lab24.ilsole24ore.com/numeri-vaccini-italia-mondo/(accessed on April 13, 2021)
Altmetric calculates a score based on the online attention an article receives. Each coloured thread in the circle represents a different type of online attention. The number in the centre is the Altmetric score. Social media and mainstream news media are the main sources that calculate the score. Reference managers such as Mendeley are also tracked but do not contribute to the score. Older articles often score higher because they have had more time to get noticed. To account for this, Altmetric has included the context data for other articles of a similar age.
