文献类型：期刊文献

英文题名：Target Tracking Control of UAV Through Deep Reinforcement Learning

作者：Ma, Bodi[1];Liu, Zhenbao[1,2];Zhao, Wen[1];Yuan, Jinbiao[1];Long, Hao[3,4,5];Wang, Xiao[1];Yuan, Zhirong

第一作者：Ma, Bodi

通讯作者：Liu, ZB[1]

机构：[1]Northwestern Polytech Univ, Sch Civil Aviat, Xian 710072, Peoples R China;[2]Northwestern Polytech Univ, Res & Dev Inst Shenzhen, Shenzhen 518071, Peoples R China;[3]Beijing Union Univ, Beijing Key Lab Informat Serv Engn, Beijing 100101, Peoples R China;[4]Beijing Union Univ, Coll Robot, Beijing 100027, Peoples R China;[5]Northwestern Polytech Univ, Inst 365, Xian 710072, Peoples R China

第一机构：Northwestern Polytech Univ, Sch Civil Aviat, Xian 710072, Peoples R China

通讯机构：[1]corresponding author), Northwestern Polytech Univ, Sch Civil Aviat, Xian 710072, Peoples R China.

年份：0

外文期刊名：IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

收录：;EI(收录号：20231113737162);Scopus(收录号：2-s2.0-85149845072);WOS:【SCI-EXPANDED(收录号:WOS:000947796800001)】；

基金：This work was supported in part by the National Natural Science Foundation Fund under Grant 52072309, in part by the Key Research and Development Program of Shaanxi Province under Grant 2019ZDLGY14-02-01, in part by the Shenzhen Fundamental Research Program under Grant JCYJ20190806152203506, in part by the Aeronautical Science Foundation of China under Grant ASFC-2018ZC53026, in part by the Beijing Institute of Spacecraft System Engineering Research Project under Grant JSZL2020203B004, in part by the Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University under Grant CX2021033, in part by the Basic Research Program of Taicang under Grant TC2021JC09, in part by the Natural Science Foundation of Shaanxi Province under Grant 2023-JC-QN-0003 and Grant 2023-JC-QN-0665, and in part by the Premium Funding Project for Academic Human Resources Development in Beijing Union University under Grant BPHR2020CZ03.

语种：英文

外文关键词：Unmanned aerial vehicles; target tracking control; reinforcement learning; intelligent control system

摘要：This study presents an innovative reinforcement-learning-based control algorithm for a vertical take-off and landing (VTOL) aircraft under wind disturbances. In our approach, the tracking control problem of the VTOL aircraft is formulated as a Markov decision process, and the appropriate system state, reward function, and soft update method are presented. To improve the control accuracy under wind disturbances, three kinds of wind fields were added in the learning environment to expand the exploration space and simulate the effect of wind disturbances on the flight control. Moreover, to ensure the tracking accuracy and the practical implementation, a quantum-inspired experience replay strategy was developed based on quantum computation theory. In this strategy, the preparation operation scheme was designed to encourage the exploration and speed up the convergence. The depreciation operation method was developed to enrich the sample diversity, which increased the robustness of the controller and allowed the control strategy learned in the numerical simulations to be directly transferred into real-world environments. Numerical simulations, hardware-in-the-loop experiments, and real-world flight experiments were conducted to evaluate the performance and merits of the proposed method. The results demonstrated high accuracy and effectiveness and good robustness of the proposed control algorithm in terms of standoff target tracking control and flight stability.