To address real-time path planning requirements for multi-unmanned aerial vehicle(multi-UAV)collaboration in environments,this study proposes an improved multi-agent deep deterministic policy gradient algorithm with p...To address real-time path planning requirements for multi-unmanned aerial vehicle(multi-UAV)collaboration in environments,this study proposes an improved multi-agent deep deterministic policy gradient algorithm with prioritized experience replay(PER-MADDPG).By designing a multi-dimensional state representation incorporating relative positions,velocity vectors,and obstacle distance fields,we construct a composite reward function integrating safe obstacle avoidance,formation maintenance,and energy efficiency for environment perception and multiobjective collaborative optimization.The prioritized experience replay mechanism dynamically adjusts sampling weights based on temporal difference(TD)errors,enhancing learning efficiency for high-value samples.Simulation experiments demonstrate that our method generates real-time collaborative paths in 3D complex obstacle environments,reducing training time by 25.3%and 16.8%compared to traditional MADDPG and multi-agent twin delayed deep deterministic policy gradient(MATD3)algorithms respectively,while achieving smaller path length variances among UAVs.Results validate the effectiveness of prioritized experience replay in multi-agent collaborative decision-making.展开更多
The emergence of Unmanned Aerial Vehicle(UAV)-enabled Wireless Energy Transfer(WET)and Simultaneous Wireless Information and Power Transfer(SWIPT)technology provide a promising solution to overcome the energy sustaina...The emergence of Unmanned Aerial Vehicle(UAV)-enabled Wireless Energy Transfer(WET)and Simultaneous Wireless Information and Power Transfer(SWIPT)technology provide a promising solution to overcome the energy sustainability limitations of traditional harvesting-reliant sensor networks.However,in large-scale Battery-free SWIPT-enabled Sensor Networks(BSSN)characterized by sparse node distribution and heterogeneous energy consumption and harvesting rates,employing a single UAV for energy replenishment often suffers from insufficient operation continuity and low charging efficiency.To overcome these challenges,a Multi-UAV Collaborative Energy Charging for BSSN Based on Multi-Agent Deep Deterministic Policy Gradient(MCEC-MADDPG)is proposed in this paper.Specifically,we construct a collaborative one-to-one precision energy supply model where UAVs hover directly above specific nodes to achieve power transmission without complex beamforming requirements.To achieve collaborative scheduling among multiple UAVs in wide-area dynamic environments,the energy replenishment problem is first formulated as a Partially Observable Markov Decision Process(POMDP).Subsequently,the Centralized Training with Decentralized Execution(CTDE)architecture of the MADDPG algorithm is leveraged to solve this POMDP,which effectively tackles the non-stationarity challenge inherent in multi-agent environments.Simulation results demonstrate that MCEC-MADDPG exhibits superior performance in terms of convergence speed and stability.It enables the adaptive emergence of spatial-division collaborative strategies,significantly enhances the average residual energy of the network,and elevates the node survival rate to nearly 90%.Compared with Deep Deterministic Policy Gradient(DDPG),the traditional static Partition-Greedy method,the heuristic K-Means algorithm and the dynamic Two-Layer task allocation strategy,the proposed approach demonstrates substantial advantages.展开更多
How multi-unmanned aerial vehicles(UAVs)carrying a payload pass an obstacle-dense environment is practically important.Up to now,there have been few results on safe motion planning for the multi-UAVs cooperative trans...How multi-unmanned aerial vehicles(UAVs)carrying a payload pass an obstacle-dense environment is practically important.Up to now,there have been few results on safe motion planning for the multi-UAVs cooperative transportation system(CTS)to pass through such an environment.The prob-lem is challenging because it is difficult to analyze and explicitly take into account the swing motion of the payload in planning.In this paper,a modeling method of virtual tube is proposed by fus-ing the advantages of the existing modeling algorithm for regu-lar virtual tube and the expansion environment method.The pro-posed method can not only generate a safe and smooth tube for UAVs,but also ensure the payload stays away from the dense obstacles.Simulation results show the effectiveness of the method and the safety of the planned tube.展开更多
Due to its flexibility and complementarity, the multiUAVs system is well adapted to complex and cramped workspaces, with great application potential in the search and rescue(SAR) and indoor goods delivery fields. Howe...Due to its flexibility and complementarity, the multiUAVs system is well adapted to complex and cramped workspaces, with great application potential in the search and rescue(SAR) and indoor goods delivery fields. However, safe and effective path planning of multiple unmanned aerial vehicles(UAVs)in the cramped environment is always challenging: conflicts with each other are frequent because of high-density flight paths, collision probability increases because of space constraints, and the search space increases significantly, including time scale, 3D scale and model scale. Thus, this paper proposes a hierarchical collaborative planning framework with a conflict avoidance module at the high level and a path generation module at the low level. The enhanced conflict-base search(ECBS) in our framework is improved to handle the conflicts in the global path planning and avoid the occurrence of local deadlock. And both the collision and kinematic models of UAVs are considered to improve path smoothness and flight safety. Moreover, we specifically designed and published the cramped environment test set containing various unique obstacles to evaluating our framework performance thoroughly. Experiments are carried out relying on Rviz, with multiple flight missions: random, opposite, and staggered, which showed that the proposed method can generate smooth cooperative paths without conflict for at least 60 UAVs in a few minutes.The benchmark and source code are released in http://gffzz188fe103f8f1460asoxu6x5kk56cv6bop.ffgz.tsg.suse.edu.cn/inin-xingtian/multi-UAVs-path-planner.展开更多
A heterogeneous coverage method with multiple unmanned aerial vehicle assisted sink nodes(MUAVSs)for multi-objective optimization problem(MOP)is proposed,which is based on quantum wolf pack evolution algorithm(QWPEA)a...A heterogeneous coverage method with multiple unmanned aerial vehicle assisted sink nodes(MUAVSs)for multi-objective optimization problem(MOP)is proposed,which is based on quantum wolf pack evolution algorithm(QWPEA)and power law entropy(PLE)theory.The method is composed of preset move and autonomous coordination stages for satisfying non-repeated coverage,connectedness,and energy balance of sink layer critical requirements,which is actualized to cover sensors layer in large-scale outside wireless sensor networks(WSNs).Simulation results show that the performance of the proposed technique is better than the existing related coverage technique.展开更多
This paper is concerned with the cooperative target stalking for a multi-unmanned surface vehicle(multi-USV)system.Based on the multi-agent deep deterministic policy gradient(MADDPG)algorithm,a multi-USV target stalki...This paper is concerned with the cooperative target stalking for a multi-unmanned surface vehicle(multi-USV)system.Based on the multi-agent deep deterministic policy gradient(MADDPG)algorithm,a multi-USV target stalking(MUTS)algorithm is proposed.Firstly,a V-type probabilistic data extraction method is proposed for the first time to overcome shortcomings of the MADDPG algorithm.The advantages of the proposed method are twofold:1)it can reduce the amount of data and shorten training time;2)it can filter out more important data in the experience buffer for training.Secondly,in order to avoid the collisions of USVs during the stalking process,an action constraint method called Safe DDPG is introduced.Finally,the MUTS algorithm and some existing algorithms are compared in cooperative target stalking scenarios.In order to demonstrate the effectiveness of the proposed MUTS algorithm in stalking tasks,mission operating scenarios and reward functions are well designed in this paper.The proposed MUTS algorithm can help the multi-USV system avoid internal collisions during the mission execution.Moreover,compared with some existing algorithms,the newly proposed one can provide a higher convergence speed and a narrower convergence domain.展开更多
The formation control of unmanned aerial vehicle(UAV)swarms is of significant importance in various fields such as transportation,emergency management,and environmental monitoring.However,the complex dynamics,nonlinea...The formation control of unmanned aerial vehicle(UAV)swarms is of significant importance in various fields such as transportation,emergency management,and environmental monitoring.However,the complex dynamics,nonlinearity,uncertainty,and interaction among agents make it a challenging problem.In this paper,we propose a distributed robust control strategy that uses only local information of UAVs to improve the stability and robustness of the formation system in uncertain environments.We establish a nominal control strategy based on position relations and a semi-definite programming model to obtain control gains.Additionally,we propose a robust control strategy under the rotation setΩto address the noise and disturbance in the system,ensuring that even when the rotation angles of the UAVs change,they still form a stable formation.Finally,we extend the proposed strategy to a quadrotor UAV system with high-order kinematic models and conduct simulation experiments to validate its effectiveness in resisting uncertain disturbances and achieving formation control.展开更多
基金supported by the open project of National Key Laboratory of Air-Based Information Perception and Fusion(No.202462)。
摘要To address real-time path planning requirements for multi-unmanned aerial vehicle(multi-UAV)collaboration in environments,this study proposes an improved multi-agent deep deterministic policy gradient algorithm with prioritized experience replay(PER-MADDPG).By designing a multi-dimensional state representation incorporating relative positions,velocity vectors,and obstacle distance fields,we construct a composite reward function integrating safe obstacle avoidance,formation maintenance,and energy efficiency for environment perception and multiobjective collaborative optimization.The prioritized experience replay mechanism dynamically adjusts sampling weights based on temporal difference(TD)errors,enhancing learning efficiency for high-value samples.Simulation experiments demonstrate that our method generates real-time collaborative paths in 3D complex obstacle environments,reducing training time by 25.3%and 16.8%compared to traditional MADDPG and multi-agent twin delayed deep deterministic policy gradient(MATD3)algorithms respectively,while achieving smaller path length variances among UAVs.Results validate the effectiveness of prioritized experience replay in multi-agent collaborative decision-making.
基金supported by the National Natural Science Foundation of China(Grant No.62461041)the Natural Science Foundation of Jiangxi Province(Grant No.20224BAB212016)the China Scholarship Council(Grant No.202106825021).
摘要The emergence of Unmanned Aerial Vehicle(UAV)-enabled Wireless Energy Transfer(WET)and Simultaneous Wireless Information and Power Transfer(SWIPT)technology provide a promising solution to overcome the energy sustainability limitations of traditional harvesting-reliant sensor networks.However,in large-scale Battery-free SWIPT-enabled Sensor Networks(BSSN)characterized by sparse node distribution and heterogeneous energy consumption and harvesting rates,employing a single UAV for energy replenishment often suffers from insufficient operation continuity and low charging efficiency.To overcome these challenges,a Multi-UAV Collaborative Energy Charging for BSSN Based on Multi-Agent Deep Deterministic Policy Gradient(MCEC-MADDPG)is proposed in this paper.Specifically,we construct a collaborative one-to-one precision energy supply model where UAVs hover directly above specific nodes to achieve power transmission without complex beamforming requirements.To achieve collaborative scheduling among multiple UAVs in wide-area dynamic environments,the energy replenishment problem is first formulated as a Partially Observable Markov Decision Process(POMDP).Subsequently,the Centralized Training with Decentralized Execution(CTDE)architecture of the MADDPG algorithm is leveraged to solve this POMDP,which effectively tackles the non-stationarity challenge inherent in multi-agent environments.Simulation results demonstrate that MCEC-MADDPG exhibits superior performance in terms of convergence speed and stability.It enables the adaptive emergence of spatial-division collaborative strategies,significantly enhances the average residual energy of the network,and elevates the node survival rate to nearly 90%.Compared with Deep Deterministic Policy Gradient(DDPG),the traditional static Partition-Greedy method,the heuristic K-Means algorithm and the dynamic Two-Layer task allocation strategy,the proposed approach demonstrates substantial advantages.
基金supported by the National Natural Science Foundation of China(6237338661973327).
摘要How multi-unmanned aerial vehicles(UAVs)carrying a payload pass an obstacle-dense environment is practically important.Up to now,there have been few results on safe motion planning for the multi-UAVs cooperative transportation system(CTS)to pass through such an environment.The prob-lem is challenging because it is difficult to analyze and explicitly take into account the swing motion of the payload in planning.In this paper,a modeling method of virtual tube is proposed by fus-ing the advantages of the existing modeling algorithm for regu-lar virtual tube and the expansion environment method.The pro-posed method can not only generate a safe and smooth tube for UAVs,but also ensure the payload stays away from the dense obstacles.Simulation results show the effectiveness of the method and the safety of the planned tube.
基金partly supported by Program for the National Natural Science Foundation of China (62373052, U1913203, 61903034)Youth Talent Promotion Project of China Association for Science and TechnologyBeijing Institute of Technology Research Fund Program for Young Scholars。
摘要Due to its flexibility and complementarity, the multiUAVs system is well adapted to complex and cramped workspaces, with great application potential in the search and rescue(SAR) and indoor goods delivery fields. However, safe and effective path planning of multiple unmanned aerial vehicles(UAVs)in the cramped environment is always challenging: conflicts with each other are frequent because of high-density flight paths, collision probability increases because of space constraints, and the search space increases significantly, including time scale, 3D scale and model scale. Thus, this paper proposes a hierarchical collaborative planning framework with a conflict avoidance module at the high level and a path generation module at the low level. The enhanced conflict-base search(ECBS) in our framework is improved to handle the conflicts in the global path planning and avoid the occurrence of local deadlock. And both the collision and kinematic models of UAVs are considered to improve path smoothness and flight safety. Moreover, we specifically designed and published the cramped environment test set containing various unique obstacles to evaluating our framework performance thoroughly. Experiments are carried out relying on Rviz, with multiple flight missions: random, opposite, and staggered, which showed that the proposed method can generate smooth cooperative paths without conflict for at least 60 UAVs in a few minutes.The benchmark and source code are released in http://gffzz188fe103f8f1460asoxu6x5kk56cv6bop.ffgz.tsg.suse.edu.cn/inin-xingtian/multi-UAVs-path-planner.
基金Supported by the National Natural Science Foundation of China(No.61571318)Key Research and Development Project of Hainan(No.ZDYF2018006)+1 种基金Independent Innovation Fund of Tianjin UniversityDoctoral Fund Funded Projects
摘要A heterogeneous coverage method with multiple unmanned aerial vehicle assisted sink nodes(MUAVSs)for multi-objective optimization problem(MOP)is proposed,which is based on quantum wolf pack evolution algorithm(QWPEA)and power law entropy(PLE)theory.The method is composed of preset move and autonomous coordination stages for satisfying non-repeated coverage,connectedness,and energy balance of sink layer critical requirements,which is actualized to cover sensors layer in large-scale outside wireless sensor networks(WSNs).Simulation results show that the performance of the proposed technique is better than the existing related coverage technique.
基金supported in part by the National Natural Science Foundation of China(61873335,61833011,62173164)the Project of Science and Technology Commission of Shanghai Municipality,China(20ZR1420200,21SQBS01600,22JC1401400,19510750300,21190780300)the Natural Science Foundation of Jiangsu Province of China(BK20201451)。
摘要This paper is concerned with the cooperative target stalking for a multi-unmanned surface vehicle(multi-USV)system.Based on the multi-agent deep deterministic policy gradient(MADDPG)algorithm,a multi-USV target stalking(MUTS)algorithm is proposed.Firstly,a V-type probabilistic data extraction method is proposed for the first time to overcome shortcomings of the MADDPG algorithm.The advantages of the proposed method are twofold:1)it can reduce the amount of data and shorten training time;2)it can filter out more important data in the experience buffer for training.Secondly,in order to avoid the collisions of USVs during the stalking process,an action constraint method called Safe DDPG is introduced.Finally,the MUTS algorithm and some existing algorithms are compared in cooperative target stalking scenarios.In order to demonstrate the effectiveness of the proposed MUTS algorithm in stalking tasks,mission operating scenarios and reward functions are well designed in this paper.The proposed MUTS algorithm can help the multi-USV system avoid internal collisions during the mission execution.Moreover,compared with some existing algorithms,the newly proposed one can provide a higher convergence speed and a narrower convergence domain.
基金supported by the National Natural Science Foundation of China(Nos.52202391,U20A20155,and 52302397)the China Postdoctoral Science Foundation(No.2023M730173).
摘要The formation control of unmanned aerial vehicle(UAV)swarms is of significant importance in various fields such as transportation,emergency management,and environmental monitoring.However,the complex dynamics,nonlinearity,uncertainty,and interaction among agents make it a challenging problem.In this paper,we propose a distributed robust control strategy that uses only local information of UAVs to improve the stability and robustness of the formation system in uncertain environments.We establish a nominal control strategy based on position relations and a semi-definite programming model to obtain control gains.Additionally,we propose a robust control strategy under the rotation setΩto address the noise and disturbance in the system,ensuring that even when the rotation angles of the UAVs change,they still form a stable formation.Finally,we extend the proposed strategy to a quadrotor UAV system with high-order kinematic models and conduct simulation experiments to validate its effectiveness in resisting uncertain disturbances and achieving formation control.