-
On Designing Multi-UAV aided Wireless Powered Dynamic Communication via Hierarchical Deep Reinforcement Learning
Authors:
Ze Yu Zhao,
Yue Ling Che,
Sheng Luo,
Gege Luo,
Kaishun Wu,
Victor C. M. Leung
Abstract:
This paper proposes a novel design on the wireless powered communication network (WPCN) in dynamic environments under the assistance of multiple unmanned aerial vehicles (UAVs). Unlike the existing studies, where the low-power wireless nodes (WNs) often conform to the coherent harvest-then-transmit protocol, under our newly proposed double-threshold based WN type updating rule, each WN can dynamic…
▽ More
This paper proposes a novel design on the wireless powered communication network (WPCN) in dynamic environments under the assistance of multiple unmanned aerial vehicles (UAVs). Unlike the existing studies, where the low-power wireless nodes (WNs) often conform to the coherent harvest-then-transmit protocol, under our newly proposed double-threshold based WN type updating rule, each WN can dynamically and repeatedly update its WN type as an E-node for non-linear energy harvesting over time slots or an I-node for transmitting data over sub-slots. To maximize the total transmission data size of all the WNs over T slots, each of the UAVs individually determines its trajectory and binary wireless energy transmission (WET) decisions over times slots and its binary wireless data collection (WDC) decisions over sub-slots, under the constraints of each UAV's limited on-board energy and each WN's node type updating rule. However, due to the UAVs' tightly-coupled trajectories with their WET and WDC decisions, as well as each WN's time-varying battery energy, this problem is difficult to solve optimally. We then propose a new multi-agent based hierarchical deep reinforcement learning (MAHDRL) framework with two tiers to solve the problem efficiently, where the soft actor critic (SAC) policy is designed in tier-1 to determine each UAV's continuous trajectory and binary WET decision over time slots, and the deep-Q learning (DQN) policy is designed in tier-2 to determine each UAV's binary WDC decisions over sub-slots under the given UAV trajectory from tier-1. Both of the SAC policy and the DQN policy are executed distributively at each UAV. Finally, extensive simulation results are provided to validate the outweighed performance of the proposed MAHDRL approach over various state-of-the-art benchmarks.
△ Less
Submitted 6 June, 2024; v1 submitted 13 December, 2023;
originally announced December 2023.
-
Energy-Efficient UAV Multicasting with Simultaneous FSO Backhaul and Power Transfer
Authors:
Yue Ling Che,
Weibin Long,
Sheng Luo,
Kaishun Wu,
Rui Zhang
Abstract:
This letter studies an unmanned aerial vehicle (UAV) aided multicasting (MC) system, which is enabled by simultaneous free space optics (FSO) backhaul and power transfer. The UAV applies the power-splitting technique to harvest wireless power and decode backhaul information simultaneously over the FSO link, while at the same time using the harvested power to multicast the backhauled information ov…
▽ More
This letter studies an unmanned aerial vehicle (UAV) aided multicasting (MC) system, which is enabled by simultaneous free space optics (FSO) backhaul and power transfer. The UAV applies the power-splitting technique to harvest wireless power and decode backhaul information simultaneously over the FSO link, while at the same time using the harvested power to multicast the backhauled information over the radio frequency (RF) links to multiple ground users (GUs). We derive the UAV's achievable MC rate under the Poisson point process (PPP) based GU distribution. By jointly designing the FSO and RF links and the UAV altitude, we maximize the system-level energy efficiency (EE), which can be equivalently expressed as the ratio of the UAV's MC rate over the optics base station (OBS) transmit power, subject to the UAV's sustainable operation and reliable backhauling constraints. Due to the non-convexity of this problem, we propose suboptimal solutions with low complexity. Numerical results show the close-to-optimal EE performance by properly balancing the power-rate tradeoff between the FSO power and the MC data transmissions.
△ Less
Submitted 7 April, 2021;
originally announced April 2021.
-
Spatial Throughput Maximization of Wireless Powered Communication Networks
Authors:
Yue Ling Che,
Lingjie Duan,
Rui Zhang
Abstract:
Wireless charging is a promising way to power wireless nodes' transmissions. This paper considers new dual-function access points (APs) which are able to support the energy/information transmission to/from wireless nodes. We focus on a large-scale wireless powered communication network (WPCN), and use stochastic geometry to analyze the wireless nodes' performance tradeoff between energy harvesting…
▽ More
Wireless charging is a promising way to power wireless nodes' transmissions. This paper considers new dual-function access points (APs) which are able to support the energy/information transmission to/from wireless nodes. We focus on a large-scale wireless powered communication network (WPCN), and use stochastic geometry to analyze the wireless nodes' performance tradeoff between energy harvesting and information transmission. We study two cases with battery-free and battery-deployed wireless nodes. For both cases, we consider a harvest-then-transmit protocol by partitioning each time frame into a downlink (DL) phase for energy transfer, and an uplink (UL) phase for information transfer. By jointly optimizing frame partition between the two phases and the wireless nodes' transmit power, we maximize the wireless nodes' spatial throughput subject to a successful information transmission probability constraint. For the battery-free case, we show that the wireless nodes prefer to choose small transmit power to obtain large transmission opportunity. For the battery-deployed case, we first study an ideal infinite-capacity battery scenario for wireless nodes, and show that the optimal charging design is not unique, due to the sufficient energy stored in the battery. We then extend to the practical finite-capacity battery scenario. Although the exact performance is difficult to be obtained analytically, it is shown to be upper and lower bounded by those in the infinite-capacity battery scenario and the battery-free case, respectively. Finally, we provide numerical results to corroborate our study.
△ Less
Submitted 7 January, 2015; v1 submitted 10 September, 2014;
originally announced September 2014.
-
On Spatial Capacity of Wireless Ad Hoc Networks with Threshold Based Scheduling
Authors:
Yue Ling Che,
Rui Zhang,
Yi Gong,
Lingjie Duan
Abstract:
This paper studies spatial capacity in a stochastic wireless ad hoc network, where multi-stage probing and data transmission are sequentially performed. We propose a novel signal-to-interference-ratio (SIR) threshold based scheduling scheme, where by starting with the first probing, each transmitter iteratively decides to further probe or stay idle, depending on whether the estimated SIR in the pr…
▽ More
This paper studies spatial capacity in a stochastic wireless ad hoc network, where multi-stage probing and data transmission are sequentially performed. We propose a novel signal-to-interference-ratio (SIR) threshold based scheduling scheme, where by starting with the first probing, each transmitter iteratively decides to further probe or stay idle, depending on whether the estimated SIR in the proceeding probing is larger or smaller than a predefined threshold. Although one can assume that the transmitters are initially deployed according to a homogeneous Poisson point process (PPP), the SIR based scheduling makes the PPP no longer applicable to model the locations of retained transmitters in the subsequent probing and data transmission phases, due to the interference induced coupling in their decisions. We first focus on single-stage probing and find that when the SIR threshold is set sufficiently small to assure an acceptable interference level in the network, the proposed scheme can greatly outperform the non-scheduling reference scheme in terms of spatial capacity. We clearly characterize the spatial capacity and obtain exact/approximate closed-form expressions, by proposing a new approximate approach to deal with the correlated SIR distributions over non-Poisson point processes. Then we successfully extend to multi-stage probing by properly designing the multiple SIR thresholds to assure gradual improvement of the spatial capacity. Furthermore, we analyze the impact of multi-stage probing overhead and present a probing-capacity tradeoff in scheduling design. Finally, extensive numerical results are presented to demonstrate the performance of the proposed scheduling as compared to existing schemes.
△ Less
Submitted 9 September, 2014;
originally announced September 2014.
-
On Design of Opportunistic Spectrum Access in the Presence of Reactive Primary Users
Authors:
Yue Ling Che,
Rui Zhang,
Yi Gong
Abstract:
Opportunistic spectrum access (OSA) is a key technique enabling the secondary users (SUs) in a cognitive radio (CR) network to transmit over the "spectrum holes" unoccupied by the primary users (PUs). In this paper, we focus on the OSA design in the presence of reactive PUs, where PU's access probability in a given channel is related to SU's past access decisions. We model the channel occupancy of…
▽ More
Opportunistic spectrum access (OSA) is a key technique enabling the secondary users (SUs) in a cognitive radio (CR) network to transmit over the "spectrum holes" unoccupied by the primary users (PUs). In this paper, we focus on the OSA design in the presence of reactive PUs, where PU's access probability in a given channel is related to SU's past access decisions. We model the channel occupancy of the reactive PU as a 4-state discrete-time Markov chain. We formulate the optimal OSA design for SU throughput maximization as a constrained finite-horizon partially observable Markov decision process (POMDP) problem. We solve this problem by first considering the conventional short-term conditional collision probability (SCCP) constraint. We then adopt a long-term PU throughput (LPUT) constraint to effectively protect the reactive PU transmission. We derive the structure of the optimal OSA policy under the LPUT constraint and propose a suboptimal policy with lower complexity. Numerical results are provided to validate the proposed studies, which reveal some interesting new tradeoffs between SU throughput maximization and PU transmission protection in a practical interaction scenario.
△ Less
Submitted 25 April, 2013;
originally announced April 2013.
