arXiv subjects

Liqun Fu

Publications and source records attributed to Liqun Fu.

At least 19 recordsLinked to original sources

arXiv2026

Delay-Robust Deep Reinforcement Learning for Ranging-Free Channel Access under Mobility in Underwater Acoustic Networks

Long propagation delays in underwater acoustic networks (UWANs) cause spatio-temporal uncertainty, constraining channel utilization in medium access control (MAC) protocols. Node mobility within autonomous underwater vehicle scenarios exacerbates these challenges by introducing dynamic propagation delays and varying spatial topologies. We present MobiU-MAC, a deep reinforcement learning (DRL)-based MAC protocol for mobile node access in UWANs that maximizes throughput via autonomous learning. MobiU-MAC incorporates CHILL-STER, a novel DRL algorithm optimized for UWANs that is both ranging-free and delay-robust. CHILL-STER employs a credit horizon-limited $λ$-return (CHILL-Return) mechanism to achieve stable learning under asynchronous delayed rewards, while the companion spatio-temporal experience replay (STER) mechanism addresses topological changes arising from node mobility. This work also demonstrates theoretically that DRL attains optimal policy learning equivalent to a standard Markov decision process under long propagation delays without requiring ranging. Performance evaluations indicate that MobiU-MAC outperforms existing DRL-based MAC protocols for UWANs by leveraging the maximum system delay boundary without ranging overhead, supporting the effectiveness of the proposed theory and algorithm in complex underwater dynamic environments.

cs.NI↗

arXiv2026

PAST: Pilot and Adaptive Orchestration for Timely and Resilient Service Delivery in Edge-Assisted UAV Networks under Spatio-Temporal Dynamics

Incentive-driven resource trading is essential for uncrewed aerial vehicle (UAV) applications with intensive, time-sensitive computing demands. Traditional spot trading suffers from negotiation delays and high energy costs, while conventional futures trading struggles to adapt to the dynamic, uncertain UAV-edge environment. To address these challenges, we propose PAST (pilot-and-adaptive stable trading), a novel framework for edge-assisted UAV networks with spatio-temporal dynamism. PAST integrates two complementary mechanisms: PilotAO (pilot trading agreements with overbooking), a risk-aware, overbooking-enabled early-stage decision-making module that establishes long-term, mutually beneficial agreements and boosts resource utilization; and AdaptAO (adaptive trading agreements with overbooking rate update), an intelligent adaptation module that dynamically updates agreements and overbooking rates based on UAV mobility, supply-demand variations, and agreement performance. Together, these mechanisms integrate stable advance agreement construction with execution-feedback-driven adaptive renewal. Under the stated modeling assumptions, PilotAO satisfies the adopted individual-rationality and strong-stability criteria, as well as the competitive equilibrium and weak Pareto properties established for the considered trading model. Comprehensive experiments further demonstrate that, by coordinating advance planning with selective agreement renewal, PAST effectively balances stability, adaptability, and decision-making efficiency under dynamic UAV-edge conditions.

cs.DC↗

arXiv2026

Achieving Rate-Concurrency Balance for Underwater Concurrent Random Access

Underwater acoustic networks face a fundamental rate--concurrency tradeoff: high-rate waveforms (e.g., OFDM, OTFS) are designed for point-to-point links and rely on orthogonal MAC protocols (e.g., TDMA) to avoid collisions, sacrificing concurrency; conversely, collision-resilient waveforms (e.g., CDMA, ZCMod) support uncoordinated access but are inherently rate-limited by spreading or sparse index modulation. We present \system, a cross-layer concurrent random-access system that combines two new components: (i) \textbf{EZCDM}, an equidistant ZC division-multiplexing waveform that activates multiple cyclic shifts of a ZC root as parallel sub-channels with a tunable rate--robustness tradeoff, and an intra-symbol differential receiver that eliminates the shared multipath channel response without explicit CIR estimation; and (ii) a \textbf{cross-layer link adaptation (LA) framework} featuring beacon-framed random access, user-specific closed-loop power control, and overlap- and CIR-aware common-MS selection. Channel-trace- and signal-trace-driven physical-layer experiments combined with PHY-in-the-loop network simulations demonstrate that \system\ achieves significant BER and throughput gains over conventional waveforms and MAC protocols by converting traditionally destructive collisions into decodable concurrent streams.

cs.NI↗

arXiv2026

FUSION: Forecast-Embedded Agent Scheduling with Service Incentive Optimization over Distributed Air-Ground Edge Networks

In this paper, we introduce a first-of-its-kind forecasting-driven, incentive-aware service provisioning framework for distributed air-ground integrated networks that explicitly accounts for human-machine coexistence. In our framework, vehicular-UAV agent pairs (APs) are proactively dispatched to overloaded hotspots to augment the computing capacity of edge servers (ESs), which in turn gives rise to a set of challenges that we jointly address: highly uncertain spatio-temporal workloads, spatio-temporal coupling between road traffic and UAV capacity, forecast-driven contracting risks, and heterogeneous quality-of-service (QoS) requirements of human users (HUs) and machine users (MUs). To address these challenges, we propose FUSION, a two-stage optimization framework, consisting of an offline stage and an online stage. In the offline stage, a liquid neural network-powered module performs multi-step spatio-temporal demand forecasting at distributed ESs, whose outputs are exploited by an enhanced ant colony optimization-based routing scheme and an auction-based incentive-aware contracting mechanism, to jointly determine ES-AP contracts and pre-planned service routes. In the online stage, we model congestion-aware task scheduling as a service demander-side exact-potential game, where HUs and MUs select either local execution or accessible ES/UAV resources, and develop a potential-guided best-response dynamics algorithm that converges to an \varepsilon-NE under a positive improvement threshold and to a pure-strategy Nash equilibrium when the threshold is zero. Experiments on both synthetic and real-world datasets show that FUSION consistently achieves higher social welfare and improved resource utilization, while maintaining latency and energy costs comparable to state-of-the-art baselines and preserving individual rationality, budget balance, and near-truthfulness.

cs.NI↗

arXiv2026

Iterative Receiver Processing at Relays in PNC-Enabled Multi-Hop Underwater Acoustic Networks

Physical-layer network coding (PNC) can increase end-to-end throughput in bi-directional multi-hop underwater acoustic (UWA) networks. However, multipath delay spread and Doppler-induced inter-carrier interference (ICI) in UWA channels can degrade the reliability of PNC transmission in a three-node relay configuration. More critically, error accumulation across multiple relay nodes leads to a pronounced increase in the end-to-end bit error rate (BER) in multi-hop networks. To address this issue, we develop an iterative detection and decoding processing strategy for relay nodes within a PNC-enabled multi-hop UWA network based on orthogonal frequency division multiplexing (OFDM) modulation. The proposed design integrates three key algorithms: (i) an adaptive channel-aware factor graph detection algorithm that is suited for time-varying UWA channels; (ii) a parity-check-constrained soft-information refinement algorithm that improves the accuracy of the information feedback from the decoder to the detector; and (iii) a linear minimum mean square error (LMMSE) detection algorithm based on a superimposed model, which offers low computational complexity as an alternative scheme. Extensive simulation results demonstrate that the adaptive detection algorithm achieves BERs on the order of $10^{-5}$ at a relative velocity of 1.5 m/s UWA channel and a signal-to-noise (SNR) of 8~dB. Both lake experiments and sea trials in the Taiwan Strait confirm that the proposed iterative receiver algorithms outperform baseline schemes in terms of BER performance under practical UWA channel conditions, showing their robustness and applicability in real multi-hop deployments.

cs.NI↗

arXiv2026

Cross-Layer Traffic Allocation and Contention Window Optimization for Wi-Fi 7 MLO: When DRL Meets LSTM

To support future diverse applications, multi-link operation (MLO) has been introduced in the Wi-Fi 7 standard (IEEE 802.11be) to enable concurrent communication over multiple frequency bands. This new capability relies on a two-tier medium access control (MAC) architecture, where the upper MAC (U-MAC) allocates traffic across links and the lower MAC (L-MAC) performs independent channel access. However, MLO optimization is challenging due to the inherent coupling between the U-MAC and L-MAC, as well as the dynamic and complex nature of wireless networks. To address these challenges, we propose a cross-layer framework that jointly optimizes traffic allocation at the U-MAC layer and initial contention window (ICW) sizes at the L-MAC layer to maximize network throughput. Specifically, we extend the single-link Bianchi Markov model to develop an analytical framework that captures the relationship among network throughput, traffic allocation, and ICW sizes. Based on this framework, we formulate a nonconvex, nonlinear cross-layer optimization problem. To solve it efficiently, we design a long short-term memory-based soft actor-critic (LSTM-SAC) algorithm that leverages LSTM to handle the partial observability and non-Markovian dynamics inherent in Wi-Fi networks. Finally, using a well-developed event-based Wi-Fi simulator, we demonstrate that the proposed LSTM-SAC substantially outperforms existing benchmark solutions across a wide range of network settings.

cs.NI↗

arXiv2026

Adaptive UAV-Assisted Hierarchical Federated Learning: Optimizing Energy, Latency, and Resilience for Dynamic Smart IoT

Hierarchical Federated Learning (HFL) extends conventional Federated Learning (FL) by introducing intermediate aggregation layers, enabling distributed learning in geographically dispersed environments, particularly relevant for smart IoT systems, such as remote monitoring and battlefield operations, where cellular connectivity is limited. In these scenarios, UAVs serve as mobile aggregators, dynamically connecting terrestrial IoT devices. This paper investigates an HFL architecture with energy-constrained, dynamically deployed UAVs prone to communication disruptions. We propose a novel approach to minimize global training costs by formulating a joint optimization problem that integrates learning configuration, bandwidth allocation, and device-to-UAV association, ensuring timely global aggregation before UAV disconnections and redeployments. The problem accounts for dynamic IoT devices and intermittent UAV connectivity and is NP-hard. To tackle this, we decompose it into three subproblems: \textit{(i)} optimizing learning configuration and bandwidth allocation via an augmented Lagrangian to reduce training costs; \textit{(ii)} introducing a device fitness score based on data heterogeneity (via Kullback-Leibler divergence), device-to-UAV proximity, and computational resources, using a TD3-based algorithm for adaptive device-to-UAV assignment; \textit{(iii)} developing a low-complexity two-stage greedy strategy for UAV redeployment and global aggregator selection, ensuring efficient aggregation despite UAV disconnections. Experiments on diverse real-world datasets validate the approach, demonstrating cost reduction and robust performance under communication disruptions.

cs.LG↗

arXiv2026

ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge

Training large language models (LLMs) at the network edge faces fundamental challenges arising from device resource constraints, severe data heterogeneity, and heightened privacy risks. To address these challenges, we propose ELSA (Efficient LLM-centric Split Aggregation), a novel framework that systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks. ELSA introduces three key innovations. First, it employs a task-agnostic, behavior-aware client clustering mechanism that constructs semantic fingerprints using public probe inputs and symmetric Kullback-Leibler (KL) divergence, augmented by prediction-consistency trust scoring and latency-aware edge assignment to jointly mitigate data heterogeneity, device unreliability, and communication constraints. Second, it employs a resource-aware dynamic model splitting strategy to adaptively partition the LLM into three segments across clients and edge servers, with the cloud used only for adapter aggregation, enabling an effective balance between on-device computation cost and global convergence stability. Third, it incorporates a lightweight communication scheme based on computational sketches combined with semantic subspace orthogonal perturbation (SS-OP) to reduce communication overhead while mitigating privacy leakage during model exchanges across the network. Extensive experiments across diverse NLP tasks demonstrate that ELSA consistently outperforms state-of-the-art baselines in terms of adaptability, convergence behavior, and robustness, establishing a scalable and privacy-aware solution for edge-side LLM fine-tuning under resource constraints.

cs.LG↗

arXiv2026

From Simulation to Reality: Practical Deep Reinforcement Learning-based Link Adaptation for Cellular Networks

Link Adaptation (LA) that dynamically adjusts the Modulation and Coding Schemes (MCS) to accommodate time-varying channels is crucial and challenging in cellular networks. Deep reinforcement learning (DRL)-based LA that learns to make decision through the interaction with the environment is a promising approach to improve throughput. However, existing DRL-based LA algorithms are typically evaluated in simplified simulation environments, neglecting practical issues such as ACK/NACK feedback delay, retransmission and parallel hybrid automatic repeat request (HARQ). Moreover, these algorithms overlook the impact of DRL execution latency, which can significantly degrade system performance. To address these challenges, we propose Decoupling-DQN (DC-DQN), a new DRL framework that separates traditional DRL's coupled training and inference processes into two modules based on Deep Q Networks (DQN): a real-time inference module and an out-of-decision-loop training module. Based on this framework, we introduce a novel DRL-based LA algorithm, DC-DQN-LA. The algorithm incorporates practical considerations by designing state, action, and reward functions that account for feedback delays, parallel HARQ, and retransmissions. We implemented a prototype using USRP software-defined radios and srsRAN software. Experimental results demonstrate that DC-DQN-LA improves throughput by 40\% to 70\% in mobile scenario compared with baseline LA algorithms, while maintaining comparable block error rates, and can quickly adapt to environment changes in mobile-to-static scenario. These results highlight the efficiency and practicality of the proposed DRL-based LA algorithm.

cs.NI↗

arXiv2025

Oh-Trust: Overbooking and Hybrid Trading-empowered Resource Scheduling with Smart Reputation Update over Dynamic Edge Networks

Incentive-driven computing resource sharing is crucial for meeting the ever-growing demands of emerging mobile applications. Although conventional spot trading offers a solution, it frequently leads to excessive overhead due to the need for real-time trading related interactions. Likewise, traditional futures trading, which depends on historical data, is susceptible to risks from network dynamics. This paper explores a dynamic and uncertain edge network comprising a computing platform, e.g., an edge server, that offers computing services as resource seller, and various types of mobile users with diverse resource demands as buyers, including fixed buyers (FBs) and uncertain occasional buyers (OBs) with fluctuating needs. To facilitate efficient and timely computing services, we propose an overbooking- and hybrid trading-empowered resource scheduling mechanism with reputation update, termed Oh-Trust. Particularly, our Oh-Trust incentivizes FBs to enter futures trading by signing long-term contracts with the seller, while simultaneously attracting OBs to spot trading, enhancing resource utilization and profitability for both parties. Crucially, to adapt to market fluctuations, a smart reputation updating mechanism is integrated, allowing for the timely renewal of long-term contracts to optimize trading performance. Extensive simulations using real-world datasets demonstrate the effectiveness of Oh-Trust across multiple evaluation metrics.

cs.NI↗

arXiv2025

Quasi-Static IRS: 3D Shaped Beamforming for Area Coverage Enhancement

Intelligent reflecting surface (IRS) is a promising paradigm to reconfigure the wireless environment for enhanced communication coverage and quality. However, to compensate for the double pathloss effect, massive IRS elements are required, raising concerns on the scalability of cost and complexity. This paper introduces a new architecture of quasi-static IRS (QS-IRS), which tunes element phases via mechanical adjustment or manually re-arranging the array topology. QS-IRS relies on massive production/assembly of purely passive elements only, and thus is suitable for ultra low-cost and large-scale deployment to enhance long-term coverage. To achieve this end, an IRS-aided area coverage problem is formulated, which explicitly considers the element radiation pattern (ERP), with the newly introduced shape masks for the mainlobe, and the sidelobe constraints to reduce energy leakage. An alternating optimization (AO) algorithm based on the difference-of-convex (DC) and successive convex approximation (SCA) procedure is proposed, which achieves shaped beamforming with power gains close to that of the joint optimization algorithm, but with significantly reduced computational complexity.

cs.IT↗

arXiv2025

Neural Channel Knowledge Map Assisted Scheduling Optimization of Active IRSs in Multi-User Systems

Intelligent Reflecting Surfaces (IRSs) have potential for significant performance gains in next-generation wireless networks but face key challenges, notably severe double-pathloss and complex multi-user scheduling due to hardware constraints. Active IRSs partially address pathloss but still require efficient scheduling in cell-level multi-IRS multi-user systems, whereby the overhead/delay of channel state acquisition and the scheduling complexity both rise dramatically as the user density and channel dimensions increase. Motivated by these challenges, this paper proposes a novel scheduling framework based on neural Channel Knowledge Map (CKM), designing Transformer-based deep neural networks (DNNs) to predict ergodic spectral efficiency (SE) from historical channel/throughput measurements tagged with user positions. Specifically, two cascaded networks, LPS-Net and SE-Net, are designed to predict link power statistics (LPS) and ergodic SE accurately. We further propose a low-complexity Stable Matching-Iterative Balancing (SM-IB) scheduling algorithm. Numerical evaluations verify that the proposed neural CKM significantly enhances prediction accuracy and computational efficiency, while the SM-IB algorithm effectively achieves near-optimal max-min throughput with greatly reduced complexity.

cs.IT↗

arXiv2025

Accelerating Stable Matching between Workers and Spatial-Temporal Tasks for Dynamic MCS: A Stagewise Service Trading Approach

Designing effective incentive mechanisms in mobile crowdsensing (MCS) networks is crucial for engaging distributed mobile users (workers) to contribute heterogeneous data for various applications (tasks). In this paper, we propose a novel stagewise trading framework to achieve efficient and stable task-worker matching, explicitly accounting for task diversity (e.g., spatio-temporal limitations) and network dynamics inherent in MCS environments. This framework integrates both futures and spot trading stages. In the former, we introduce the \textbf{f}utures \textbf{t}rading-driven \textbf{s}table \textbf{m}atching and \textbf{p}re-\textbf{p}ath-\textbf{p}lanning mechanism (FT-SMP$^3$), which enables long-term task-worker assignment and pre-planning of workers' trajectories based on historical statistics and risk-aware analysis. In the latter, we develop the \textbf{s}pot \textbf{t}rading-driven \textbf{D}QN-based \textbf{p}ath \textbf{p}lanning and onsite \textbf{w}orker \textbf{r}ecruitment mechanism (ST-DP$^2$WR), which dynamically improves the practical utilities of tasks and workers by supporting real-time recruitment and path adjustment. We rigorously prove that the proposed mechanisms satisfy key economic and algorithmic properties, including stability, individual rationality, competitive equilibrium, and weak Pareto optimality. Extensive experiements further validate the effectiveness of our framework in realistic network settings, demonstrating superior performance in terms of service quality, computational efficiency, and decision-making overhead.

cs.DC↗

arXiv2025

Future Resource Bank for ISAC: Achieving Fast and Stable Win-Win Matching for Both Individuals and Coalitions

Future wireless networks must support emerging applications where environmental awareness is as critical as data transmission. Integrated Sensing and Communication (ISAC) enables this vision by allowing base stations (BSs) to allocate bandwidth and power to mobile users (MUs) for communications and cooperative sensing. However, this resource allocation is highly challenging due to: (i) dynamic resource demands from MUs and resource supply from BSs, and (ii) the selfishness of MUs and BSs. To address these challenges, existing solutions rely on either real-time (online) resource trading, which incurs high overhead and failures, or static long-term (offline) resource contracts, which lack flexibility. To overcome these limitations, we propose the Future Resource Bank for ISAC, a hybrid trading framework that integrates offline and online resource allocation through a level-wise client model, where MUs and their coalitions negotiate with BSs. We introduce two mechanisms: (i) Role-Friendly Win-Win Matching (offRFW$^2$M), leveraging overbooking to establish risk-aware, stable contracts, and (ii) Effective Backup Win-Win Matching (onEBW$^2$M), which dynamically reallocates unmet demand and surplus supply. We theoretically prove stability, individual rationality, and weak Pareto optimality of these mechanisms. Through simulations, we show that our framework improves social welfare, latency, and energy efficiency compared to existing methods.

cs.DC↗

arXiv2025

Intelligent Channel Allocation for IEEE 802.11be Multi-Link Operation: When MAB Meets LLM

WiFi networks have achieved remarkable success in enabling seamless communication and data exchange worldwide. The IEEE 802.11be standard, known as WiFi 7, introduces Multi-Link Operation (MLO), a groundbreaking feature that enables devices to establish multiple simultaneous connections across different bands and channels. While MLO promises substantial improvements in network throughput and latency reduction, it presents significant challenges in channel allocation, particularly in dense network environments. Current research has predominantly focused on performance analysis and throughput optimization within static WiFi 7 network configurations. In contrast, this paper addresses the dynamic channel allocation problem in dense WiFi 7 networks with MLO capabilities. We formulate this challenge as a combinatorial optimization problem, leveraging a novel network performance analysis mechanism. Given the inherent lack of prior network information, we model the problem within a Multi-Armed Bandit (MAB) framework to enable online learning of optimal channel allocations. Our proposed Best-Arm Identification-enabled Monte Carlo Tree Search (BAI-MCTS) algorithm includes rigorous theoretical analysis, providing upper bounds for both sample complexity and error probability. To further reduce sample complexity and enhance generalizability across diverse network scenarios, we put forth LLM-BAI-MCTS, an intelligent algorithm for the dynamic channel allocation problem by integrating the Large Language Model (LLM) into the BAI-MCTS algorithm. Numerical results demonstrate that the BAI-MCTS algorithm achieves a convergence rate approximately $50.44\%$ faster than the state-of-the-art algorithms when reaching $98\%$ of the optimal value. Notably, the convergence rate of the LLM-BAI-MCTS algorithm increases by over $63.32\%$ in dense networks.

cs.NI↗

arXiv2025

AFDM Channel Estimation in Multi-Scale Multi-Lag Channels

Affine Frequency Division Multiplexing (AFDM) is a brand new chirp-based multi-carrier (MC) waveform for high mobility communications, with promising advantages over Orthogonal Frequency Division Multiplexing (OFDM) and other MC waveforms. Existing AFDM research focuses on wireless communication at high carrier frequency (CF), which typically considers only Doppler frequency shift (DFS) as a result of mobility, while ignoring the accompanied Doppler time scaling (DTS) on waveform. However, for underwater acoustic (UWA) communication at much lower CF and propagating at speed of sound, the DTS effect could not be ignored and poses significant challenges for channel estimation. This paper analyzes the channel frequency response (CFR) of AFDM under multi-scale multi-lag (MSML) channels, where each propagating path could have different delay and DFS/DTS. Based on the newly derived input-output formula and its characteristics, two new channel estimation methods are proposed, i.e., AFDM with iterative multi-index (AFDM-IMI) estimation under low to moderate DTS, and AFDM with orthogonal matching pursuit (AFDM-OMP) estimation under high DTS. Numerical results confirm the effectiveness of the proposed methods against the original AFDM channel estimation method. Moreover, the resulted AFDM system outperforms OFDM as well as Orthogonal Chirp Division Multiplexing (OCDM) in terms of channel estimation accuracy and bit error rate (BER), which is consistent with our theoretical analysis based on CFR overlap probability (COP), mutual incoherent property (MIP) and channel diversity gain under MSML channels.

cs.IT↗

arXiv2025

Fast Online Movement Optimization of Aerial Base Stations Based on Global Connectivity Map

Aerial base stations (ABSs) mounted on unmanned aerial vehicles (UAVs) are capable of extending wireless connectivity to ground users (GUs) across a variety of scenarios. However, it is an NP-hard problem with exponential complexity in $M$ and $N$, in order to maximize the coverage rate (CR) of $M$ GUs by jointly placing $N$ ABSs with limited coverage range. The complexity of the problem escalates in environments where the signal propagation is obstructed by localized obstacles such as buildings, and is further compounded by the dynamic GU positions. In response to these challenges, this paper focuses on the optimization of a multi-ABS movement problem, aiming to improve the mean CR for mobile GUs within a site-specific environment. Our proposals include 1) introducing the concept of global connectivity map (GCM) which contains the connectivity information between given pairs of ABS/GU locations; 2) partitioning the ABS movement problem into ABS placement sub-problems and formulate each sub-problem into a binary integer linear programming (BILP) problem based on GCM; 3) and proposing a fast online algorithm to execute (one-pass) projected stochastic subgradient descent within the dual space to rapidly solve the BILP problem with near-optimal performance. Numerical results demonstrate that our proposed method achieves a high CR performance close to the upper bound obtained by the open-source solver (SCIP), yet with significantly reduced running time. Moreover, our method also outperforms common benchmarks in the literature such as the K-means initiated evolutionary algorithm or the ones based on deep reinforcement learning (DRL), in terms of CR performance and/or time efficiency.

cs.IT↗

arXiv2025

Integrated Sensing and Communications for Low-Altitude Economy: A Deep Reinforcement Learning Approach

This paper studies an integrated sensing and communications (ISAC) system for low-altitude economy (LAE), where a ground base station (GBS) provides communication and navigation services for authorized unmanned aerial vehicles (UAVs), while sensing the low-altitude airspace to monitor the unauthorized mobile target. The expected communication sum-rate over a given flight period is maximized by jointly optimizing the beamforming at the GBS and UAVs' trajectories, subject to the constraints on the average signal-to-noise ratio requirement for sensing, the flight mission and collision avoidance of UAVs, as well as the maximum transmit power at the GBS. Typically, this is a sequential decision-making problem with the given flight mission. Thus, we transform it to a specific Markov decision process (MDP) model called episode task. Based on this modeling, we propose a novel LAE-oriented ISAC scheme, referred to as Deep LAE-ISAC (DeepLSC), by leveraging the deep reinforcement learning (DRL) technique. In DeepLSC, a reward function and a new action selection policy termed constrained noise-exploration policy are judiciously designed to fulfill various constraints. To enable efficient learning in episode tasks, we develop a hierarchical experience replay mechanism, where the gist is to employ all experiences generated within each episode to jointly train the neural network. Besides, to enhance the convergence speed of DeepLSC, a symmetric experience augmentation mechanism, which simultaneously permutes the indexes of all variables to enrich available experience sets, is proposed. Simulation results demonstrate that compared with benchmarks, DeepLSC yields a higher sum-rate while meeting the preset constraints, achieves faster convergence, and is more robust against different settings.

cs.NI↗