Hybrid PPO-Based Multi-Objective Optimization for UAV-mounted STAR-RIS with Energy Harvesting
This paper investigates a multi-objective optimiza tion framework for UAV-mounted reconfigurable intelligent sur face (STAR-RIS)-assisted Sixth Generation (6G)-enabled Internet of Things (IoT) systems. The considered framework jointly supports communication enhancement and radio-frequency (RF) energy harvesting (EH) through adaptive STAR-RIS transmission and reflection control. A weighted multi-objective optimization problem is formulated to jointly optimize the base station (BS) beamforming vectors, STAR-RIS transmission and reflection coefficients, RF power-splitting factor, and UAV trajectory under quality-of-service (QoS), power, and mobility constraints. Due to the strong coupling among the communications, EH, and UAV mobility variables, the resulting problem is highly non convex and difficult to solve using conventional optimization methods. To address this challenge, a hybrid proximal policy optimization (H-PPO) framework is proposed. The developed hybrid actor-critic architecture combines a categorical policy for discrete UAV movement decisions with Beta-distributed policies for continuous communication-control variables. In addition, a two-stage reward mechanism is introduced to enforce QoS feasibility while balancing system sum rate, energy efficiency, and harvested energy. Simulation results demonstrate that the proposed H-PPO framework achieves faster convergence and more stable training behavior than proximal policy optimization (PPO) and twin delayed deep deterministic policy gradient (TD3) baselines. In particular, H-PPO achieves approximately 66% and 60% faster convergence than PPO and TD3, respectively, while maintaining more stable training behavior. Furthermore, the proposed framework effectively balances communication performance and EH under different QoS requirements, STAR RIS sizes, transmit power levels, and user densities.