AgenticRL: Agentic Reinforcement Learning with Self-Refinement for Complex UAV Navigation

Abstract

Deep reinforcement learning enables autonomous robots to learn complex navigation tasks, but still relies heavily on time consuming manual reward design and fine tuning. Existing automated reward generation and refinement methods reduce this effort, yet often lack task-level behavioral diagnosis for directing subsequent reward revisions. We introduce AgenticRL, a multimodal closed loop framework in which role-specialized agents generate executable rewards, diagnose failures of the resulting policies, formulate targeted refinement instructions, and regenerate improved rewards. Before training, a task grounding stage automatically selects a compatible action profile, together with its observation and reward interfaces. Each generated reward is used to train a policy using Proximal Policy Optimization (PPO), which is subsequently evaluated under randomized conditions. Task-level behavioral, geometric, and safety measurements are organized into a structured diagnosis packet and jointly analyzed with the current reward code, task specification, behavioral summary, and visual scene context. Unlike one-shot reward generation, human-guided refinement, or broad candidate search, AgenticRL uses automated diagnosis of the behavior induced by a reward to direct its next revision. We evaluate the framework across eight UAV tasks covering navigation, obstacle interaction, trajectory tracking, agile manoeuvres, and cluttered flight. Under the reported comparative evaluation, AgenticRL achieves success rates of 100% in racing and 88% in cluttered navigation, exceeding the strongest Eureka and Text2Reward baselines, respectively. Reward refinement increases mean simulation success from 37.2% to 96.4%, while the resulting policies achieve a collective real-world success rate of 90.0% and a sim-to-real accuracy of 93.4%.