arXiv subjects

Xu Fu

Publications and source records attributed to Xu Fu.

4 recordsLinked to original sources

arXiv2026

Rose: Efficient Embodied RL through Graph Optimization and GPU Compute Resource Control

Embodied reinforcement learning (RL) coordinates policy inference, simulation, and model training with highly heterogeneous resource demands. Although individual components frequently underutilize GPU compute during execution, existing systems only overlap work at coarse component boundaries (e.g., pipelining inference and simulation across simulator groups), leaving substantial low-activity GPU windows within subcomponent phases and nested loops. Exploiting these fine-grained windows manually requires tedious dependency analysis and workload-specific coordination. Furthermore, even when independent work is scheduled concurrently, cross-process GPU time slicing and streaming multiprocessor (SM) contention can negate the benefits of overlap. We present Rose, a runtime-aware system that automates fine-grained overlap and GPU resource management for embodied RL while preserving synchronous training semantics. Through a lightweight programmable interface that exposes execution hierarchy and state accesses, Rose tracks runtime data readiness and GPU activity to perform a two-stage optimization. First, a window-guided graph optimizer identifies low-activity windows and applies graph transformations to overlap complementary work across components and loop levels. Second, leveraging NVIDIA Multi-Process Service (MPS) for cross-process GPU concurrency, a slack-aware resource manager tunes process-level SM quotas by exploiting branch synchronization slack to mitigate interference on the critical path. Both stages combine local profiling with end-to-end iteration validation to guarantee throughput gains with minimal tuning overhead. Across diverse policies, simulators, RL algorithms, and GPU platforms, Rose achieves 1.20--1.92$\times$ training throughput improvement and increases average SM activity by 23.8--39.3\% compared with the state-of-the-art systems.

cs.DC↗

arXiv2026

Learning from Fleet Experience: Collective Experience Learning Across 96 Robots

Robot deployment generates experience that can support policy improvement, but continued reliance on expert corrective actions makes learning difficult to scale across growing fleets. We propose learning from fleet experience: using deployed robots' own executions as a source of supervision for further policy improvement. We call this paradigm Collective Experience Learning (CEL), in which robots pool execution experience to update a shared policy that returns to the fleet. Following expert-supervised initialization, we realize CEL through failure-guided experience retrieval fine-tuning (FERT). FERT uses failure contexts to retrieve relevant segments from successful trajectories and prioritizes their recorded actions for supervised updates, without collecting additional expert demonstrations or corrective actions. Operators provide binary episode-outcome labels and environment resets. A fault-tolerant asynchronous training system (FATS) coordinates robot execution, cloud inference and policy optimization despite local interruptions. Across five real-world manipulation tasks, the resulting policies achieve success rates of 93.3-99.2%, evaluated over 120 trials per task on robots excluded from training data collection. With collection time per robot held fixed, larger fleets provide more experience and generally improve performance across the evaluated sizes. On multi-stage phone packaging, learning with 96 robots raises success from 79.2% to 98.3%. These results support CEL as a scalable approach to improving robot policy reliability through experience generated during deployment.

cs.RO↗

arXiv2025

RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformation

Reinforcement learning (RL) has demonstrated immense potential in advancing artificial general intelligence, agentic intelligence, and embodied intelligence. However, the inherent heterogeneity and dynamicity of RL workflows often lead to low hardware utilization and slow training on existing systems. In this paper, we present RLinf, a high-performance RL training system based on our key observation that the major roadblock to efficient RL training lies in system flexibility. To maximize flexibility and efficiency, RLinf is built atop a novel RL system design paradigm called macro-to-micro flow transformation (M2Flow), which automatically breaks down high-level, easy-to-compose RL workflows at both the temporal and spatial dimensions, and recomposes them into optimized execution flows. Supported by RLinf worker's adaptive communication capability, we devise context switching and elastic pipelining to realize M2Flow transformation, and a profiling-guided scheduling policy to generate optimal execution plans. Extensive evaluations on both reasoning RL and embodied RL tasks demonstrate that RLinf consistently outperforms state-of-the-art systems, achieving $1.07\times-2.43\times$ speedup in end-to-end training throughput.

cs.LG↗

arXiv2025

FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion

Large-scale Mixture-of-Experts (MoE) models rely on \emph{expert parallelism} for efficient training and inference, which splits experts across devices and necessitates distributed data shuffling to route each token to its assigned experts. However, existing communication libraries handle this shuffling poorly; its overhead can account for over half of end-to-end runtime. We present FUSCO, an MoE-friendly communication library that achieves efficient and lightweight data shuffling through fused data transformation and communication, based on the key observation that MoE's expert-major data layout conflicts with the device-major layout expected by communication operations. FUSCO captures the fine-grained data layout, which is then interpreted by a pipelined communication engine that performs the required shuffling efficiently along the communication path. Lightweight planning and load-balancing mechanisms complement the engine by eliminating redundant communication and dispersing traffic. Evaluations on representative benchmarks illustrate that FUSCO achieves up to 3.84$\times$ and 2.01$\times$ speedups over NCCL and DeepEP (the state-of-the-art MoE communication library), respectively. In end-to-end MoE tasks, compared to NCCL and DeepEP, FUSCO reduces the training latency by 1.17-1.39$\times$ and 1.10-1.19$\times$, and lowers the first-token generation latency in inference by 1.09-1.25$\times$ and 1.06-1.16$\times$.

cs.DC↗