arXiv subjects

Chendi Qu

Publications and source records attributed to Chendi Qu.

10 recordsLinked to original sources

arXiv2026

Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies

Generalist robot policies increasingly benefit from large-scale pretraining, but offline data alone is insufficient for robust real-world deployment. Deployed robots encounter distribution shifts, long-tail failures, task variations, and human correction opportunities that fixed demonstration datasets cannot fully capture. We present Learning While Deploying (LWD), a fleet-scale offline-to-online reinforcement learning framework for continual post-training of generalist Vision-Language-Action (VLA) policies. Starting from a pretrained VLA policy, LWD closes the loop between deployment, shared physical experience, policy improvement, and redeployment by using autonomous rollouts and human interventions collected across a robot fleet. To stabilize learning from heterogeneous, sparse-reward fleet data, LWD combines Distributional Implicit Value Learning (DIVL) for robust value estimation with Q-learning via Adjoint Matching (QAM) for policy extraction in flow-based VLA action generators. We validate LWD on a fleet of 16 dual-arm robots across eight real-world manipulation tasks, including semantic grocery restocking and 3--5 minute long-horizon tasks. A single generalist policy improves as fleet experience accumulates, reaching an average success rate of 95%, with the largest gains on long-horizon tasks.

cs.RO↗

arXiv2026

SOP: A Scalable Online Post-Training System for Vision-Language-Action Models

Vision-language-action (VLA) models achieve strong generalization through large-scale pre-training, but real-world deployment requires expert-level task proficiency in addition to broad generality. Existing post-training approaches for VLA models are typically offline, single-robot, or task-specific, limiting effective on-policy adaptation and scalable learning from real-world interaction. We introduce a Scalable Online Post-training (SOP) system that enables online, distributed, multi-task post-training of generalist VLA models directly in the physical world. SOP tightly couples execution and learning through a closed-loop architecture in which a fleet of robots continuously streams on-policy experience and human intervention signals to a centralized cloud learner, and asynchronously receives updated policies. This design supports prompt on-policy correction, scales experience collection through parallel deployment, and preserves generality during adaptation. SOP is agnostic to the choice of post-training algorithm; we instantiate it with both interactive imitation learning (HG-DAgger) and reinforcement learning (RECAP). Across a range of real-world manipulation tasks including cloth folding, box assembly, and grocery restocking, we show that SOP substantially improves the performance of large pretrained VLA models while maintaining a single shared policy across tasks. Effective post-training can be achieved within hours of real-world interaction, and performance scales near-linearly with the number of robots in the fleet. These results suggest that tightly coupling online learning with fleet-scale deployment is instrumental to enabling efficient, reliable, and scalable post-training of generalist robot policies in the physical world.

cs.RO↗

arXiv2025

Optimal Unpredictable Control for Linear Systems

In this paper, we investigate how to achieve the unpredictability against malicious inferences for linear systems. The key idea is to add stochastic control inputs, named as unpredictable control, to make the outputs irregular. The future outputs thus become unpredictable and the performance of inferences is degraded. The major challenges lie in: i) how to formulate optimization problems to obtain an optimal distribution of stochastic input, under unknown prediction accuracy of the adversary; and ii) how to achieve the trade-off between the unpredictability and control performance. We first utilize both variance and confidence probability of prediction error to quantify unpredictability, then formulate two two-stage stochastic optimization problems, respectively. Under variance metric, the analytic optimal distribution of control input is provided. With probability metric, it is a non-convex optimization problem, thus we present a novel numerical method and convert the problem into a solvable linear optimization problem. Last, we quantify the control performance under unpredictable control, and accordingly design the unpredictable LQR and cooperative control. Simulations demonstrate the unpredictability of our control algorithm. The obtained optimal distribution outperforms Gaussian and Laplace distributions commonly used in differential privacy under proposed metrics.

eess.SY↗

arXiv2025

IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment

Large Language Models (LLMs) have achieved impressive performance through Supervised Fine-tuning (SFT) on diverse instructional datasets. When training on multiple capabilities simultaneously, the mixture training dataset, governed by volumes of data from different domains, is a critical factor that directly impacts the final model's performance. Unlike many studies that focus on enhancing the quality of training datasets through data selection methods, few works explore the intricate relationship between the compositional quantity of mixture training datasets and the emergent capabilities of LLMs. Given the availability of a high-quality multi-domain training dataset, understanding the impact of data from each domain on the model's overall capabilities is crucial for preparing SFT data and training a well-balanced model that performs effectively across diverse domains. In this work, we introduce IDEAL, an innovative data equilibrium adaptation framework designed to effectively optimize volumes of data from different domains within mixture SFT datasets, thereby enhancing the model's alignment and performance across multiple capabilities. IDEAL employs a gradient-based approach to iteratively refine the training data distribution, dynamically adjusting the volumes of domain-specific data based on their impact on downstream task performance. By leveraging this adaptive mechanism, IDEAL ensures a balanced dataset composition, enabling the model to achieve robust generalization and consistent proficiency across diverse tasks. Experiments across different capabilities demonstrate that IDEAL outperforms conventional uniform data allocation strategies, achieving a comprehensive improvement of approximately 7% in multi-task evaluation scores.

cs.AI↗

arXiv2024

3DIOC: Direct Data-Driven Inverse Optimal Control for LTI Systems

This paper addresses the Direct Data-Driven Inverse Optimal Control (3DIOC) problem for linear time-invariant (LTI) systems under the linear quadratic (LQ) control. Unlike traditional approaches that require system identification, the proposed method learns the underlying objective function directly from measured input-output trajectories. Leveraging the input-output representation of LTI systems via the Fundamental Lemma, we derive a model-free optimality necessary condition (ONC) for the forward LQ problem, which forms the basis for formulating and solving an inverse optimal control problem. We also provide an identifiability condition to ensure the uniqueness of the inverse solution. While the ONC-based IOC approach is effective in the noise-free case, its performance is not promising when the data is corrupted with noises. We then reformulate the 3DIOC as a bi-level optimization problem, which is solved using iterative gradient descent and offers solution guarantees. Furthermore, we analyze the relationship between the solution sets of the two proposed formulations, providing practical insights into their selection. The simulation results validate the effectiveness and performance of our proposed methods.

math.OC↗

arXiv2023

Model Selection for Inverse Reinforcement Learning via Structural Risk Minimization

Inverse reinforcement learning (IRL) usually assumes the reward function model is pre-specified as a weighted sum of features and estimates the weighting parameters only. However, how to select features and determine a proper reward model is nontrivial and experience-dependent. A simplistic model is less likely to contain the ideal reward function, while a model with high complexity leads to substantial computation cost and potential overfitting. This paper addresses this trade-off in the model selection for IRL problems by introducing the structural risk minimization (SRM) framework from statistical learning. SRM selects an optimal reward function class from a hypothesis set minimizing both estimation error and model complexity. To formulate an SRM scheme for IRL, we estimate the policy gradient from given demonstration as the empirical risk, and establish the upper bound of Rademacher complexity as the model penalty of hypothesis function classes. The SRM learning guarantee is further presented. In particular, we provide the explicit form for the linear weighted sum setting. Simulations demonstrate the performance and efficiency of our algorithm.

cs.LG↗

arXiv2023

Observation-based Optimal Control Law Learning with LQR Reconstruction

Designing controllers to generate various trajectories has been studied for years, while recently, recovering an optimal controller from trajectories receives increasing attention. In this paper, we reveal that the inherent linear quadratic regulator (LQR) problem of a moving agent can be reconstructed based on its trajectory observations only, which enables one to learn the optimal control law of the agent autonomously. Specifically, the reconstruction of the optimization problem requires estimation of three unknown parameters including the target state, weighting matrices in the objective function and the control horizon. Our algorithm considers two types of objective function settings and identifies the weighting matrices with proposed novel inverse optimal control methods, providing the well-posedness and identifiability proof. We obtain the optimal estimate of the control horizon using binary search and finally reconstruct the LQR problem with above estimates. The strength of learning control law with optimization problem recovery lies in less computation consumption and strong generalization ability. We apply our algorithm to the future control input prediction and the discrepancy loss is further derived. Numerical simulations and hardware experiments on a self-designed robot platform illustrate the effectiveness of our work.

eess.SY↗

arXiv2023

Control Input Inference of Mobile Agents under Unknown Objective

Trajectory and control secrecy is an important issue in robotics security. This paper proposes a novel algorithm for the control input inference of a mobile agent without knowing its control objective. Specifically, the algorithm first estimates the target state by applying external perturbations. Then we identify the objective function based on the inverse optimal control, providing the well-posedness proof and the identifiability analysis. Next, we obtain the optimal estimate of the control horizon using binary search. Finally, the agent's control optimization problem is reconstructed and solved to predict its input. Simulation illustrates the efficiency and the performance of the algorithm.

cs.RO↗

arXiv2022

Multi-period Optimal Control for Mobile Agents Considering State Unpredictability

The optimal control for mobile agents is an important and challenging issue. Recent work shows that using randomized mechanism in agents' control can make the state unpredictable, and thus improve the security of agents. However, the unpredictable design is only considered in single period, which can lead to intolerable control performance in long time horizon. This paper aims at the trade-off between the control performance and state unpredictability of mobile agents in long time horizon. Utilizing random perturbations consistent with uniform distributions to maximize the attackers' prediction errors of future states, we formulate the problem as a multi-period convex stochastic optimization problem and solve it through dynamic programming. Specifically, we design the optimal control strategy considering both unconstrained and input constrained systems. The analytical iterative expressions of the control are further provided. Simulation illustrates that the algorithm increases the prediction errors under Kalman filter while achieving the control performance requirements successfully.

eess.SY↗

arXiv2022

Moving Target Interception Considering Dynamic Environment

The interception of moving targets is a widely studied issue. In this paper, we propose an algorithm of intercepting the moving target with a wheeled mobile robot in a dynamic environment. We first predict the future position of the target through polynomial fitting. The algorithm then generates an interception trajectory with path and speed decoupling. We use Hybrid A* search to plan a path and optimize it via gradient decent method. To avoid the dynamic obstacles in the environment, we introduce ST graph for speed planning. The speed curve is represented by piecewise B\'ezier curves for further optimization. Compared with other interception algorithms, we consider a dynamic environment and plan a safety trajectory which satisfies the kinematic characteristics of the wheeled robot while ensuring the accuracy of interception. Simulation illustrates that the algorithm successfully achieves the interception tasks and has high computational efficiency.

cs.RO↗