Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.26414

RoboMMT: Learning Stable Tracking of Modulated Expressive and Interactive Robot Motion

Till Hielscher·Kai O. Arras
View original source ↗Read source document ↗Export metadata ↓

Abstract

Expressive robot motion can improve naturalness, effectiveness, and perceived likability in human-robot interaction. When deployed on dynamically unstable platforms, however, reactive execution of expressive body motion may bring the robot to its stability limits, especially when expressivity is decoupled from physical control. In this paper, we present RoboMMT (Robot Modulated Motion Tracking), a modular control architecture that couples parameterized expressive motion synthesis, physics-based character control, and reactive adaptation to spatial interaction goals. Our approach trains a reference-conditioned tracking policy across a broad domain of expressivity, enabling it to faithfully realize the commanded motions while maintaining dynamic stability. Goal reactivity is achieved by training lightweight, task-specific interaction policies that modulate the reference generator's spatial goal-attraction point, with the stabilizing tracking policy kept in the loop. Evaluations on a quadruped and two humanoid robots show that the learned tracker remains stable and accurate across the entire modulation domain, whereas trackers trained without this coverage lose balance, tracking accuracy, or both. We show that layering closed-loop goal adaptation over this foundation enables interactive facing and pointing behaviors, while maintaining stable execution and comparable reference-tracking fidelity. The robot learns to proactively redistribute its weight and take corrective steps. Zero-shot sim-to-real transfer is demonstrated through real-robot experiments on a quadruped under heavy disturbances.

Explore related subjects

cs.RO

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Till Hielscher, Kai O. Arras. 2026-10-19. RoboMMT: Learning Stable Tracking of Modulated Expressive and Interactive Robot Motion. https://arxiv.org/abs/2610.26414

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Synergistic Action and Style Priors for Learning Legged Locomotion

Deep Reinforcement Learning (DRL) enables robots to learn complex behaviors through interaction with the environment. However, due to the unrestricted nature of the learning algorithms, the resulting solutions are often brittle and appear unnatural. This is especially true for learning direct joint-level torque control, as inductive biases are difficult to integrate into the learning process. We propose an inductive bias for learning locomotion that is especially useful for torque control: latent actions learned from a small dataset of expert demonstrations. This prior allows the policy to directly leverage knowledge contained in the expert's actions and facilitates more efficient exploration. We observe that the agent is not restricted to the reward levels of the demonstration, and performance in transfer tasks is improved significantly. Latent action priors combined with style rewards for imitation lead to a closer replication of the expert's behavior. Videos are available at https://sites.google.com/view/latent-action-priors.

cs.RO↗
arXiv2026

PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings

Loco-manipulation has recently shown promising capabilities; however, achieving high-precision control, managing the high-dimensional action space induced by many degrees of freedom (DoFs), and fully exploiting the inherent redundancy of whole-body systems remain challenging. In this paper, we propose a novel whole-body control framework that effectively addresses these challenges by decomposing the complex loco-manipulation problem into partial reference motion generation and low-level imitation control. We introduce a new Kinematic Normalizing Flow (KNF) model, trained on a large-scale kinematic dataset, that generates diverse partial kinematic references. A high-level controller is then trained to navigate the KNF's latent space to exploit redundant solutions, while a low-level controller learns dynamically consistent motion execution. We validate our approach on the quadrupedal robot equipped with a six-DoF robotic arm. In simulation, PAKE outperforms the evaluated baselines in tracking accuracy and feasible workspace coverage on the 4,000-target benchmark. For hardware deployment, we evaluate the system over 24 episodes across 8 different mobile loco-manipulation tasks. The system achieves end-effector pose-tracking errors of 4.5 cm and 0.14 rad, while maintaining accurate locomotion tracking with linear and angular velocity errors of 0.1 m/s and 0.01 rad/s, respectively. Our method represents a practical and powerful solution for accurate and generalized whole-body loco-manipulation in high-DoF robotic systems, with promising potential for diverse downstream robotic tasks.

cs.RO↗
arXiv2026

Learning Where it Matters: Autonomous Data Collection and Offline Reinforcement Learning for Local Contact-Rich Manipulation

Learned policies trained end-to-end on large datasets often remain brittle in high-precision contact-rich tasks. Further, they typically require large amounts of manually collected data and still struggle to generalize to scenes outside of the training scenarios. Classical motion planning, in turn, generalizes well but lacks the precision and contact awareness for tight-tolerance tasks. To reliably solve challenging contact-rich tasks, we propose to leverage robot data and learning where it matters in a modular approach. First, we introduce an autonomous data collection scheme for the real world that collects dense data locally in the contact-rich segment of the task, eliminating reliance on a teleoperator's skill. Second, we show that using offline RL, the collected data can be translated into policies with high success rates. We then integrate our local policies with traditional motion planning to solve long-horizon assembly tasks. Across four real-world tasks, using only 2-2.5 h of autonomous data collection, we achieve an average success rate of 96% for long-horizon tasks, compared to 55% for a planning-only baseline and 37% for the strongest end-to-end baseline. Our local policies also reduce applied forces by 49%. Notably, performance remains at 90% for scene-level out-of-distribution scenarios, while end-to-end approaches struggle. In total, we run over 1500 real-world evaluation rollouts. Our results pave the way for targeted data collection in contact-rich tasks and for high success rates in precision applications. Videos and qualitative results are available at https://learnsyslab.github.io/data_and_learning_where_it_matters/

cs.RO↗