arXiv subjects

Lu Shi

Publications and source records attributed to Lu Shi.

At least 19 recordsLinked to original sources

arXiv2026

Why Koopman Structure Does Not Help Differentiable World Models for Humanoid Whole-Body Tracking

We test whether Koopman structure, linear dynamics in a lifted state space, helps when a humanoid whole-body tracking policy is trained by backpropagating through a differentiable world model, replacing the MLP world model of an otherwise unchanged pipeline for simulated Unitree G1 tracking. It does not: return is at best equal (0.823 versus 0.830), one-step error on identical data is 1.3 to 1.6 times the MLP's, and training is about 15% slower. Lifted rollouts diverge or fail to train a policy; of the schemes tested, re-lifting at every imagined step trains best. Re-lifting never uses the lifted linear dynamics and leaves a model affine in the action, with state-dependent offset and slope; an action-affine MLP also predicts worse, so the cost belongs to this class. At a fixed state, the curvature of the true dynamics in the action explains only part of it. Across states, the policy's actions span a curved region, so an affine model fits policy data only as the tangent at the policy's action, the operating point: on G1, the MLP's tangent there is as accurate as the MLP; at the mean action, 31 times worse. The offset must encode, from the state alone, where the policy acts, whereas an MLP reads the action. Supplying the policy's action removes most of the remaining gap on G1, Ant and Humanoid. The only online repair approaching the MLP feeds the action into the nonlinear lift, giving up the structure. The MLP world model thus remains the better choice here.

cs.RO↗

arXiv2026

USDCraft: Geometrically Grounded Programmatic Modeling of Articulated 3D Assets for Simulation

Geometrically faithful and functional articulated 3D assets are essential for real-to-sim robot manipulation, where policies trained in simulation must transfer to physical objects. Recent mesh-based methods learn to infer articulation from annotated 3D assets, but deployment remains challenging when real-world objects fall outside the training distribution or their meshes are incomplete or corrupted. To address these limitations, we formulate articulated asset reconstruction as programmatic modeling grounded in partial geometric evidence and introduce USDCraft, a framework in which a pretrained LLM writes and revises executable programs for simulation-ready articulated assets without task-specific training. We propose source geometry analysis, which converts the source mesh into a metric textual description that distinguishes observed surface from unknown space, and iterative geometric rechecking, which re-encodes each candidate in the same representation so that discrepancies point to program edits while unobserved regions remain open to completion. Visual feedback and physical authoring guidance complete the modeling process, which produces articulated USD assets with explicit physical properties that load into Isaac Sim without manual adjustment. Experiments demonstrate leading articulation recovery on two benchmarks and validate USDCraft's effectiveness for real-to-sim-to-real robot manipulation.

cs.RO↗

arXiv2026

An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer

The sim-to-real gap remains a critical challenge in robotics, hindering the deployment of algorithms trained in simulation to real-world systems. We propose a flexible Real-to-Sim-to-Real (RSR) framework whose central contribution is an information-theoretic cost function that explicitly accounts for sim-to-real discrepancies. This cost balances two objectives, completing the task and steering the policy to collect real-world samples that are maximally informative for improving transfer. It can be integrated seamlessly into existing reinforcement learning algorithms (e.g., PPO, SAC) and ensures a balanced exploration of critical regions in the real domain. The framework treats differentiable simulation as optional: when a differentiable simulator is available, the collected informative data can also be used to tune simulator parameters. We implement the framework with the MuJoCo MJX platform and demonstrate its generality by evaluating on both manipulation tasks with a 6-DOF robotic arm and locomotion tasks on a legged robot. Empirical results show that our RSR loop yields more efficient data acquisition and substantially improves task performance in real-world that achieves a smoother sim-to-real transfer.

cs.RO↗

arXiv2026

EBRL: Asynchronous Embodied RL by Multi-Grained Resource Management

Embodied reinforcement learning (RL) improves model capabilities with a pipeline of environment simulation, action generation, and model updates. These stages show heterogeneous CPU and GPU demands, making efficient resource utilization difficult. Recent systems overlap rollout (simulation and generation) with training for efficiency, but exclusive GPU allocation and synchronized barrier in rollout still leave substantial hardware resource waste. In this paper, we present EBRL, an asynchronous embodied RL training system with two core techniques. The asynchronous pipelined scheduler overlaps rollout and training, pipelines simulation and generation across environment groups, and carries out each environment independently, eliminating synchronization stalls. The fine-grained resource manager pools CPU cores and GPU streaming multiprocessors, and uses stage profiles and runtime feedback to adjust resource quotas and batch sizes to meet the shifting demands among stages. We implement EBRL on RLinf and evaluate it with four embodied policies and four simulation benchmarks across heterogeneous GPU testbeds. Experiments show that EBRL achieves 1.30-3.47 times the end-to-end rollout throughput and 2.5 times of training convergency compared to the SOTA embodied RL systems.

cs.LG↗

arXiv2026

Covariance-Regulated Recursive Koopman Learning for Nonlinear Systems with Uncertain Time-Varying Dynamics

Offline models for autonomous robots often fail under time-varying dynamics outside their training distribution. While Koopman operator theory offers a linear representation of nonlinear dynamics via lifting, its transition to online recursive estimation may suffer numerical vulnerabilities: covariance windup under low excitation when using exponential forgetting, and vanishing gain without forgetting. To prevent covariance explosion and parameter freezing, this paper introduces a unified Covariance-Regulated Recursive Koopman Learning (CR-RKL) framework that systematically incorporates and evaluates several covariance-regulated methods. Specifically, we propose a novel error dead-zone covariance skipping strategy, along with adaptations of constant-trace normalization and directional forgetting to the Koopman-specific setting. Validated on a non-holonomic differential-drive robot with wheel slip and Stribeck friction, as well as on a two-winged, tailless flapping-wing micro aerial vehicle, CR- RKL achieves numerically stable and accurate online modeling relative to its offline and vanilla recursive counterparts. When embedded in model predictive control, it maintains reliable tracking performance under uncertain, time-varying dynamics with bounded dictionaries. However, we also identify a structural limitation: while covariance-regulated methods achieve low off-policy prediction error, pairing them with unbounded dictionaries can lead to control failure.

cs.RO↗

arXiv2026

GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning

Embodied AI research is undergoing a shift toward vision-centric perceptual paradigms. While massively parallel simulators have catalyzed breakthroughs in proprioception-based locomotion, their potential remains largely untapped for vision-informed tasks due to the prohibitive computational overhead of large-scale photorealistic rendering. Furthermore, the creation of simulation-ready 3D assets heavily relies on labor-intensive manual modeling, while the significant sim-to-real physical gap hinders the transfer of contact-rich manipulation policies. To address these bottlenecks, we propose GS-Playground, a multi-modal simulation framework designed to accelerate end-to-end perceptual learning. We develop a novel high-performance parallel physics engine, specifically designed to integrate with a batch 3D Gaussian Splatting (3DGS) rendering pipeline to ensure high-fidelity synchronization. Our system achieves a breakthrough throughput of 10^4 FPS at 640x480 resolution, significantly lowering the barrier for large-scale visual RL. Additionally, we introduce an automated Real2Sim workflow that reconstructs photorealistic, physically consistent, and memory-efficient environments, streamlining the generation of complex simulation-ready scenes. Extensive experiments on locomotion, navigation, and manipulation demonstrate that GS-Playground effectively bridges the perceptual and physical gaps across diverse embodied tasks. Project homepage: https://gsplayground.github.io.

cs.RO↗

arXiv2026

Try Once, Then Optimal: De-Redundified Procedure Memory for Cross-Episode Exploration Amortization

Manipulating objects with hidden internal state, such as a latched microwave, forces a robot to probe before it can act. Yet a robot that has solved an instance once re-runs the same probes whenever it encounters that instance again, because existing cross-episode memories target task success and organize reuse around states, not the object or the cost of re-exploring it. We present Instance-Oriented Memory (IOM), an object-centric framework that amortizes this exploration: from a single encounter that uncovers the hidden state, whether or not it succeeds, IOM records a short procedure for manipulating that instance, keys it on the object's identifiable features, and injects it as a soft bias on a procedure-conditioned policy. A later encounter recognizes the object and recalls its procedure instead of re-exploring. We instantiate this distillation with an off-the-shelf vision-language model (VLM) that parses each encounter into the procedure without task-specific training. Across four articulated-object tasks, two in simulation (microwave, door) and two on a real robot (bottle, cabinet), an oracle procedure memory cuts manipulation operations by 16-30% over re-exploration at non-regressing success, and the VLM instantiation recovers 69-88% of that saving out of the box. Because the procedure is a soft bias on a feedback-driven policy, an incorrect memory is recovered from rather than obeyed: success holds even when a retrieved procedure is wrong, as for $\approx$12% of door instances. Across all tasks the benefit is purely one of efficiency: success never regresses, and on the real robot even improves. Code will be released upon acceptance.

cs.RO↗

arXiv2026

Atomically Thin Amorphous Carbon with an Ultralow Dielectric Constant

Two-dimensional (2D) materials exhibit excellent properties at monolayer thickness and are viable replacements for various microelectronic components as scaling gradually approaches the atomic limit. Despite significant advancements in the ongoing 2D revolution of integrated circuits, one crucial building block, namely a 2D ultralow-k (ULK) dielectric, remains unreported. The challenge lies in achieving a dielectric constant less than 3, as traditional low-k dielectrics are inherently unstable at the 2D limit due to their amorphous or porous nature. The realisation of ultrathin dielectrics with low-k is also needed to address current bottlenecks in integrated circuits scaling. Specifically, low-k materials are necessary to minimise parasitic capacitances as the distance between conductive elements shrinks below 10 nm. Moreover, advanced architectures like gate-all-around field effect transistors (GAA FET) require even lower dielectric constants (k<2) at sub-3nm thickness. Here, we show that layer-by-layer grown multilayer amorphous carbon (ML-AC), as thin as 0.8 nm, is a mechanically robust 2D ULK dielectric with k of 1.35 and dielectric strength of 28-31 MV cm-1. The lack of any long-range order, its intrinsic 2D nature, sp2 carbon character and low density are all essential for minimising dielectric permittivity. Moreover, ML-AC overcomes the vulnerability of existing dielectrics to ion diffusion degradation with a record metal ion diffusion time to failure (TTF) of 10^10 s for even a single layer. Therefore, otherwise necessary additional layers occupying up to 3 nm can be eliminated, which is especially significant as metal line widths approach 10 nm. Combined with its low-temperature, direct and conformal growth even on a dielectric, these critical features enable substantial improvements in silicon-based semiconductor electronics and ensure compatibility with future 2D electronics.

cond-mat.mtrl-sci↗

arXiv2026

FILIC: Dual-Loop Force-Guided Imitation Learning with Impedance Torque Control for Contact-Rich Manipulation Tasks

Many contact-rich manipulation tasks require precise force regulation. However, most imitation learning (IL) policies remain position-centric and lack explicit force awareness, and adding force/torque sensors to collaborative robot arms is often costly and requires additional hardware design. To overcome these issues, we propose FILIC, a Force-guided Imitation Learning framework with impedance torque control. FILIC integrates a Transformer-based IL policy with an impedance controller in a dual-loop structure, enabling compliant force-informed, force-executed manipulation. For robots without force/torque sensors, we introduce a cost-effective end-effector force estimator using joint torque measurements through analytical Jacobian-based inversion while compensating with model-predicted torques from a digital twin. Experiments show that FILIC significantly outperforms vision-only and joint-torque-based methods, achieving safer, more compliant, and adaptable contact-rich manipulation. The source code is available at https://github.com/OpenGHz/mujoco-wrench-estimator.git.

cs.RO↗

arXiv2026

When Video Misreads: Closed-Loop Distillation of Reading Heuristics for Exploratory Manipulation Trace QA

Exploratory manipulation often turns an apparent failed attempt into the key evidence for what to do next. For example, a robot pulls a locked cabinet drawer, fails, and only succeeds after opening the lock. The failed pull reveals a latent precondition (the drawer is locked) that determines the minimal-success action chain (the fewest actions that complete the task), here [lock-open, drawer-pull]. Correctly reading this trace is therefore the prerequisite for recovering that chain. We formalize this setting as Exploratory Manipulation Trace QA (EMT-QA): given synchronized video and proprioception from an exploratory trace, predict the minimal-success action chain under the latent precondition revealed by the probe. However, even state-of-the-art VLMs and embodied multimodal LLMs misread this evidence: they do not reliably recover the chain from raw video, raw proprioception, or their combination. We introduce Closed-Loop Trace Distillation, a pipeline that uses a per-task coding agent to inspect labeled training traces and distill a one-line natural-language prompt over the trace, which we call the Distilled Reading Heuristic (DRH). At inference, no agent is invoked and no model weights are updated; a frozen VLM receives the raw trace plus the DRH as a prompt entry. Across three simulator and two real-robot tasks, the DRH improves chain accuracy by +0.38 to +0.47 over the best raw-modality baseline. The same DRH also serves as the sole specification for one-shot programmatic classifiers that match the prompted VLM.

cs.RO↗

arXiv2026

Dual-Thresholded Heatmap-Guided Proposal Clustering and Negative Certainty Supervision with Enhanced Base Network for Weakly Supervised Object Detection

Weakly supervised object detection (WSOD) has attracted significant attention in recent years, as it does not require box-level annotations. State-of-the-art methods generally adopt a multi-module network, which employs WSDDN as the multiple instance detection network module and uses multiple instance refinement modules to refine performance. However, these approaches suffer from three key limitations. First, existing methods tend to generate pseudo GT boxes that either focus only on discriminative parts, failing to capture the whole object, or cover the entire object but fail to distinguish between adjacent intra-class instances. Second, the foundational WSDDN architecture lacks a crucial background class representation for each proposal and exhibits a large semantic gap between its branches. Third, prior methods discard ignored proposals during optimization, leading to slow convergence. To address these challenges, we propose the Dual-thresholded heAtmap-guided proposal clustering and Negative Certainty supervision with Enhanced base network (DANCE) method for WSOD. Specifically, we first devise a heatmap-guided proposal selector (HGPS) algorithm, which utilizes dual thresholds on heatmaps to pre-select proposals, enabling pseudo GT boxes to both capture the full object extent and distinguish between adjacent intra-class instances. We then construct a weakly supervised basic detection network (WSBDN), which augments each proposal with a background class representation and uses heatmaps for pre-supervision to bridge the semantic gap between matrices. At last, we introduce a negative certainty supervision (NCS) loss on ignored proposals to accelerate convergence. Extensive experiments on the challenging PASCAL VOC and MS COCO datasets demonstrate the effectiveness and superiority of our method. Our code is publicly available at https://github.com/gyl2565309278/DANCE.

cs.CV↗

arXiv2026

Koopman Identification of Nonlinear Systems via Reservoir Liftings

Learning tractable linear representations of nonlinear dynamical systems via Koopman operator theory is often hindered by dictionary selection, temporal memory encoding, and numerical ill-conditioning. Inspired by Reservoir Computing (RC) paradigm, this paper introduces the RC-Koopman framework, which interprets reservoir as a stateful, finite-dimensional Koopman dictionary whose temporal depth is explicitly controlled by its spectral radius. We show that the Echo State Property (ESP) guarantees well-posedness and favorable numerical conditioning of the lifted Koopman approximation. A correlation-based spectral radius selection algorithm aligns reservoir memory with dominant system timescales. Analysis reveals how the finite memory of the reservoir determines which Koopman eigenfunctions remain observable from the lifted features. Evaluation on synthetic benchmarks demonstrates that RC-Koopman achieves a favorable balance between reconstruction accuracy of the underlying nonlinear dynamics and dynamical stability, compared to Extended Dynamic Mode Decomposition (EDMD) and Hankel-based lifting approaches. Code available at: https://github.com/NEAR-the-future/RC-Koopman.git

cs.LG↗

arXiv2026

Not all tokens contribute equally to diffusion learning

With the rapid development of conditional diffusion models, significant progress has been made in text-to-video generation. However, we observe that these models often neglect semantically important tokens during inference, leading to biased or incomplete generations under classifier-free guidance. We attribute this issue to two key factors: distributional bias caused by the long-tailed token frequency in training data, and spatial misalignment in cross-attention where semantically important tokens are overshadowed by less informative ones. To address these issues, we propose Distribution-Aware Rectification and Spatial Ensemble (DARE), a unified framework that improves semantic guidance in diffusion models from the perspectives of distributional debiasing and spatial consistency. First, we introduce Distribution-Rectified Classifier-Free Guidance (DR-CFG), which regularizes the training process by dynamically suppressing dominant tokens with low semantic density, encouraging the model to better capture underrepresented semantic cues and learn a more balanced conditional distribution. This design mitigates the risk of the model distribution overfitting to tokens with low semantic density. Second, we propose Spatial Representation Alignment (SRA), which adaptively reweights cross-attention maps according to token importance and enforces representation consistency, enabling semantically important tokens to exert stronger spatial guidance during generation. This mechanism effectively prevents low semantic-density tokens from dominating the attention allocation, thereby avoiding the dilution of the spatial and distributional guidance provided by high semantic-density tokens. Extensive experiments on multiple benchmark datasets demonstrate that DARE consistently improves generation fidelity and semantic alignment, achieving significant gains over existing approaches.

cs.CV↗

arXiv2026

ScDiVa: Masked Discrete Diffusion for Joint Modeling of Single-Cell Identity and Expression

Single-cell RNA-seq profiles are high-dimensional, sparse, and unordered, causing autoregressive generation to impose an artificial ordering bias and suffer from error accumulation. To address this, we propose scDiVa, a masked discrete diffusion foundation model that aligns generation with the dropout-like corruption process by defining a continuous-time forward masking mechanism in token space. ScDiVa features a bidirectional denoiser that jointly models discrete gene identities and continuous values, utilizing entropy-normalized serialization and a latent anchor token to maximize information efficiency and preserve global cell identity. The model is trained via depth-invariant time sampling and a dual denoising objective to simulate varying sparsity levels while ensuring precise recovery of both identity and magnitude. Pre-trained on 59 million cells, scDiVa achieves strong transfer performance across major benchmarks, including batch integration, cell type annotation, and perturbation response prediction. These results suggest that masked discrete diffusion serves as a biologically coherent and effective alternative to autoregression.

cs.LG↗

arXiv2025

Condition Weaving Meets Expert Modulation: Towards Universal and Controllable Image Generation

The image-to-image generation task aims to produce controllable images by leveraging conditional inputs and prompt instructions. However, existing methods often train separate control branches for each type of condition, leading to redundant model structures and inefficient use of computational resources. To address this, we propose a Unified image-to-image Generation (UniGen) framework that supports diverse conditional inputs while enhancing generation efficiency and expressiveness. Specifically, to tackle the widely existing parameter redundancy and computational inefficiency in controllable conditional generation architectures, we propose the Condition Modulated Expert (CoMoE) module. This module aggregates semantically similar patch features and assigns them to dedicated expert modules for visual representation and conditional modeling. By enabling independent modeling of foreground features under different conditions, CoMoE effectively mitigates feature entanglement and redundant computation in multi-condition scenarios. Furthermore, to bridge the information gap between the backbone and control branches, we propose WeaveNet, a dynamic, snake-like connection mechanism that enables effective interaction between global text-level control from the backbone and fine-grained control from conditional branches. Extensive experiments on the Subjects-200K and MultiGen-20M datasets across various conditional image generation tasks demonstrate that our method consistently achieves state-of-the-art performance, validating its advantages in both versatility and effectiveness. The code has been uploaded to https://github.com/gavin-gqzhang/UniGen.

cs.CV↗

arXiv2025

DISCOVERSE: Efficient Robot Simulation in Complex High-Fidelity Environments

We present the first unified, modular, open-source 3DGS-based simulation framework for Real2Sim2Real robot learning. It features a holistic Real2Sim pipeline that synthesizes hyper-realistic geometry and appearance of complex real-world scenarios, paving the way for analyzing and bridging the Sim2Real gap. Powered by Gaussian Splatting and MuJoCo, Discoverse enables massively parallel simulation of multiple sensor modalities and accurate physics, with inclusive supports for existing 3D assets, robot models, and ROS plugins, empowering large-scale robot learning and complex robotic benchmarks. Through extensive experiments on imitation learning, Discoverse demonstrates state-of-the-art zero-shot Sim2Real transfer performance compared to existing simulators. For code and demos: https://air-discoverse.github.io/.

cs.RO↗

arXiv2025

High mechanical strength Si anode synthesis with interlayer bonded expanded graphite structure for lithium-ion batteries

Despite advancements in silicon-based anodes for high-capacity lithium-ion batteries, their widespread commercial adoption is still hindered by significant volume expansion during cycling, especially at high active mass loadings crucial for practical use. The root of these challenges lies in the mechanical instability of the material, which subsequently leads to the structural failure of the electrode. Here, we present a novel synthesis of a composite combining expanded graphite and silicon nanoparticles. This composite features a unique interlayer-bonded graphite structure, achieved through the application of a modified spark plasma sintering method. Notably, this innovative structure not only facilitates efficient ion and electron transport but also provides exceptional mechanical strength (Vickers hardness: up to 658 MPa, Young's modulus: 11.6 GPa). This strength effectively accommodates silicon expansion, resulting in an impressive areal capacity of 2.9 mA h cm-2 (736 mA h g-1) and a steady cycle life (93% after 100 cycles). Such outstanding performance is paired with features appropriate for large-scale industrial production of silicon batteries, such as active mass loading of at least 3.9 mg cm-2, a high-tap density electrode material of 1.68 g cm-3 (secondary clusters: 1.12 g cm-3), and a production yield of up to 1 kg per day.

cond-mat.mtrl-sci↗

arXiv2025

Koopman Operators in Robot Learning

Koopman operator theory offers a rigorous treatment of dynamics and has been emerging as an alternative modeling and learning-based control method across various robotics sub-domains. Due to its ability to represent nonlinear dynamics as a linear (but higher-dimensional) operator, Koopman theory offers a fresh lens through which to understand and tackle the modeling and control of complex robotic systems. Moreover, it enables incremental updates and is computationally inexpensive, thus making it particularly appealing for real-time applications and online active learning. This review delves deeply into the foundations of Koopman operator theory and systematically builds a bridge from theoretical principles to practical robotic applications. We begin by explaining the mathematical underpinnings of the Koopman framework and discussing approximation approaches for incorporating inputs into Koopman-based modeling. Foundational considerations, such as data collection strategies as well as the design of lifting functions for effective system embedding, are also discussed. We then explore how Koopman-based models serve as a unifying tool for a range of robotics tasks, including model-based control, real-time state estimation, and motion planning. The review proceeds to a survey of cutting-edge research that demonstrates the versatility and growing impact of Koopman methods across diverse robotics sub-domains: from aerial and legged platforms to manipulators, soft-bodied systems, and multi-agent networks. A presentation of more advanced theoretical topics, necessary to push forward the overall framework, is included. Finally, we reflect on some key open challenges that remain and articulate future research directions that will shape the next phase of Koopman-inspired robotics. To support practical adoption, we provide a hands-on tutorial with executable code at https://shorturl.at/ouE59.

cs.RO↗