arXiv subjects

Zhuoran Wang

Publications and source records attributed to Zhuoran Wang.

At least 19 recordsLinked to original sources

arXiv2026

RA-MoE: Routing-Aligned Fine-Tuning for Multilingual Adaptation of Mixture-of-Experts Models

Mixture-of-Experts (MoE) models enable efficient LLM scaling, yet adapting them to non-English downstream tasks remains challenging. Standard multilingual fine-tuning largely ignores their heterogeneous routing structure. Across multiple MoE models and tasks, we find strong cross-lingual routing alignment in middle layers, with routing divergence associated with target-language performance gaps. Motivated by this observation, we propose RA-MoE (Routing-Aligned MoE Fine-Tuning), a three-stage framework for multilingual MoE adaptation. RA-MoE categorizes parallel examples into four correctness groups (cc/ci/ic/ii) and identifies task-relevant experts in middle layers. It then selectively aligns target-language routing on ci examples toward successful English routing patterns, jointly matching the total routing mass assigned to task experts and its relative allocation among them. Experiments across three MoE models, three downstream tasks, and six target languages show that RA-MoE consistently outperforms standard SFT and strong routing-aware baselines. Further analyses confirm the intended routing changes and reveal that middle-layer task routing is largely shared and transferable across languages, providing mechanistic evidence for the cross-language transferability of task-specific routing.

cs.CL↗

arXiv2026

RoboRecover: Benchmarking Robot Policy Recovery under Execution Deviations

Robot-policy benchmarks increasingly cover diverse tasks and preset out-of-distribution conditions, but typically evaluate complete trajectories from predefined initial states. These evaluations often focus on the initialized scene and the final outcome, while paying less attention to the dynamic interaction process. During closed-loop execution, actions and contacts can alter object relations and task progress, producing off-nominal intermediate states that need recovery. Recovery requires a policy to infer how task progress has changed, correct the relevant relations, and continue the original goal. We introduce RoboRecover, a benchmark for robot policy recovery under execution deviations. RoboRecover selects deviation states from trajectories, reconstructs them by replaying action prefixes, and evaluates policies on the original task. RoboRecover contains 2,000 scenarios across RoboTwin and LIBERO, with 1,000 scenarios and a fixed 800/200 train/test split on each platform. Results show that initial-state performance does not determine recovery performance and policies exhibit different recovery strengths across scenarios. Using its training split, RoboRecover further supports study on recovery interventions. RoboRecover establishes recovery from execution-induced intermediate states as a distinct dimension of robot policy evaluation.

cs.RO↗

arXiv2026

Vision Transformer-Based Multi-Level Feature Fusion for Multi-Label Sewer Defect Classification

Automated classification of sewer defects is essential for infrastructure condition assessment and maintenance decision-making, but existing deep learning methods struggle to balance classification accuracy and computational complexity in large-scale multi-label scenarios. This study develops Sewer-Transformer-ML, a hierarchical vision Transformer with multi-level feature fusion, together with two lightweight architectures, Sewer-MobileNet-ML and Sewer-Mobile-TransNet, for resource-constrained inspection scenarios. On the Sewer-ML test set, Sewer-Transformer-ML-Base achieved an $F2_{\text{CIW}}$ of 65.68% and an $F1_{\text{Normal}}$ of 92.68%, ranking first on the public leaderboard and exceeding the second-ranked method by 7.6 percentage points in $F2_{\text{CIW}}$. Sewer-MobileNet-ML achieved an $F2_{\text{CIW}}$ of 65.73% with only 17 M parameters, representing an approximately 95% parameter reduction relative to the base model. Under the standard Sewer-Capsule data split, Sewer-Mobile-TransNet achieved 96.43% classification accuracy. When the training set was reduced to 1,177 images, pretraining on Sewer-ML consistently improved model performance. Ablation experiments further showed that direct concatenation was more effective for Transformer features, whereas attention-based fusion better supported multiscale CNN features. These findings provide a computational basis for automated sewer inspection, lightweight model design, and adaptation across civil infrastructure inspection platforms.

cs.CV↗

arXiv2026

TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation

Mixture-of-Experts (MoE) language models route each token through a small subset of experts, making routing patterns useful for identifying task-relevant experts during downstream adaptation. Yet current approaches have two limitations: task experts are typically identified from aggregate routing statistics that reflect usage rather than association with successful task completion, and task-expert activations remain underexplored as signals for supervision allocation. We introduce Task-Expert-Aware Supervision (TEXAS), which combines correctness-conditioned task expert discovery with token-level supervision allocation. TEXAS compares expert activations on instances that the base model solves successfully and those it fails to solve, and retains experts more strongly activated on successful instances. During fine-tuning, it upweights answer tokens in failed instances when they activate these experts. TEXAS therefore leverages existing routing behavior without restricting adaptation to a fixed expert subset or imposing an explicit target routing distribution. Across three MoE models and six benchmarks, TEXAS achieves the best or tied-best performance in 17 of 18 settings and improves over the strongest baseline by 1.3--1.5 points on average. Ablations and further analyses validate both the discovered experts and the resulting supervision strategy.

cs.CL↗

arXiv2026

TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification

The promise of multimodal fusion lies in combining complementary sources of evidence, yet more evidence does not always yield a better prediction. Recent multimodal models have advanced fusion through richer cross-modal interaction and sample-adaptive fusion. However, the influence assigned to a modality during fusion does not reveal whether that source is unreliable, redundant, or poorly matched to a specialized expert. To address this limitation, we introduce TIER-MoE, a risk-guided subspace mixture-of-experts model that defines sample-specific modality reliability as the prediction loss its unimodal predictor is expected to incur. This risk is learned from out-of-fold predictions generated by models that were not trained on the corresponding sample. TIER-MoE combines the estimated risk with expert-specific subspace compatibility for sparse modality-expert routing, while an always-active shared path preserves multimodal complementarity. We evaluate TIER-MoE on four public multimodal biomedical datasets spanning Alzheimer's disease status, skin-lesion malignancy, and retinal classification. Results demonstrate its superiority over state-of-the-art methods in predictive performance and probability calibration, with consistent improvements in Macro-F1 and Brier score and strong zero-shot generalization to an external cohort.

cs.LG↗

arXiv2026

Tamed Stochastic Gradient Hamiltonian Monte Carlo

In this paper, we propose a novel tamed stochastic gradient Hamiltonian Monte Carlo (tSGHMC) algorithm for sampling and stochastic optimization problems with superlinearly growing stochastic gradients. Under a certain continuity in average condition and a strong convexity condition, we establish a non-asymptotic error bound in Wasserstein-2 distance for tSGHMC with the rate of convergence equal to $1/4$. Then, we derive an upper estimate for the associated expected excess risk, which provides a theoretical guarantee for the performance of tSGHMC. To illustrate the effectiveness of the proposed algorithm, we apply tSGHMC to practical examples, including a newsvendor problem and a Conditional Value-at-Risk minimization problem, using synthetic and real-world datasets. Numerical results support our theoretical findings. Furthermore, we compare tSGHMC with its first-order counterpart, namely, the tamed unadjusted stochastic Langevin algorithm. Simulation results demonstrate that tSGHMC achieves lower root mean square error and expected excess risk across a range of tasks.

math.OC↗

arXiv2026

Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision

Cross-embodiment transfer in vision-language-action (VLA) models remains challenging because low-level state and action spaces differ fundamentally across robot platforms. We observe that the high-level cognitive process underlying manipulation, including scene perception, object identification, task planning, and sub-task decomposition, is largely shared across embodiments. Based on this observation, we present ZR-0, a 2.6 billion parameter end-to-end VLA model that uses dense Embodied Chain-of-Thought (ECoT) supervision to align cross-embodiment representations within the vision-language model (VLM). ZR-0 adopts a dual-stream architecture: a pre-trained VLM (System 2) generates structured ECoT reasoning during training, while a Diffusion Transformer-based action expert (System 1) produces continuous action chunks via flow matching. The two components are coupled through cross-attention, with an attention mask that restricts the action expert to input prompt features only, enabling ECoT generation to be entirely skipped at inference without any performance loss. ZR-0 is pre-trained on ProcCorpus-60M, a large-scale dataset comprising approximately 60 million frames (approximately 1,000 hours) from over 400K trajectories, with dense ECoT annotations covering 96.8% of all frames. We evaluate ZR-0 on three simulation benchmarks spanning single-arm (LIBERO), bimanual (RoboTwin 2.0), and humanoid (RoboCasa GR-1 Tabletop) embodiments, as well as real-world experiments on the xArm platform, demonstrating strong performance across all settings. Code and model checkpoints are available at https://github.com/RUCKBReasoning/ZR-0.

cs.RO↗

arXiv2026

SemBlock: Semantic Boundary Dynamic Blocks for Diffusion LLMs

Diffusion language models (DLMs) generate text through iterative denoising, and blockwise decoding improves their practicality by committing tokens in local blocks. However, existing blockwise methods typically rely on fixed block sizes or delimiter-based runtime signals, which do not necessarily align with semantic boundaries. In this paper, we propose SemBlock, a semantic-boundary-driven dynamic block decoding framework for diffusion LLMs. SemBlock formulates dynamic block construction as semantic boundary prediction and trains lightweight predictors on frozen LLaDA hidden states. To provide supervision, we construct SemBound, a semantic-boundary dataset that derives boundary labels from discourse units, reasoning steps, and implementation spans across natural language, math, and code tasks. During inference, SemBlock uses predicted boundary probabilities to select the ending position of each dynamic block. Experiments on GSM8K, IFEval, MATH, and HumanEval show that SemBlock consistently improves over fixed-block decoding and AdaBlock. Our code is publicly available: https://github.com/TH-AI-Lab-PKU/SemBlock.

cs.CL↗

arXiv2026

A geometry-dependent, force balance-driven model of Staphylococcus epidermidis biofilm cell cluster detachment

Biofilms, bacteria cells surrounded by a self-produced polymeric matrix, are common on medical devices and lead to many hospital infections. The biofilm lifecycle includes disassembly and dispersion, where bacteria clusters detach from the biofilm, circulate in the bloodstream, and potentially colonize secondary infection sites. Existing models often simplify detachment to a function of biofilm thickness or extracellular polymeric substance (EPS) density, without tracking properties of detached clusters that impact their biological fate, including cluster size and morphology. Addressing this gap, our detachment model accounts for drag and adhesion in tagged sections of the biofilm determined by the cluster geometry and local arrangement of bacteria and EPS. A stickiness parameter controls local EPS adhesion strength, which is modulated to disrupt (or compromise) EPS biomass. We specifically model the detachment of clusters from a Staphylococcus epidermidis biofilm grown for 24 hours. Experimental data for biofilm microstructural features are utilized to benchmark the simulated biofilm, which is then subjected to different EPS disruption levels. We examine parameters that influence detached biofilm cell cluster frequency, size, and shape, providing mechanistic insights into how compromised EPS influences detachment dynamics. This integrated modeling framework is a significant advance in the predictive capabilities for biofilm detachment processes.

q-bio.QM↗

arXiv2026

Action Draft and Verify: A Self-Verifying Framework for Vision-Language-Action Model

Vision-Language-Action (VLA) models have recently demonstrated strong performance across embodied tasks. Modern VLAs commonly employ diffusion action experts to efficiently generate high-precision continuous action chunks, while auto-regressive generation can be slower and less accurate at low-level control. Yet auto-regressive paradigms still provide complementary priors that can improve robustness and generalization in out-of-distribution environments. To leverage both paradigms, we propose Action-Draft-and-Verify (ADV): diffusion action expert drafts multiple candidate action chunks, and the VLM selects one by scoring all candidates in a single forward pass with a perplexity-style metric. Under matched backbones, training data, and action-chunk length, ADV improves success rate by +4.3 points in simulation and +19.7 points in real-world over diffusion-based baseline, with a single-pass VLM reranking overhead.

cs.CV↗

arXiv2026

KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs

Large language models rely on kv-caches to avoid redundant computation during autoregressive decoding, but as context length grows, reading and writing the cache can quickly saturate GPU memory bandwidth. Recent work has explored KV-cache compression, yet most approaches neglect the data-dependent nature of kv-caches and their variation across layers. We introduce KV-CoRE KV-cache Compressibility by Rank Evaluation), an SVD-based method for quantifying the data-dependent low-rank compressibility of kv-caches. KV-CoRE computes the optimal low-rank approximation under the Frobenius norm and, being gradient-free and incremental, enables efficient dataset-level, layer-wise evaluation. Using this method, we analyze multiple models and datasets spanning five English domains and sixteen languages, uncovering systematic patterns that link compressibility to model architecture, training data, and language coverage. As part of this analysis, we employ the Normalized Effective Rank as a metric of compressibility and show that it correlates strongly with performance degradation under compression. Our study establishes a principled evaluation framework and the first large-scale benchmark of kv-cache compressibility in LLMs, offering insights for dynamic, data-aware compression and data-centric model development.

cs.CL↗

arXiv2025

Few-Shot-Based Modular Image-to-Video Adapter for Diffusion Models

Diffusion models (DMs) have recently achieved impressive photorealism in image and video generation. However, their application to image animation remains limited, even when trained on large-scale datasets. Two primary challenges contribute to this: the high dimensionality of video signals leads to a scarcity of training data, causing DMs to favor memorization over prompt compliance when generating motion; moreover, DMs struggle to generalize to novel motion patterns not present in the training set, and fine-tuning them to learn such patterns, especially using limited training data, is still under-explored. To address these limitations, we propose Modular Image-to-Video Adapter (MIVA), a lightweight sub-network attachable to a pre-trained DM, each designed to capture a single motion pattern and scalable via parallelization. MIVAs can be efficiently trained on approximately ten samples using a single consumer-grade GPU. At inference time, users can specify motion by selecting one or multiple MIVAs, eliminating the need for prompt engineering. Extensive experiments demonstrate that MIVA enables more precise motion control while maintaining, or even surpassing, the generation quality of models trained on significantly larger datasets.

cs.CV↗

arXiv2025

Parameter-free inexact block Schur complement preconditioning for linear poroelasticity under a hybrid Bernardi-Raugel and weak Galerkin finite element discretization

This work investigates inexact block Schur complement preconditioning for linear poroelasticity problems discretized using a hybrid approach: Bernardi-Raugel elements for solid displacement and lowest-order weak Galerkin elements for fluid pressure. When pure Dirichlet boundary conditions are applied to the displacement, the leading block of the resulting algebraic system becomes almost singular in the nearly incompressible (locking) regime, hindering efficient iterative solution. To overcome this, the system is reformulated as a three-field problem with an inherent regularization that maintains the original solution while ensuring nonsingularity. Analysis shows that both the minimal residual (MINRES) and generalized minimal residual (GMRES) methods, when preconditioned with inexact block diagonal and triangular Schur complement preconditioners, achieve convergence independent of mesh size and the locking parameter for the regularized system. Similar theoretical results are established for the situation with displacement subject to mixed boundary conditions, even without regularization. Numerical experiments in 2D and 3D confirm the benefits of regularization under pure Dirichlet conditions and the robustness of the preconditioners with respect to mesh size and the locking parameter in both boundary condition scenarios. Finally, a spinal cord simulation with discontinuous material parameters further illustrates the effectiveness and robustness of the proposed iterative solvers.

math.NA↗

arXiv2025

Wafer-scale integration of single nanodiamonds via electrostatic-trapping

Nanodiamonds (NDs) are key materials for building nanoscale quantum sensing, imaging and communication devices. Scalable configuration of single NDs on heterogeneous platforms, forming photonic quantum source arrays, will be an essential solution towards realizing next-generation practical and industrial quantum devices. However, NDs are challenging to manipulate because their size, shape and surface chemistry vary substantially. Here, we show a simple method based on electrostatic-trapping to rapidly and reliably pattern single ND arrays on arbitrary substrates at scale. Our method, which uses carefully engineered microscale hole templates and electrostatic force, captures single NDs across 8-inch wafers with 82.5% yields within 5 min. Systematic experimental and theoretical studies show the number of deposited NDs primarily depends on the diameter of the hole trap. The method is compatible with mature CMOS technologies, enabling the mass production of scalable and integrable quantum devices. This advancement is expected to accelerate the commercialization and industrial adoption of ND-based technologies.

physics.optics↗

arXiv2025

HateClipSeg: A Segment-Level Annotated Dataset for Fine-Grained Hate Video Detection

Detecting hate speech in videos remains challenging due to the complexity of multimodal content and the lack of fine-grained annotations in existing datasets. We present HateClipSeg, a large-scale multimodal dataset with both video-level and segment-level annotations, comprising over 11,714 segments labeled as Normal or across five Offensive categories: Hateful, Insulting, Sexual, Violence, Self-Harm, along with explicit target victim labels. Our three-stage annotation process yields high inter-annotator agreement (Krippendorff's alpha = 0.817). We propose three tasks to benchmark performance: (1) Trimmed Hateful Video Classification, (2) Temporal Hateful Video Localization, and (3) Online Hateful Video Classification. Results highlight substantial gaps in current models, emphasizing the need for more sophisticated multimodal and temporally aware approaches. The HateClipSeg dataset are publicly available at https://github.com/Social-AI-Studio/HateClipSeg.git.

cs.CV↗

arXiv2025

An inherent regularization approach to parameter-free preconditioning for nearly incompressible linear poroelasticity and elasticity

An inherent regularization strategy and block Schur complement preconditioning are studied for linear poroelasticity problems discretized using the lowest-order weak Galerkin FEM in space and the implicit Euler scheme in time. At each time step, the resulting saddle point system becomes nearly singular in the locking regime, where the solid is nearly incompressible. This near-singularity stems from the leading block, which corresponds to a linear elasticity system. To enable efficient iterative solution, this nearly singular system is first reformulated as a saddle point problem and then regularized by adding a term to the (2,2) block. This regularization preserves the solution while ensuring the non-singularity of the new system. As a result, block Schur complement preconditioning becomes effective. It is shown that the preconditioned MINRES and GMRES converge essentially independent of the mesh size and the locking parameter. Both two- and three-field formulations are considered for the iterative solution of the linear poroelasticity. The efficient solution of the two-field formulation builds upon the effective iterative solution of linear elasticity. For this case, MINRES and GMRES achieve parameter-free convergence when used with block Schur complement preconditioning, where the inverse of the leading block leverages efficient solvers for linear elasticity. The poroelasticity problem can also be reformulated as a three-field system by introducing a numerical pressure variable into the linear elasticity part. The inherent regularization strategy extends naturally to this formulation, and preconditioned MINRES and GMRES also show parameter-free convergence for the regularized system. Numerical experiments in both two and three dimensions confirm the effectiveness of the regularization strategy and the robustness of the block preconditioners.

math.NA↗

arXiv2025

Efficient parameter-robust preconditioners for linear poroelasticity and elasticity in the primal formulation

Poroelasticity problems play an important role in various engineering, geophysical, and biological applications. Their full discretization results in a large-scale saddle-point system at each time step that is becoming singular for locking cases and needs effective preconditioners for its fast iterative solution. Instead of constructing spectrally equivalent ones, we develop nonsingular preconditioners so that the eigenvalues of the preconditioned system consist of a cluster around $1$ and an outlier in the order of $1/λ$, where $λ$ is a Lamé constant that is large for locking cases. It is known that the convergence factor of GMRES is bounded by the radius of the cluster for this type of systems. Both two- and three-field block triangular Schur complement preconditioners are studied. Upper bounds of the radius of the eigenvalue cluster for those systems are obtained and shown to be related to the inf-sup condition but independent of mesh size, time step, and locking parameters, which reflects the robustness of the preconditioners with respect to parameter variations. Moreover, the developed preconditioners do not need to compute the Schur complement and neither require exact inversion of diagonal blocks except the leading one. A locking-free weak Galerkin finite element method and the implicit Euler scheme are used for the discretization of the governing equation. Both two- and three-dimensional numerical results are presented to confirm the effectiveness and parameter-robustness of the developed preconditioners.

math.NA↗

arXiv2025

A general regularization strategy for singular Stokes problems and convergence analysis for corresponding discretization and iterative solution

A general regularization strategy is considered for the efficient iterative solution of the lowest-order weak Galerkin approximation of singular Stokes problems. The strategy adds a rank-one regularization term to the zero (2,2) block of the underlying singular saddle point system. This strategy includes the existing pressure pinning and mean-zero enforcement regularization as special examples. It is shown that the numerical error maintains the optimal-order convergence provided that the nonzero Dirichlet boundary datum is approximated numerically with sufficient accuracy. Inexact block diagonal and triangular Schur complement preconditioners are considered for the regularized system. The convergence analysis for MINRES and GMRES with corresponding block preconditioners is provided for different choices of the regularization term. Numerical experiments in two and three dimensions are presented to verify the theoretical findings and the effectiveness of the preconditioning for solving the regularized system.

math.NA↗