arXiv subjects

Jie Zhang

Publications and source records attributed to Jie Zhang.

At least 127 records · Page 7Linked to original sources

arXiv2026

Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data

Learning generalizable robot manipulation policies requires large-scale and diverse demonstration data. Egocentric human manipulation videos offer rich scene and task diversity, and prior work has shown that retargeting and rendering such videos into robot-format data can yield effective per-task policies at small scale. However, whether this approach can provide pretraining benefits for vision-language-action models at scale remains unexplored. We present \textbf{Ego2Robot}, a scalable pipeline that converts egocentric human manipulation videos into robot training data through action retargeting, robot-arm visual synthesis, and multi-level quality curation. Ego2Robot supports both curated datasets and in-the-wild videos, producing 18,561 hours of robot training data spanning 15 robot morphologies, making it the largest ego-to-robot dataset to date. To evaluate generalization, we extend RoboTwin2.0 with disentangled perturbation axes covering visual appearance, scene layout, embodiment morphology, and task semantics. Experiments show that joint pretraining on Ego2Robot-synthesized and robot data consistently improves out-of-distribution generalization across multiple perturbation types, with benefits validated on real-robot deployment. Project page: https://www-ye.github.io/ego2robot_blog/

cs.RO↗

arXiv2026

OPTD: On-Policy Transition Distillation with Consistency-Guided Adaptive Compression for Few-Step Diffusion Language Models

Diffusion language models (dLLMs) can predict many tokens in parallel, but accurate generation still requires many iterative denoising steps. Few-step distillation accelerates decoding by compressing multiple teacher steps into a single student transition. However, existing methods construct supervision on off-policy trajectories. At inference, the student's early parallel commitments alter the context of later predictions, so the states it actually visits drift away from the supervised ones--precisely when step compression is most aggressive. On-policy distillation is a natural remedy for this mismatch, but it leaves open how far each transition should advance: matching only the teacher's next action limits compression, while indiscriminately merging future actions can violate intermediate dependencies. To address this limitation, we propose OPTD, On-Policy Transition Distillation with consistency-guided adaptive compression. It samples partial states from the few-step student's own trajectories, uses a frozen, question-only teacher to identify outcome-aligned future candidates, and orders them by current-state confidence. The method then selects the longest prefix whose joint commitment preserves the teacher's rollout outcome. A set-bottleneck objective promotes every verified future candidate to the decoder's release threshold, while a frozen-teacher KL anchor regularizes all other active positions. Neither target construction nor training uses a gold response. Across four mathematical reasoning and code-generation benchmarks, OPTD consistently improves the quality--efficiency trade-off and attains the strongest overall quality-constrained AUP among the evaluated few-step baselines.

cs.CL↗

arXiv2026

TIDE-MC: Two-Sided Interpolative Decomposition for Billion-Scale GPU Matrix Completion

Matrix completion supports large-scale recommendation and scientific computing, yet existing GPU solvers commonly assume that the observed matrix or its dense factors fit in device memory. On real workloads, this assumption leads to out-of-memory failures or severe PCIe overhead under naive paging. We present TIDE-MC, a bounded-memory GPU framework built on Two-Sided Interpolative Decomposition (TSID). TSID uses a sampled template submatrix as an anchor for reconstructing the full low-rank matrix, allowing computation and storage to scale with the template and active data chunks rather than the complete matrix. TIDE-MC realizes this formulation through two execution stages. First, a conflict-free synchronization engine recovers the template using parallel factorization and hierarchical gradient aggregation. Second, a chunked reconstruction pipeline extends the recovered template to the remaining matrix while overlapping PCIe transfers with GPU computation. An asymmetric gradient-clipping scheme stabilizes mixed-precision Tensor Core execution. Across 15 benchmarks, TIDE-MC completes workloads that cause existing GPU solvers to run out of memory. Compared with the evaluated state-of-the-art baselines, it achieves up to 11,647x speedup, reduces peak memory usage by up to 8.5x, and lowers reconstruction error by up to 99.7%. These results show that template-anchored decomposition and stage-specific GPU execution can scale matrix completion beyond device-memory capacity.

cs.DC↗

arXiv2026

Generative Adversarial Reconstruction with Adaptive Thresholding for Obstructed Targets in Computational Microwave Imaging

In this work, an end-to-end generative adversarial framework for computational microwave imaging (CMI) is proposed to reconstruct the targets of interest directly from the measurements of targets obstructed by undesired objects. It integrates a conditional generative adversarial network (cGAN) with a learnable soft-threshold module (STM) to adaptively suppress non-target related information. The proposed framework is evaluated on a diverse dataset comprising MNIST digits obstructed by E-MNIST letters for training and testing, as well as on measurements acquired with an experimental CMI system. In addition, further studies involving objects with different geometries are conducted, demonstrating that the proposed approach can be adapted to other types of objects. Numerical experiments show that the proposed cGAN-STM achieves a normalized mean square error (NMSE) of 0.066 and a structural similarity index (SSIM) of 0.876 under ideal conditions. Comprehensive analyses, including benchmarking, analysis of the STM mechanism, and evaluation under different obstruction sizes, are also conducted. The performance of the model under different signal-to-noise ratio (SNR) scenarios is also evaluated, achieving reasonable reconstruction quality at 15 dB SNR with an NMSE of 0.168 and an SSIM of 0.700. Even at low SNR levels, recognizable target outlines are preserved. These results highlight the effectiveness and adaptability of the proposed method.

eess.SP↗

arXiv2026

Sphere Retraction Normalizations

Residual connections are the de facto mechanism for training deep neural networks stably. Geodesic Normalization (GeoNorm) recasts them on a Riemannian manifold, orthogonalizing each layer output against the current hidden state and applying the resulting update through the Riemannian exponential map. Every hidden state thus keeps a constant $\ell_{2}$-norm, confining the residual stream to a hypersphere. The exponential map, however, is only one member of a broad family of retraction maps. We show that on the hypersphere this entire family collapses to a single scalar design choice. What distinguishes one retraction from another is only how the magnitude of an update is converted into a rotation angle within the plane spanned by the hidden state and the update. This view places Euclidean residual connections and GeoNorm in one framework. Instantiating it with the metric projection retraction and the Cayley retraction yields Proj-SpheretNorm and Cay-SpheretNorm, which are exactly norm-preserving yet require only algebraic operations. Both prove to be members of a one-parameter family of angular retractions, $p$-SpheretNorm, whose rotation angle saturates rather than growing without bound. The two methods above are recovered exactly at $p = 1$ and $p = 2$, while the identity map and GeoNorm arise only as limits at either end. On nanoGPT, all three methods outperform existing lightweight deep connection schemes, and the best validation loss is attained at finite $p$, indicating that the exponential map is not the preferred retraction for spherical residual streams but merely one end of a spectrum.

cs.LG↗

arXiv2026

SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery

Modern neuroscience relies on integrating multi-scale, multimodal datasets to uncover the neural principles underlying intelligence. However, analytical challenges posed by highly heterogeneous data and fragmented workflows increasingly constrain discoveries. Here we introduce SeekBrain, an autonomous multi-agent framework designed to accelerate neuroscience discovery through domain-grounded hierarchical planning and cross-modal data analysis. SeekBrain dynamically constructs a repertoire of analysis recipes extracted from code-paper pairs. By coupling this codified expertise with agentic planning and execution engines, the framework scalably generates hypotheses and analytical pipelines on demand. Systematic evaluation on the expert-annotated BrainArena benchmark demonstrates that SeekBrain substantially outperforms state-of-the-art agent baselines across various analysis tasks. Crucially, when deployed in real-world research, SeekBrain integrated behavioral, neural, and anatomical data to reveal structured, distributed neural representations of larval zebrafish behavior and a shared axis of regional decoding strength across the brain in a mouse decision-making task. These results establish SeekBrain as a scalable and practical tool for accelerating data-driven discoveries in neuroscience.

cs.MA↗

arXiv2026

State-Dependent Safety Failures in Multi-Turn Language Model Interaction

Safety alignment in large language models is typically evaluated under isolated queries, yet real-world use is inherently multi-turn. Although multi-turn jailbreaks are empirically effective, the structure of conversational safety failure remains insufficiently understood. In this work, we study safety failures from a state-space perspective and show that many multi-turn safety failures in current safety-aligned language models arise from contextual state evolution, a regime that is not fully captured by isolated prompt-level analyses alone. We introduce STAR, a state-oriented diagnostic framework that treats dialogue history as a state transition operator and enables controlled analysis of safety behavior along interaction trajectories. Rather than optimizing attack strength, STAR provides a principled probe of how aligned models traverse the safety boundary under autoregressive conditioning. Across multiple frontier language models, we find that systems that appear robust under static evaluation can undergo rapid and reproducible safety collapse under structured multi-turn interaction. Mechanistic analysis reveals monotonic drift away from refusal-related representations and abrupt phase transitions induced by role-conditioned context. Together, these findings motivate viewing language model safety as a dynamic, state-dependent process defined over conversational trajectories.

cs.CR↗

arXiv2026

The Semantic Least-Energy Principle: A Hypothesis for Intelligence

Despite remarkable advances in artificial intelligence and cognitive neuroscience, no generally accepted first-principle explains why intelligent systems organize latent semantic states as they do. Existing frameworks such as information theory, the Information Bottleneck, the Degree of Information Abstraction, predictive coding and the Free Energy Principle provide powerful frameworks for understanding communication, learning, and prediction, but do not explicitly explain the emergence and organization of semantic intelligence. Here we propose the \textbf{Semantic Least-Energy Principle (SLEP)} as a hypothesis that intelligent systems evolve internal representations by maximizing semantic utility while progressively minimizing semantic, predictive, and computational energy. We formulate this hypothesis within a variational framework in which semantic cognition is governed by a Semantic Action Functional whose stationary solutions define efficient trajectories on a latent semantic manifold. This formulation emerges a series of theoretical predictions, including semantic geometry, semantic thermodynamics, and low-energy latent semantic states as complementary consequences of the same underlying optimization process. SLEP unifies semantic abstraction, reasoning, planning, and communication within a common mathematical framework while generating experimentally testable predictions for both artificial and biological intelligence. Although the hypothesis remains to be rigorously validated, it provides a principled foundation for investigating semantic intelligence from a first-principle perspective.

cs.IT↗

arXiv2026

DisDP: Disaggregating Compute, Network, and Storage for Model-Sharded Data-Parallel Training

Model-sharded data parallelism (MSDP), e.g., ZeRO, evenly shards the model states across all GPUs, and thus has been widely adopted by LLM pre-training, such as Llama and DeepSeek, due to its low GPU memory capacity requirement. However, MSDP introduces severe overhead from additional network communication collectives (i.e., AllGather and ReduceScatter). Although the collectives themselves only occupy fewer than 10% of GPU SMs, their execution time increases by 41% due to the serial execution of aggregated CPU/GPU-managed compute (i.e., GEMM), network (i.e., NCCL), and storage (i.e., optimizer states). To this end, we present DisDP, a fully disaggregated distributed data-parallel architecture that first fully disaggregates compute, network, and storage for MSDP, such that GPUs only focus on the computing part, and thus the GPU utilization is maximized. The key idea is 1) fully offloading collectives to SmartNICs and SmartSwitch to avoid interference between GEMM kernels and collective kernels, and 2) fully offloading storage to a SmartSwitch-enhanced parameter server that allows a single PS to serve massive workers with linear scalability. DisDP on 8 distributed GPUs outperforms the state-of-the-art training systems by 3.98x when training on a 175B model, validating the efficiency of disaggregation.

cs.DC↗

arXiv2026

Dynamic Modeling and Load-Following Control of Small Modular Reactors with Moving-Boundary Steam Generators and Thermodynamically Coupled Rankine Cycles

Small modular reactors (SMRs) are increasingly considered for flexible power generation; however, many dynamic studies still neglect the thermodynamic coupling between the primary and secondary loops that is essential for accurate assessment of load-following capability. In this study, we develop a hybrid dynamic framework that couples an equation-based model of a NuScale-type integral pressurized water reactor, including the reactor, primary loop, and moving-boundary helical-coil once-through steam generator, with a physics-based secondary Rankine cycle comprising the steam throttle valve, turbine, condenser, and feedwater pump. This approach enforces mass and energy conservation across the coupled system while preserving physically consistent pressure-flow and enthalpy-flow interactions across the domain boundary. The integrated model reproduces nominal design-point conditions and is used to analyze a 5% step reduction in turbine mechanical-power demand under five control configurations, including a decentralized three-loop control architecture for the valve, feedwater pump, and control rods. The results show that partial control strategies can satisfy individual objectives but leave pressure, thermal, or phase-boundary deviations, whereas simultaneous action of all three actuators provides the most balanced response by stabilizing steam pressure, limiting primary-loop thermal deviations, and maintaining acceptable steam-generator operating margins during load-following maneuvers. Compared with a conventional linear steam-cycle representation, the coupled framework captures dynamic back-pressure and variable turbine enthalpy drop that are otherwise neglected, leading to different predictions of transient behavior and required steam flow.

eess.SY↗

arXiv2026

FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities

Multimodal Federated Learning is often challenged by arbitrary modality missingness and Non-IID data distributions, which lead to severe representation drift and hinder effective collaboration across clients. Existing methods typically rely on generative imputation, external auxiliary data, or isolated unimodal training to bridge modality gaps, often incurring substantial communication and computational costs as well as potential privacy risks. To address these limitations, we propose FedTaste, a parameter-efficient framework for topology-aware structural transfer in Multimodal Federated Learning with missing modalities. Instead of aligning fragile first-order features, FedTaste focuses on more stable group-level semantic relations. Specifically, FedTaste leverages frozen foundation models to extract a joint multimodal topology from full-modality clients, which is then consolidated by the server into a global structural blueprint. To adapt clients with missing modalities, we introduce Modality-Adaptive Structural Prompts together with spectral consistency regularization, enabling lightweight branch-specific adaptation that aligns local partial representations with the shared blueprint. In this way, FedTaste avoids explicit modality imputation while preserving shared semantic structure across clients. Extensive experiments demonstrate that FedTaste consistently achieves superior performance across multiple datasets and challenging Non-IID settings, while substantially reducing communication overhead compared with existing methods.

cs.MM↗

arXiv2026

Comparative 3D Asymmetric Expansion and Angular Widths Evolution of Fast and Slow Coronal Mass Ejections

The radial and lateral dimensions of coronal mass ejections (CMEs) influence their duration and probability of encounter at Earth. These properties are linked to the expansion speed of CMEs in different radial and lateral directions; however, most earlier studies modeled CME evolution using a projected full ice-cream cone geometry, which does not distinguish between radial and lateral expansion. Our study investigates the asymmetric expansion (relative radial and lateral components) and kinematics of seven fast and seven slow CMEs within coronagraphic heights, using the Graduated Cylindrical Shell model. Our study confirms that CMEs expand asymmetrically, with lateral expansion exceeding radial expansion in both CME populations. This asymmetry limits the accuracy of the full ice-cream cone model. For both fast and slow CMEs, higher leading edge speeds are associated with higher expansion speeds. At a height of 10 Rs, slow CMEs with larger expansion speeds (lateral and radial) have larger angular widths (face-on and edge-on), whereas fast CMEs exhibit a negative correlation between lateral expansion speed and face-on angular width. We find that the expansion and propagation speeds of slow CMEs exhibit a two-phase evolution, whereas those of fast CMEs display more diverse trends. Overall, this study suggests that fast and slow CMEs evolve differently and should not be treated as a single population in statistical estimates of their physical parameters. Our study highlights the importance of estimating CME angular widths and expansion speeds along different directions, and beyond standard coronagraphic heights, to capture their complete physical evolution.

astro-ph.SR↗

arXiv2026

End-to-End Markov State Sequence Learning for Auditory Attention Decoding

Auditory attention decoding (AAD) identifies the speaker a listener attends to from neural responses like electroencephalography (EEG), making it a key algorithm in neuro-steered hearing aids. However, most neural AAD models are trained as independent short-window classifiers, despite auditory attention being a temporally persistent cognitive state and short-window EEG--audio evidence often being noisy and ambiguous. We propose an end-to-end Markov AAD framework based on conditional random field (CRF) that trains window-level neural emissions under a two-state attention prior. The framework treats the logits of any AAD backbone as Markov emissions, learns the transition rate from a standard HMM initialization, and jointly optimizes cross-entropy and CRF objectives, allowing temporal continuity to guide representation learning rather than merely smoothing predictions after training. We also introduce ESCNet, an EEG--speech correlation backbone that preserves time-aligned features and converts the difference between two mean Pearson correlations into state logits. We evaluate the framework with four emission backbones spanning correlation-based, convolutional, recurrent, and attention-based designs. On the dynamic AVGC dataset, CRF training generally outperforms post-hoc HMM smoothing; with ESCNet, it achieves $86.5\%$ causal and $92.4\%$ non-causal accuracy using $1$s windows. On the static KUL and USTC datasets, it improves causal decoding over fixed-rate post-hoc HMM baselines by $5.6\%$ and $2.0\%$, respectively, showing the superiority of learning AAD as attention state sequence over isolated-window classification.

cs.SD↗

arXiv2026

Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model

While Large Vision-Language Models (LVLMs), represented by LLaVA and GPT-4V, have demonstrated remarkable capabilities, their visual inputs remain vulnerable to adversarial attacks, posing significant security risks. Existing defense methods predominantly target single-task scenarios (e.g., zero-shot classification) and consequently lack generalizability across various multimodal tasks. To address this limitation, we propose a dual adversarial fine-tuning framework that jointly optimizes visual and semantic supervision signals from two modalities, enhancing model robustness while generalizing across multiple downstream tasks. The proposed framework comprises two core components, i.e., $\textbf{Visual}$ supervision branch and $\textbf{Semantic}$ supervision branch. The former branch leverages features from clean images, extracted via a frozen original vision encoder, to guide adversarial robustness while the latter incorporates caption-image alignment as a contextual signal to preserve semantic coherence under attack. Moreover, our method achieves cross-task robustness by simply replacing the CLIP vision encoder in the original model, with no need of separate task-specific retraining or architecture modifications.Extensive experiments demonstrate that our approach outperforms the state-of-the-art method in adversarial robustness evaluation across zero-shot classification, image captioning, and visual question answering (VQA) tasks.

cs.CV↗

arXiv2026

Massive MIMO-OFDM ISAC for Sparse ISAR Imaging: Joint Power and Subcarrier Allocation

This paper investigates a massive multiple-input multiple-output (mMIMO) orthogonal frequency-division multiplexing (OFDM) framework for integrated sensing and communication (ISAC) with inverse synthetic aperture radar (ISAR) imaging, supporting applications such as the Internet of Things (IoT). A dual-function architecture combines communication precoding and dedicated sensing beamforming to enable simultaneous downlink communication and ISAR imaging. Due to intermittent pilot transmission and sparse sensing-subcarrier activation, the received echoes provide incomplete measurements, resulting in a sparse-aperture ISAR reconstruction problem. To address this issue, an adaptive reweighted two-dimensional alternating direction method of multipliers (ADMM) algorithm is developed for high-resolution image recovery from sparse observations. A joint resource-allocation framework is also proposed to optimize communication-subcarrier assignment, sensing-subcarrier selection, and transmit power allocation subject to communication quality-of-service and sensing constraints. Exploiting channel hardening, analytical full-band sensing benchmarks based solely on statistical channel state information (CSI) are derived for maximum-ratio (MR) and zero-forcing (ZF) precoding, while a soft actor-critic (SAC)-based method is developed for sparse-sensing resource allocation. Numerical results show that the proposed adaptive ADMM algorithm improves sparse ISAR reconstruction over conventional methods. The SAC-based design also achieves substantial gains in sum spectral efficiency over the full-band benchmarks while satisfying communication and sensing constraints, thereby revealing the tradeoff between ISAR reconstruction accuracy and communication spectral efficiency.

eess.SP↗

arXiv2026

VOPE: Revisiting Hallucination of Vision-Language Models in Voluntary Imagination Task

Most research on hallucinations in Large Vision-Language Models (LVLMs) focuses on factual description tasks that prohibit any output absent from the image. However, little attention has been paid to hallucinations in voluntary imagination tasks, such as story writing, despite this human-like cognitive ability being essential for real-world generative applications. To address this limitation, we introduce Voluntary-imagined Object Presence Evaluation (VOPE) -- a recheck-based evaluation benchmark for assessing LVLMs' grounding behavior in voluntary imagination tasks. Specifically, VOPE poses recheck-based questions to evaluate how an LVLM interprets the presence of the imagined objects in its own response. Rather than penalizing the imagined content itself, VOPE identifies hallucinations based on the correctness of the model's presence judgments for the generated objects. Built on this idea, we construct a dataset covering captioning, reasoning, and writing tasks with different levels of voluntary imagination. We apply VOPE to several mainstream LVLMs and hallucination mitigation methods, revealing two key findings: (1) most LVLMs hallucinate heavily during voluntary imagination, and their performance in presence evaluation is notably poor on imagined objects; (2) existing hallucination mitigation methods show limited effect in voluntary imagination tasks, making this an important direction for future research.

cs.CV↗

arXiv2026

Should Missing Modalities Always Be Necessary to Repair for Multi-modal Sentiment Analysis?

Existing methods for multimodal sentiment analysis (MSA) under missing modalities usually follow a repair-first paradigm. We revisit this assumption and ask: \emph{should every missing modality be repaired?} A per-sample oracle analysis shows the answer is not always: full-modality input is optimal for only a small fraction of samples, and every modality subset is preferred by some samples. These results suggest that adding or repairing modalities may not always improve prediction, and that the utility of each modality is sample-dependent. Building on this finding, we propose \textbf{S}ufficiency-\textbf{I}nformed \textbf{E}vidential \textbf{V}al\textbf{vE} (\textbf{SIEVE}) that turns ``whether to repair'' into an explicit, learnable decision at the sample level. SIEVE compares a direct prediction branch with a repair branch, derives an empirical sufficiency signal from their per-sample loss gap, and routes each input through an evidential gate that jointly models sufficiency and its epistemic uncertainty. SIEVE is repair-agnostic: it operates as a plug-and-play decision on top of any explicit or implicit repair module, without modifying its internal design. Experiments on CMU-MOSI and IEMOCAP show that SIEVE consistently improves representative repair backbones across evaluated missing rates, and approaches the per-sample dual-branch achievable optimum.

cs.CL↗

arXiv2026

Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking

Large language models (LLMs) are increasingly applied to ranking tasks in retrieval and recommendation. Although reasoning prompting can enhance ranking utility, our preliminary exploration reveals that its benefits are inconsistent and come at a substantial computational cost, suggesting that when to reason is as crucial as how to reason. To address this issue, we propose a reasoning routing framework that employs a lightweight, plug-and-play router head to decide whether to use direct inference (Non-Think) or reasoning (Think) for each instance before generation. The router head relies solely on pre-generation signals: i) compact ranking-aware features (e.g., candidate dispersion) and ii) model-aware difficulty signals derived from a diagnostic checklist reflecting the model's estimated need for reasoning. By leveraging these features before generation, the router outputs a controllable token that determines whether to apply the Think mode. Furthermore, the router can adaptively select its operating policy along the validation Pareto frontier during deployment, enabling dynamic allocation of computational resources toward instances most likely to benefit from Think under varying system constraints. Experiments on three public ranking datasets with different scales of open-source LLMs show consistent improvements in ranking utility with reduced token consumption (e.g., +6.3\% NDCG@10 with -49.5\% tokens on MovieLens with Qwen3-4B), demonstrating reasoning routing as a practical solution to the accuracy-efficiency trade-off.

cs.IR↗