Search arXiv⌕ Search

arXiv subjects

Xiao Chen

Publications and source records attributed to Xiao Chen.

At least 73 records · Page 4Linked to original sources

Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanism

Long video understanding is a key challenge that plagues the advancement of \emph{Multimodal Large language Models} (MLLMs). In this paper, we study this problem from the perspective of visual memory mechanism, and proposed a novel and training-free approach, termed \emph{Flexible Memory} (\textbf{FlexMem}). In principle, FlexMem aims to mimic human behavior of video watching, \emph{i.e.}, continually watching video content and recalling the most relevant memory fragments to answer the question. In this way, FlexMem can help MLLMs achieve video understanding of infinite lengths, unlike previous methods that process all video information at once and have input upper-limit. Concretely, FlexMem first consider the visual KV caches as the memory sources, and realize the effective memory transfer and writing via a dual-pathway compression design. Afterwards, FlexMem also explores different memory reading strategies for the diverse video understanding tasks, including the popular streaming one. To validate FlexMem, we apply it to two popular video-MLLMs, and conduct extensive experiments on five long video and one streaming video task. The experimental results show that on \textbf{a single 3090 GPU}, our FlexMem can achieve obvious improvements than existing efficient video understanding methods and process more than \textbf{1k frames}, which also helps the base MLLMs achieve comparable or even better performance than SOTA MLLMs on some benchmarks, \emph{e.g.} , GPT-4o and Gemini-1.5 Pro.

cs.CV↗

Class-Imbalanced-Aware Adaptive Dataset Distillation for Scalable Pretrained Model on Credit Scoring

The advent of artificial intelligence has significantly enhanced credit scoring technologies. Despite the remarkable efficacy of advanced deep learning models, mainstream adoption continues to favor tree-structured models due to their robust predictive performance on tabular data. Although pretrained models have seen considerable development, their application within the financial realm predominantly revolves around question-answering tasks and the use of such models for tabular-structured credit scoring datasets remains largely unexplored. Tabular-oriented large models, such as TabPFN, has made the application of large models in credit scoring feasible, albeit can only processing with limited sample sizes. This paper provides a novel framework to combine tabular-tailored dataset distillation technique with the pretrained model, empowers the scalability for TabPFN. Furthermore, though class imbalance distribution is the common nature in financial datasets, its influence during dataset distillation has not been explored. We thus integrate the imbalance-aware techniques during dataset distillation, resulting in improved performance in financial datasets (e.g., a 2.5% enhancement in AUC). This study presents a novel framework for scaling up the application of large pretrained models on financial tabular datasets and offers a comparative analysis of the influence of class imbalance on the dataset distillation process. We believe this approach can broaden the applications and downstream tasks of large models in the financial domain.

cs.LG↗

GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning

Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable perceptual and reasoning abilities. However, they struggle to perceive fine-grained geometric structures, constraining their ability of geometric understanding and visual reasoning. To address this, we propose GeoTikzBridge, a framework that enhances local geometric perception and visual reasoning through tikz-based code generation. Within this framework, we build two models supported by two complementary datasets. The GeoTikzBridge-Base model is trained on GeoTikz-Base dataset, the largest image-to-tikz dataset to date with 2.5M pairs (16 $\times$ larger than existing open-sourced datasets). This process is achieved via iterative data expansion and a localized geometric transformation strategy. Subsequently, GeoTikzBridge-Instruct is fine-tuned on GeoTikz-Instruct dataset which is the first instruction-augmented tikz dataset supporting visual reasoning. Extensive experimental results demonstrate that our models achieve state-of-the-art performance among open-sourced MLLMs. Furthermore, GeoTikzBridge models can serve as plug-and-play reasoning modules for any MLLM(LLM), enhancing reasoning performance in geometric problem-solving. Datasets and codes are publicly available at: https://github.com/sjy-1995/GeoTikzBridge.

cs.CV↗

Recoil corrections to pentaquark molecules with an SU(3) anti-triplet heavy baryon

Recoil corrections, which appear at order $\mathcal{O}(\frac{1}{M})$, turn out to be crucial for the pentaquark molecules with heavy flavor. In the past, such corrections were typically regarded as negligible for the formation of heavy molecules. However, our research manifests the important impacts on the spectra of possible baryon-meson bound states. In certain cases, the binding energy sharply decreases after considering the recoil corrections. Using the one-boson-exchange (OBE) model, we have studied a series of double heavy and hidden heavy pentaquark states with an SU(3) anti-triplet baryon, including $Ξ_{c}\bar{D}^{(*)}$, $Λ_{c}\bar{D}^{(*)}$, $Ξ_{c}D^{(*)}$, $Λ_{c}D^{(*)}$, along with their bottom analogues. Considering both the $S$\--{}$D$ wave mixing effect and the recoil corrections, we find that recoil corrections, working against the stability of bound states in certain isospin-singlet channels, which cannot be ignored. For the $Ξ_{c}\bar{D}$ and $Ξ_{c}D$ systems with $I(J^P)=0(\frac{1}{2}^{-})$, the $Ξ_{c}\bar{D}^{*}$ and $Ξ_{c}D^{*}$ systems, as well as their bottom counterparts, with $I(J^P)=0(\frac{1}{2}^{-})$ and $0(\frac{3}{2}^{-})$, and the $Λ_{b}\bar{B}^{*}$ system with $I(J^P)=\frac{1}{2}(\frac{1}{2}^{-})$ and $\frac{1}{2}(\frac{3}{2}^{-})$, although the bound state solution can be found both with and without considering the recoil corrections, the inclusion of recoil corrections weakens the attraction of the molecules. This phenomenon arising from recoil corrections is especially prominent in the $Ξ_{c}\bar{D}^{*}$ and $Ξ_{c}D^{*}$ systems.

hep-ph↗

Boundary Perturbation Effects in Quantum Systems with Conserved Energy and Continuous Symmetry

We investigate one-dimensional systems with both energy conservation and a continuous symmetry, focusing on the impact of a boundary perturbation that breaks the continuous symmetry. Our study reveals two distinct dynamical phases: one in which the corresponding charge exhibits extensive fluctuations, and another where the charge remains conserved. These phases appear in both free and interacting models. We interpret this behavior through a boundary-induced pumping mechanism, which estimates the amplitude connecting two degenerate states from different charge sectors via a local charge-non-conserving operator. In the Floquet setting, we show that the frozen phase can survive at high driving frequencies but vanishes at low frequencies. This phenomenon is exact in free-fermion systems in the thermodynamic limit, but in interacting systems it appears only at finite system size. The emergence of the charge-frozen phase is attributed to effective energy conservation, and we demonstrate that this phase disappears when effective energy conservation is broken or replaced by other symmetries.

quant-ph↗

Critical behaviors of magic and participation entropy at measurement induced phase transitions

We study the participation and stabilizer entropy of non-unitary quantum circuit dynamics, focusing on the critical line that separates the low-entanglement spin-glass phase and the paramagnetic phase. Along this critical line, the entanglement has a logarithmic scaling, which enables us to access the critical regime using large-scale matrix product state simulations with modest bond dimension. We find that both the participation entropy and stabilizer entropy exhibit critical slowing down: their saturation time scales linearly with the system size, in stark contrast to purely unitary dynamics, where saturation occurs on logarithmic time scales. In addition, we study bipartite participation and stabilizer mutual information, and find that it shows similar scaling behavior to the entanglement entropy. Finally, by analyzing the participation entropy of several paradigmatic Clifford circuits, we identify similar slow dynamical behavior near their respective critical points.

quant-ph↗

Speech-Omni-Lite: Portable Speech Interfaces for Vision-Language Models

While large-scale omni-models have demonstrated impressive capabilities across various modalities, their strong performance heavily relies on massive multimodal data and incurs substantial computational costs. This work introduces Speech-Omni-Lite, a cost-efficient framework for extending pre-trained Visual-Language (VL) backbones with speech understanding and generation capabilities, while fully preserving the backbones' vision-language performance. Specifically, the VL backbone is equipped with two lightweight, trainable plug-and-play modules, a speech projector and a speech token generator, while keeping the VL backbone fully frozen. To mitigate the scarcity of spoken QA corpora, a low-cost data construction strategy is proposed to generate Question-Text Answer-Text-Speech (QTATS) data from existing ASR speech-text pairs, facilitating effective speech generation training. Experimental results show that, even with only thousands of hours of speech training data, Speech-Omni-Lite achieves excellent spoken QA performance, which is comparable to omni-models trained on millions of hours of speech data. Furthermore, the learned speech modules exhibit strong transferability across VL backbones.

eess.AS↗

Different polarized components of the quasar 3C 286 revealed by FAST

3C 286, a well-known radio calibrator, exhibits the stability in both of total flux density (FD) and polarization parameters. However, its stable and luminous interstellar radio signal may encounter interplanetary scintillation (IPS) due to density irregularities in the solar wind within the heliosphere. In this work, we analyze high-time-resolution observations of 3C 286 obtained with the Five-hundred-meter Aperture Spherical radio Telescope (FAST) from 2019 to 2023. Our analysis reveals that IPS affects the polarized flux densities of the Stokes I, Q, and U parameters, whereas Stokes V shows no detectable IPS-induced variations. The IPS variations detected in Stokes I are synchronous with those in Stokes U, while those in Stokes Q exhibit greater randomness. The crosscorrelation function (CCF) results indicate no time delay between Stokes I and U, but a delay of approximately 2.8 seconds between Stokes I and Q. This suggests that the different polarized radio emissions of 3C 286 originate from distinct emission regions, specifically the core and the southwestern jet. Furthermore, the projections of the radio core and jet component onto the scintillation screen at 1 AU yield a solar wind plasma speed of $\sim 637$ km/s.

astro-ph.HE↗

Learning a Shape-adaptive Assist-as-needed Rehabilitation Policy from Therapist-informed Input

Therapist-in-the-loop robotic rehabilitation has shown great promise in enhancing rehabilitation outcomes by integrating the strengths of therapists and robotic systems. However, its broader adoption remains limited due to insufficient safe interaction and limited adaptation capability. This article proposes a novel telerobotics-mediated framework that enables therapists to intuitively and safely deliver assist-as-needed~(AAN) therapy based on two primary contributions. First, our framework encodes the therapist-informed corrective force into via-points in a latent space, allowing the therapist to provide only minimal assistance while encouraging patient maintaining own motion preferences. Second, a shape-adaptive ANN rehabilitation policy is learned to partially and progressively deform the reference trajectory for movement therapy based on encoded patient motion preferences and therapist-informed via-points. The effectiveness of the proposed shape-adaptive AAN strategy was validated on a telerobotic rehabilitation system using two representative tasks. The results demonstrate its practicality for remote AAN therapy and its superiority over two state-of-the-art methods in reducing corrective force and improving movement smoothness.

eess.SY↗

A Linear Bound on the Diameter of the Kakimizu Complex for Hyperbolic Knots

This paper focuses on the Kakimizu complex of a hyperbolic knot $K$. We define a complex $IS_\ell(K)$ to study incompressible Seifert surfaces of genus at most $\ell$, and prove that it is connected and that its diameter admits a linear upper bound in terms of $\ell$. As a corollary, we show that the diameter of the Kakimizu complex $MS(K)$ of a hyperbolic knot grows linearly with the genus $g$, confirming a conjecture of Sakuma--Shackleton. More precisely, it is bounded above by $6g-4$.

math.GT↗

RADAR: Benchmarking Vision-Language-Action Generalization via Real-World Dynamics, Spatial-Physical Intelligence, and Autonomous Evaluation

VLA models have achieved remarkable progress in embodied intelligence; however, their evaluation remains largely confined to simulations or highly constrained real-world settings. This mismatch creates a substantial reality gap, where strong benchmark performance often masks poor generalization in diverse physical environments. We identify three systemic shortcomings in current benchmarking practices that hinder fair and reliable model comparison. (1) Existing benchmarks fail to model real-world dynamics, overlooking critical factors such as dynamic object configurations, robot initial states, lighting changes, and sensor noise. (2) Current protocols neglect spatial--physical intelligence, reducing evaluation to rote manipulation tasks that do not probe geometric reasoning. (3) The field lacks scalable fully autonomous evaluation, instead relying on simplistic 2D metrics that miss 3D spatial structure or on human-in-the-loop systems that are costly, biased, and unscalable. To address these limitations, we introduce RADAR (Real-world Autonomous Dynamics And Reasoning), a benchmark designed to systematically evaluate VLA generalization under realistic conditions. RADAR integrates three core components: (1) a principled suite of physical dynamics; (2) dedicated tasks that explicitly test spatial reasoning and physical understanding; and (3) a fully autonomous evaluation pipeline based on 3D metrics, eliminating the need for human supervision. We apply RADAR to audit multiple state-of-the-art VLA models and uncover severe fragility beneath their apparent competence. Performance drops precipitously under modest physical dynamics, with the expectation of 3D IoU declining from 0.261 to 0.068 under sensor noise. Moreover, models exhibit limited spatial reasoning capability. These findings position RADAR as a necessary bench toward reliable and generalizable real-world evaluation of VLA models.

cs.RO↗

PROST-LLM: Progressively Enhancing the Speech-to-Speech Translation Capability in LLMs

Although Large Language Models (LLMs) excel in many tasks, their application to Speech-to-Speech Translation (S2ST) is underexplored and hindered by data scarcity. To bridge this gap, we propose PROST-LLM (PROgressive Speech-to-speech Translation) to enhance the S2ST capabilities in LLMs progressively. First, we fine-tune the LLMs with the CVSS corpus, employing designed tri-task learning and chain of modality methods to boost the initial performance. Then, leveraging the fine-tuned model, we generate preference pairs through self-sampling and back-translation without human evaluation. Finally, these preference pairs are used for preference optimization to enhance the model's S2ST capability further. Extensive experiments confirm the effectiveness of our proposed PROST-LLM in improving the S2ST capability of LLMs.

cs.CL↗

A predictive modular approach to constraint satisfaction under uncertainty -- with application to glycosylation in continuous monoclonal antibody biosimilar production

The paper proposes a modular-based approach to constraint handling in process optimization and control. This is partly motivated by the recent interest in learning-based methods, e.g., within bioproduction, for which constraint handling under uncertainty is a challenge. The proposed constraint handler, called predictive filter, is combined with an adaptive constraint margin and a constraint violation cost monitor to minimize the cost of violating soft constraints due to model uncertainty and disturbances. The module can be combined with any controller and is based on minimally modifying the controller output, in a least squares sense, such that constraints are satisfied within the considered horizon. The proposed method is computationally efficient and suitable for real-time applications. The effectiveness of the method is illustrated through a realistic case study of glycosylation constraint satisfaction in continuous monoclonal antibody biosimilar production using Chinese hamster ovary cells, employing a metabolic network model consisting of 23 extracellular metabolites and 126 reactions. In the case study, the average constraint-violation cost is reduced by more than 60% compared to the case without the proposed constraint-handling method.

eess.SY↗

Context-Free Grammar Inference for Complex Programming Languages in Black Box Settings

Grammar inference for complex programming languages remains a significant challenge, as existing approaches fail to scale to real world datasets within practical time constraints. In our experiments, none of the state-of-the-art tools, including Arvada, Treevada and Kedavra were able to infer grammars for complex languages such as C, C++, and Java within 48 hours. Arvada and Treevada perform grammar inference directly on full-length input examples, which proves inefficient for large files commonly found in such languages. While Kedavra introduces data decomposition to create shorter examples for grammar inference, its lexical analysis still relies on the original inputs. Additionally, its strict no-overgeneralization constraint limits the construction of complex grammars. To overcome these limitations, we propose Crucio, which builds a decomposition forest to extract short examples for lexical and grammar inference via a distributional matrix. Experimental results show that Crucio is the only method capable of successfully inferring grammars for complex programming languages (where the number of nonterminals is up to 23x greater than in prior benchmarks) within reasonable time limits. On the prior simple benchmark, Crucio achieves an average recall improvement of 1.37x and 1.19x over Treevada and Kedavra, respectively, and improves F1 scores by 1.21x and 1.13x.

cs.PL↗

Optimizing View Change for Byzantine Fault Tolerance in Parallel Consensus

The parallel Byzantine Fault Tolerant (BFT) protocol is viewed as a promising solution to address the consensus scalability issue of the permissioned blockchain. One of the main challenges in parallel BFT is the view change process that happens when the leader node fails, which can lead to performance bottlenecks. Existing parallel BFT protocols typically rely on passive view change mechanisms with blind leader rotation. Such approaches frequently select unavailable or slow nodes as leaders, resulting in degraded performance. To address these challenges, we propose a View Change Optimization (VCO) model based on mixed integer programming that optimizes leader selection and follower reassignment across parallel committees by considering communication delays and failure scenarios. We applied a decomposition method with efficient subproblems and improved benders cuts to solve the VCO model. Leveraging the results of improved decomposition solution method, we propose an efficient iterative backup leader selection algorithm as views proceed. By performing experiments in Microsoft Azure cloud environments, we demonstrate that the VCO-driven parallel BFT outperforms existing configuration methods under both normal operation and faulty condition. The results show that the VCO model is effective as network size increases, making it a suitable solution for high-performance parallel BFT systems.

cs.DC↗

Simulating decoherence of two coupled spins using the generalized cluster correlation expansion

We simulate the coherence of two coupled electron spins interacting with a bath of nuclei using the generalized cluster correlation expansion (gCCE) method. An exchange interaction between the electrons facilitates a family of entangling gates that can be spoiled by nuclear-induced dephasing. Consequently, we study the dephasing of the coherent two-electron system by characterizing the $T_2$ and $T_2^*$ of the two-electron reduced density matrix for various system parameters in the range mimicking magnetic molecules, including magnetic field strength and orientation, exchange interaction strength, distance between the two spins, minimum distance between electron and nuclei and between nuclei, and nuclei density. We find the optimal regime for each parameter in which the coherence time is maximized and provide a physical understanding of it.

quant-ph↗

Multi-Agent Collaborative Reward Design for Enhancing Reasoning in Reinforcement Learning

We present CRM (Multi-Agent Collaborative Reward Model), a framework that replaces a single black-box reward model with a coordinated team of specialist evaluators to improve robustness and interpretability in RLHF. Conventional reward models struggle to jointly optimize multiple, sometimes conflicting, preference dimensions (e.g., factuality, helpfulness, safety) and offer limited transparency into why a score is assigned. CRM addresses these issues by decomposing preference evaluation into domain-specific agents that each produce partial signals, alongside global evaluators such as ranker-based and embedding-similarity rewards. A centralized aggregator fuses these signals at each timestep, balancing factors like step-wise correctness, multi-agent agreement, and repetition penalties, yielding a single training reward compatible with standard RL pipelines. The policy is optimized with advantage-based updates (e.g., GAE), while a value model regresses to the aggregated reward, enabling multi-perspective reward shaping without requiring additional human annotations beyond those used to train the evaluators. To support training and assessment, we introduce rewardBench, a benchmark and training suite aligned with the collaborative structure of CRM. Together, CRM and rewardBench provide a practical, modular path to more transparent reward modeling and more stable optimization.

cs.AI↗

LLHA-Net: A Hierarchical Attention Network for Two-View Correspondence Learning

Establishing the correct correspondence of feature points is a fundamental task in computer vision. However, the presence of numerous outliers among the feature points can significantly affect the matching results, reducing the accuracy and robustness of the process. Furthermore, a challenge arises when dealing with a large proportion of outliers: how to ensure the extraction of high-quality information while reducing errors caused by negative samples. To address these issues, in this paper, we propose a novel method called Layer-by-Layer Hierarchical Attention Network, which enhances the precision of feature point matching in computer vision by addressing the issue of outliers. Our method incorporates stage fusion, hierarchical extraction, and an attention mechanism to improve the network's representation capability by emphasizing the rich semantic information of feature points. Specifically, we introduce a layer-by-layer channel fusion module, which preserves the feature semantic information from each stage and achieves overall fusion, thereby enhancing the representation capability of the feature points. Additionally, we design a hierarchical attention module that adaptively captures and fuses global perception and structural semantic information using an attention mechanism. Finally, we propose two architectures to extract and integrate features, thereby improving the adaptability of our network. We conduct experiments on two public datasets, namely YFCC100M and SUN3D, and the results demonstrate that our proposed method outperforms several state-of-the-art techniques in both outlier removal and camera pose estimation. Source code is available at http://www.linshuyuan.com.

cs.CV↗