arXiv subjects

Chunyu Li

Publications and source records attributed to Chunyu Li.

At least 37 records · Page 2Linked to original sources

arXiv2026

VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion

Compared to images, videos better reflect real-world acquisition and possess valuable temporal cues. However, existing multi-sensor fusion research predominantly integrates complementary context from multiple images rather than videos due to the scarcity of large-scale multi-sensor video datasets, limiting research in video fusion and the inherent difficulty of jointly modeling spatial and temporal dependencies in a unified framework. To this end, we construct M3SVD, a benchmark dataset with $220$ temporally synchronized and spatially registered infrared-visible videos comprising $153,797$ frames, bridging the data gap. Secondly, we propose VideoFusion, a multi-modal video fusion model that exploits cross-modal complementarity and temporal dynamics to generate spatio-temporally coherent videos from multi-modal inputs. Specifically, 1) a differential reinforcement module is developed for cross-modal information interaction and enhancement, 2) a complete modality-guided fusion strategy is employed to adaptively integrate multi-modal features, and 3) a bi-temporal co-attention mechanism is devised to dynamically aggregate forward-backward temporal contexts to reinforce cross-frame feature representations. Experiments reveal that VideoFusion outperforms existing image-oriented fusion paradigms in sequences, effectively mitigating temporal inconsistency and interference. Project and M3SVD: https://github.com/Linfeng-Tang/VideoFusion.

cs.CV↗

arXiv2026

HomoFM: Deep Homography Estimation with Flow Matching

Deep homography estimation has broad applications in computer vision and robotics. Remarkable progresses have been achieved while the existing methods typically treat it as a direct regression or iterative refinement problem and often struggling to capture complex geometric transformations or generalize across different domains. In this work, we propose HomoFM, a new framework that introduces the flow matching technique from generative modeling into the homography estimation task for the first time. Unlike the existing methods, we formulate homography estimation problem as a velocity field learning problem. By modeling a continuous and point-wise velocity field that transforms noisy distributions into registered coordinates, the proposed network recovers high-precision transformations through a conditional flow trajectory. Furthermore, to address the challenge of domain shifts issue, e.g., the cases of multimodal matching or varying illumination scenarios, we integrate a gradient reversal layer (GRL) into the feature extraction backbone. This domain adaptation strategy explicitly constrains the encoder to learn domain-invariant representations, significantly enhancing the network's robustness. Extensive experiments demonstrate the effectiveness of the proposed method, showing that HomoFM outperforms state-of-the-art methods in both estimation accuracy and robustness on standard benchmarks. Code and data resource are available at https://github.com/hmf21/HomoFM.

cs.CV↗

arXiv2026

Predictive models for strain energy in condensed phase reactions

Molecular modeling of thermally activated chemistry in condensed phases is essential to understand polymerization, depolymerization, and other processing steps of molecular materials. Current methods typically combine molecular dynamics (MD) simulations to describe short-time relaxation with a stochastic description of predetermined chemical reactions. Possible reactions are often selected on the basis of geometric criteria, such as a capture distance between reactive atoms. Although these simulations have provided valuable insight, the approximations used to determine possible reactions often lead to significant molecular strain and unrealistic structures. We show that the local molecular environment surrounding the reactive site plays a crucial role in determining the resulting molecular strain energy and, in turn, the associated reaction rates. We develop a graph neural network capable of predicting the strain energy associated with a cyclization reaction from the pre-reaction, local, molecular environment surrounding the reactive site. The model is trained on a large dataset of condensed-phase reactions during the activation of polyacrylonitrile (PAN) obtained from MD simulations and can be used to adjust relative reaction rates in condensed systems and advance our understanding of thermally activated chemical processes in complex materials

cond-mat.mtrl-sci↗

arXiv2026

Multi-Fidelity Predictive Model for Shock Response of Energetic Materials Using Conditional U-Net

Mapping microstructure to properties is central to materials science. Perhaps most famously, the Hall-Petch relationship relates average grain size to strength. More challenging has been deriving relationships for properties that depend on subtle microstructural features and not average properties. One such example is the initiation of energetic materials under dynamical loading, dominated by energy localization on microstructural features such as pores, cracks, and interfaces. We propose a conditional convolutional neural network to predict the shock-induced temperature field as a function of shock strength, for a wide range of microstructures, and obtained via two different simulation methods. The proposed model, denoted MISTnet2, significantly extends prior work that was limited to a single shock strength, model, and type of microstructure. MISTnet2 can contribute to bridging atomistics with coarse-grain simulations and enable first principles predictions of detonation initiation and safety of this class of materials.

cond-mat.mtrl-sci↗

arXiv2025

MemOS: A Memory OS for AI System

Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the development of long-context reasoning, continual personalization, and knowledge consistency.Existing models mainly rely on static parameters and short-lived contextual states, limiting their ability to track user preferences or update knowledge over extended periods.While Retrieval-Augmented Generation (RAG) introduces external knowledge in plain text, it remains a stateless workaround without lifecycle control or integration with persistent representations.Recent work has modeled the training and inference cost of LLMs from a memory hierarchy perspective, showing that introducing an explicit memory layer between parameter memory and external retrieval can substantially reduce these costs by externalizing specific knowledge. Beyond computational efficiency, LLMs face broader challenges arising from how information is distributed over time and context, requiring systems capable of managing heterogeneous knowledge spanning different temporal scales and sources. To address this challenge, we propose MemOS, a memory operating system that treats memory as a manageable system resource. It unifies the representation, scheduling, and evolution of plaintext, activation-based, and parameter-level memories, enabling cost-efficient storage and retrieval. As the basic unit, a MemCube encapsulates both memory content and metadata such as provenance and versioning. MemCubes can be composed, migrated, and fused over time, enabling flexible transitions between memory types and bridging retrieval with parameter-based learning. MemOS establishes a memory-centric system framework that brings controllability, plasticity, and evolvability to LLMs, laying the foundation for continual learning and personalized modeling.

cs.CL↗

arXiv2025

Gaussian-process reconstructions and model building of quintom dark energy from latest cosmological observations

In this article we use the latest cosmological observations, including SNe, BAO, CC and RSD, to reconstruct the cosmological evolution via the Gaussian process. At the background level, we find consistency with the quintom dynamics for different data combinations and divide the characteristics of dark energy into three different categories, which are negative-energy dark energy, late-dominated dark energy and oscillating dark energy, respectively. Considering the effect of modified gravity on the growth of matter perturbations, the reconstruction results at the perturbative level show that we only need minor corrections to general relativity. Furthermore, we provide theoretical interpretation for the three different types of dynamical dark-energy behavior, in the framework of modified gravity, scalar fields, and dark-energy equation-of-state parametrizations. Finally, we show that all of these models can be unified in the framework of effective field theory.

astro-ph.CO↗

arXiv2025

TWC-SLAM: Multi-Agent Cooperative SLAM with Text Semantics and WiFi Features Integration for Similar Indoor Environments

Multi-agent cooperative SLAM often encounters challenges in similar indoor environments characterized by repetitive structures, such as corridors and rooms. These challenges can lead to significant inaccuracies in shared location identification when employing point cloud-based techniques. To mitigate these issues, we introduce TWC-SLAM, a multi-agent cooperative SLAM framework that integrates text semantics and WiFi signal features to enhance location identification and loop closure detection. TWC-SLAM comprises a single-agent front-end odometry module based on FAST-LIO2, a location identification and loop closure detection module that leverages text semantics and WiFi features, and a global mapping module. The agents are equipped with sensors capable of capturing textual information and detecting WiFi signals. By correlating these data sources, TWC-SLAM establishes a common location, facilitating point cloud alignment across different agents' maps. Furthermore, the system employs loop closure detection and optimization modules to achieve global optimization and cohesive mapping. We evaluated our approach using an indoor dataset featuring similar corridors, rooms, and text signs. The results demonstrate that TWC-SLAM significantly improves the performance of cooperative SLAM systems in complex environments with repetitive architectural features.

cs.RO↗

arXiv2025

BIBERT-Pipe on Biomedical Nested Named Entity Linking at BioASQ 2025

Entity linking (EL) for biomedical text is typically benchmarked on English-only corpora with flat mentions, leaving the more realistic scenario of nested and multilingual mentions largely unexplored. We present our system for the BioNNE 2025 Multilingual Biomedical Nested Named Entity Linking shared task (English & Russian), closing this gap with a lightweight pipeline that keeps the original EL model intact and modifies only three task-aligned components: Two-stage retrieval-ranking. We leverage the same base encoder model in both stages: the retrieval stage uses the original pre-trained model, while the ranking stage applies domain-specific fine-tuning. Boundary cues. In the ranking stage, we wrap each mention with learnable [Ms] / [Me] tags, providing the encoder with an explicit, language-agnostic span before robustness to overlap and nesting. Dataset augmentation. We also automatically expand the ranking training corpus with three complementary data sources, enhancing coverage without extra manual annotation. On the BioNNE 2025 leaderboard, our two stage system, bilingual bert (BIBERT-Pipe), ranks third in the multilingual track, demonstrating the effectiveness and competitiveness of these minimal yet principled modifications. Code are publicly available at https://github.com/Kaggle-Competitions-Code/BioNNE-L.

cs.CL↗

arXiv2025

Xinyu AI Search: Enhanced Relevance and Comprehensive Results with Rich Answer Presentations

Traditional search engines struggle to synthesize fragmented information for complex queries, while generative AI search engines face challenges in relevance, comprehensiveness, and presentation. To address these limitations, we introduce Xinyu AI Search, a novel system that incorporates a query-decomposition graph to dynamically break down complex queries into sub-queries, enabling stepwise retrieval and generation. Our retrieval pipeline enhances diversity through multi-source aggregation and query expansion, while filtering and re-ranking strategies optimize passage relevance. Additionally, Xinyu AI Search introduces a novel approach for fine-grained, precise built-in citation and innovates in result presentation by integrating timeline visualization and textual-visual choreography. Evaluated on recent real-world queries, Xinyu AI Search outperforms eight existing technologies in human assessments, excelling in relevance, comprehensiveness, and insightfulness. Ablation studies validate the necessity of its key sub-modules. Our work presents the first comprehensive framework for generative AI search engines, bridging retrieval, generation, and user-centric presentation.

cs.IR↗

arXiv2025

Spall strength of symmetric tilt grain boundaries in 6H silicon carbide

Characterizing microstructural effects on the dynamical response of materials is challenging due to the extreme conditions and the short timescales involved. For example, little is known about how grain boundary characteristics affect spall strength. This study explores 6H-SiC bicrystals under shock waves via large-scale molecular dynamics simulations. We focused on symmetric tilt grain boundaries with a wide range of misorientations and found that spall strength and dynamical fracture surface energy are strongly affected by the grain boundary microstructure, especially the excess free volume. Grain boundary energy also plays a considerable role. As expected, low-angle grain boundaries tend to have higher spallation strengths. We also extracted cohesive models for the dynamical strength of bulk systems and grain boundaries that can be used in continuum simulations.

cond-mat.mtrl-sci↗

arXiv2025

Quantum Gravity Meets DESI: Dynamical Dark Energy in Light of the Trans-Planckian Censorship Conjecture

Recent DESI DR2 observations indicate that dark energy has crossed from phantom to quintessence regime, a behavior known as the quintom-B realization. In this work we constrain dynamical dark energy and modified gravity using the swampland Trans-Planckian Censorship Conjecture (TCC), which forbids eternal acceleration since in this case any trans-Planckian quantum fluctuation would eventually stretch beyond the Hubble radius, breaking the applicability of any effective field theory and cosmological techniques. By combining DESI DR2 data with the TCC criterion, we impose tight constraints on the dark energy equation of state and its parameter space in scenarios such as the Chevallier-Polarski-Linder, Barboza-Alcaniz, Jassal-Bagla-Padmanabhan, EXP and LOG parameterizations, significantly constraining the quintom-A behavior. Also we examine models within the framework of $f(T)$ and $f(Q)$ modified gravity theories, demonstrating that TCC is very powerful to constrain or exclude them, a result that indicates the necessity to consider infrared modifications on General Relativity apart from the usual ultraviolet ones. Our findings imply that viable dynamical dark energy scenarios must asymptotically transit to deceleration, shedding light on new physics consistent with both cosmological observations and quantum gravity principles.

astro-ph.CO↗

arXiv2025

DSPFusion: Image Fusion via Degradation and Semantic Dual-Prior Guidance

Existing fusion methods are tailored for high-quality images but struggle with degraded images captured under harsh circumstances, thus limiting the practical potential of image fusion. This work presents a \textbf{D}egradation and \textbf{S}emantic \textbf{P}rior dual-guided framework for degraded image \textbf{Fusion} (\textbf{DSPFusion}), utilizing degradation priors and high-quality scene semantic priors restored via diffusion models to guide both information recovery and fusion in a unified model. In specific, it first individually extracts modality-specific degradation priors, while jointly capturing comprehensive low-quality semantic priors. Subsequently, a diffusion model is developed to iteratively restore high-quality semantic priors in a compact latent space, enabling our method to be over $20 \times$ faster than mainstream diffusion model-based image fusion schemes. Finally, the degradation priors and high-quality semantic priors are employed to guide information enhancement and aggregation via the dual-prior guidance and prior-guided fusion modules. Extensive experiments demonstrate that DSPFusion mitigates most typical degradations while integrating complementary context with minimal computational cost, greatly broadening the application scope of image fusion.

cs.CV↗

arXiv2025

LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervision

End-to-end audio-conditioned latent diffusion models (LDMs) have been widely adopted for audio-driven portrait animation, demonstrating their effectiveness in generating lifelike and high-resolution talking videos. However, direct application of audio-conditioned LDMs to lip-synchronization (lip-sync) tasks results in suboptimal lip-sync accuracy. Through an in-depth analysis, we identified the underlying cause as the "shortcut learning problem", wherein the model predominantly learns visual-visual shortcuts while neglecting the critical audio-visual correlations. To address this issue, we explored different approaches for integrating SyncNet supervision into audio-conditioned LDMs to explicitly enforce the learning of audio-visual correlations. Since the performance of SyncNet directly influences the lip-sync accuracy of the supervised model, the training of a well-converged SyncNet becomes crucial. We conducted the first comprehensive empirical studies to identify key factors affecting SyncNet convergence. Based on our analysis, we introduce StableSyncNet, with an architecture designed for stable convergence. Our StableSyncNet achieved a significant improvement in accuracy, increasing from 91% to 94% on the HDTF test set. Additionally, we introduce a novel Temporal Representation Alignment (TREPA) mechanism to enhance temporal consistency in the generated videos. Experimental results show that our method surpasses state-of-the-art lip-sync approaches across various evaluation metrics on the HDTF and VoxCeleb2 datasets.

cs.CV↗

arXiv2025

Exploring Embodied Multimodal Large Models: Development, Datasets, and Future Directions

Embodied multimodal large models (EMLMs) have gained significant attention in recent years due to their potential to bridge the gap between perception, cognition, and action in complex, real-world environments. This comprehensive review explores the development of such models, including Large Language Models (LLMs), Large Vision Models (LVMs), and other models, while also examining other emerging architectures. We discuss the evolution of EMLMs, with a focus on embodied perception, navigation, interaction, and simulation. Furthermore, the review provides a detailed analysis of the datasets used for training and evaluating these models, highlighting the importance of diverse, high-quality data for effective learning. The paper also identifies key challenges faced by EMLMs, including issues of scalability, generalization, and real-time decision-making. Finally, we outline future directions, emphasizing the integration of multimodal sensing, reasoning, and action to advance the development of increasingly autonomous systems. By providing an in-depth analysis of state-of-the-art methods and identifying critical gaps, this paper aims to inspire future advancements in EMLMs and their applications across diverse domains.

cs.RO↗

arXiv2024

Influence of Polymer on Shock-Induced Pore Collapse: Hotspot Criticality through Reactive Molecular Dynamics

The shock initiation of energetic materials is mediated by the localization of mechanical energy into hotspots. These originate through the interaction of the shock and material microstructure; the most potent hotspots are formed by the collapse of porosity. Recent work using molecular dynamics (MD) has shed light on the molecular mechanisms responsible for the shock-to-deflagration transition following pore collapse in pure energetic materials. However, explosive formulations are composites of energetic crystals and a polymer binder, which differs from the prior focus on pure materials. The role of polymer phases on hotspot formation and its criticality is not well-understood. We use reactive MD simulations to investigate the role of polystyrene and polyvinyl nitrate films around pores in the shock-induced pore collapse of RDX. The polymer affects the hotspots' temperature and their criticality. While the presence of inert polymer often delays or hinders chemical reactions of the energetic material, certain geometries accelerate chemistry. The simulations provide a mechanistic understanding of these phenomena.

cond-mat.mtrl-sci↗

arXiv2023

HEViTPose: High-Efficiency Vision Transformer for Human Pose Estimation

Human pose estimation in complicated situations has always been a challenging task. Many Transformer-based pose networks have been proposed recently, achieving encouraging progress in improving performance. However, the remarkable performance of pose networks is always accompanied by heavy computation costs and large network scale. In order to deal with this problem, this paper proposes a High-Efficiency Vision Transformer for Human Pose Estimation (HEViTPose). In HEViTPose, a Cascaded Group Spatial Reduction Multi-Head Attention Module (CGSR-MHA) is proposed, which reduces the computational cost through feature grouping and spatial degradation mechanisms, while preserving feature diversity through multiple low-dimensional attention heads. Moreover, a concept of Patch Embedded Overlap Width (PEOW) is defined to help understand the relationship between the amount of overlap and local continuity. By optimising PEOW, our model gains improvements in performance, parameters and GFLOPs. Comprehensive experiments on two benchmark datasets (MPII and COCO) demonstrate that the small and large HEViTPose models are on par with state-of-the-art models while being more lightweight. Specifically, HEViTPose-B achieves 90.7 PCK@0.5 on the MPII test set and 72.6 AP on the COCO test-dev2017 set. Compared with HRNet-W32 and Swin-S, our HEViTPose-B significantly reducing Params ($\downarrow$62.1%,$\downarrow$80.4%,) and GFLOPs ($\downarrow$43.4%,$\downarrow$63.8%,). Code and models are available at \url{here}.

cs.CV↗

arXiv2023

Mapping microstructure to shock-induced temperature fields using deep learning

The response of materials to dynamical, or shock, loading is important to planetary science, aerospace engineering, and energetic materials. Thermal-activated processes, including chemical reactions and phase transitions, are significantly accelerated by the localization of the energy deposited into hotspots. These results from the interaction of a supersonic wave with the materials microstructure and are governed by complex, coupled processes, including the collapse of porosity, interfacial friction, and localized plastic deformation. These mechanisms are not fully understood and today we lack predictive models to, for example, predict the shock to detonation transition from chemistry and microstructure alone. We demonstrate that deep learning techniques can be used to predict the resulting shock-induced temperature fields in complex composite materials obtained from large-scale molecular dynamics simulations with the initial microstructure as the only input. The accuracy of the Microstructure-Informed Shock-induced Temperature net (MISTnet) model is higher than the current state of the art at a fraction of the computation cost.

cond-mat.mtrl-sci↗

arXiv2022

Multi-View Neural Surface Reconstruction with Structured Light

Three-dimensional (3D) object reconstruction based on differentiable rendering (DR) is an active research topic in computer vision. DR-based methods minimize the difference between the rendered and target images by optimizing both the shape and appearance and realizing a high visual reproductivity. However, most approaches perform poorly for textureless objects because of the geometrical ambiguity, which means that multiple shapes can have the same rendered result in such objects. To overcome this problem, we introduce active sensing with structured light (SL) into multi-view 3D object reconstruction based on DR to learn the unknown geometry and appearance of arbitrary scenes and camera poses. More specifically, our framework leverages the correspondences between pixels in different views calculated by structured light as an additional constraint in the DR-based optimization of implicit surface, color representations, and camera poses. Because camera poses can be optimized simultaneously, our method realizes high reconstruction accuracy in the textureless region and reduces efforts for camera pose calibration, which is required for conventional SL-based methods. Experiment results on both synthetic and real data demonstrate that our system outperforms conventional DR- and SL-based methods in a high-quality surface reconstruction, particularly for challenging objects with textureless or shiny surfaces.

cs.CV↗