arXiv subjects

Bin Yang

Publications and source records attributed to Bin Yang.

At least 181 records · Page 10Linked to original sources

arXiv2025

QwenLong-CPRS: Towards $\infty$-LLMs with Dynamic Context Optimization

This technical report presents QwenLong-CPRS, a context compression framework designed for explicit long-context optimization, addressing prohibitive computation overhead during the prefill stage and the "lost in the middle" performance degradation of large language models (LLMs) during long sequence processing. Implemented through a novel dynamic context optimization mechanism, QwenLong-CPRS enables multi-granularity context compression guided by natural language instructions, achieving both efficiency gains and improved performance. Evolved from the Qwen architecture series, QwenLong-CPRS introduces four key innovations: (1) Natural language-guided dynamic optimization, (2) Bidirectional reasoning layers for enhanced boundary awareness, (3) Token critic mechanisms with language modeling heads, and (4) Window-parallel inference. Comprehensive evaluations across five benchmarks (4K-2M word contexts) demonstrate QwenLong-CPRS's threefold effectiveness: (1) Consistent superiority over other context management methods like RAG and sparse attention in both accuracy and efficiency. (2) Architecture-agnostic integration with all flagship LLMs, including GPT-4o, Gemini2.0-pro, Claude3.7-sonnet, DeepSeek-v3, and Qwen2.5-max, achieves 21.59$\times$ context compression alongside 19.15-point average performance gains; (3) Deployed with Qwen2.5-32B-Instruct, QwenLong-CPRS surpasses leading proprietary LLMs by 4.85 and 10.88 points on Ruler-128K and InfiniteBench, establishing new SOTA performance.

cs.CL↗

arXiv2025

RadarRGBD A Multi-Sensor Fusion Dataset for Perception with RGB-D and mmWave Radar

Multi-sensor fusion has significant potential in perception tasks for both indoor and outdoor environments. Especially under challenging conditions such as adverse weather and low-light environments, the combined use of millimeter-wave radar and RGB-D sensors has shown distinct advantages. However, existing multi-sensor datasets in the fields of autonomous driving and robotics often lack high-quality millimeter-wave radar data. To address this gap, we present a new multi-sensor dataset:RadarRGBD. This dataset includes RGB-D data, millimeter-wave radar point clouds, and raw radar matrices, covering various indoor and outdoor scenes, as well as low-light environments. Compared to existing datasets, RadarRGBD employs higher-resolution millimeter-wave radar and provides raw data, offering a new research foundation for the fusion of millimeter-wave radar and visual sensors. Furthermore, to tackle the noise and gaps in depth maps captured by Kinect V2 due to occlusions and mismatches, we fine-tune an open-source relative depth estimation framework, incorporating the absolute depth information from the dataset for depth supervision. We also introduce pseudo-relative depth scale information to further optimize the global depth scale estimation. Experimental results demonstrate that the proposed method effectively fills in missing regions in sensor data. Our dataset and related documentation will be publicly available at: https://github.com/song4399/RadarRGBD.

eess.IV↗

arXiv2025

UVTM: Universal Vehicle Trajectory Modeling with ST Feature Domain Generation

Vehicle movement is frequently captured in the form of GPS trajectories, i.e., sequences of timestamped GPS locations. Such data is widely used for various tasks such as travel-time estimation, trajectory recovery, and trajectory prediction. A universal vehicle trajectory model could be applied to different tasks, removing the need to maintain multiple specialized models, thereby reducing computational and storage costs. However, creating such a model is challenging when the integrity of trajectory features is compromised, i.e., in scenarios where only partial features are available or the trajectories are sparse. To address these challenges, we propose the Universal Vehicle Trajectory Model (UVTM), which can effectively adapt to different tasks without excessive retraining. UVTM incorporates two specialized designs. First, it divides trajectory features into three distinct domains. Each domain can be masked and generated independently to accommodate tasks with only partially available features. Second, UVTM is pre-trained by reconstructing dense, feature-complete trajectories from sparse, feature-incomplete counterparts, enabling strong performance even when the integrity of trajectory features is compromised. Experiments involving four representative trajectory-related tasks on three real-world vehicle trajectory datasets provide insight into the performance of UVTM and offer evidence that it is capable of meeting its objectives.

cs.LG↗

arXiv2025

Droplet coalescence kinetics: Coalescence mechanisms and thermodynamic non-equilibrium effects under isothermal and non-isothermal conditions

This study investigates the droplet coalescence mechanisms and the interplay between various thermodynamic non-equilibrium (TNE) effects under isothermal and non-isothermal conditions kinetically. The main findings include: (1) Coalescence initiation and cut-through mechanisms: In non-isothermal conditions, the temperature rise caused by the release of latent heat during phase transition slightly increases the surface tension gradient (driving force) near the contact point of the two droplets, while significantly enhancing the pressure gradient (resistance). This results in a significantly prolonged coalescence initiation time compared to the isothermal case. In both cases, pressure extends the liquid-vapor interface in opposite directions, promoting the growth of the liquid bridge. (2) TNE effects: Latent heat-induced temperature rise significantly refrains the TNE intensity in thermal case. Before and after droplet contact, non-equilibrium quantities driven by the temperature gradient and those driven by the velocity gradient, alternate in dominating the coalescence process. This competition and interplay result in a more complex spatial and spatiotemporal evolution of TNE effects compared to the isothermal case. (3) Entropy production mechanisms: In the non-isothermal case, entropy production is contributed not only by $\bm Δ^{\ast}_2$ but also by $\bm Δ^{\ast}_{3,1}$, with the former being the dominant contributor. The temperature field reduces the entropy production rate, while extends its duration, and increases the total entropy production. This research provides kinetic insights for dynamic, cross-scale regulation and multifunctional integration of coalescence processes in industrial applications.

physics.flu-dyn↗

arXiv2025

Air Quality Prediction with Physics-Guided Dual Neural ODEs in Open Systems

Air pollution significantly threatens human health and ecosystems, necessitating effective air quality prediction to inform public policy. Traditional approaches are generally categorized into physics-based and data-driven models. Physics-based models usually struggle with high computational demands and closed-system assumptions, while data-driven models may overlook essential physical dynamics, confusing the capturing of spatiotemporal correlations. Although some physics-guided approaches combine the strengths of both models, they often face a mismatch between explicit physical equations and implicit learned representations. To address these challenges, we propose Air-DualODE, a novel physics-guided approach that integrates dual branches of Neural ODEs for air quality prediction. The first branch applies open-system physical equations to capture spatiotemporal dependencies for learning physics dynamics, while the second branch identifies the dependencies not addressed by the first in a fully data-driven way. These dual representations are temporally aligned and fused to enhance prediction accuracy. Our experimental results demonstrate that Air-DualODE achieves state-of-the-art performance in predicting pollutant concentrations across various spatial scales, thereby offering a promising solution for real-world air quality challenges.

cs.LG↗

arXiv2025

AimTS: Augmented Series and Image Contrastive Learning for Time Series Classification

Time series classification (TSC) is an important task in time series analysis. Existing TSC methods mainly train on each single domain separately, suffering from a degradation in accuracy when the samples for training are insufficient in certain domains. The pre-training and fine-tuning paradigm provides a promising direction for solving this problem. However, time series from different domains are substantially divergent, which challenges the effective pre-training on multi-source data and the generalization ability of pre-trained models. To handle this issue, we introduce Augmented Series and Image Contrastive Learning for Time Series Classification (AimTS), a pre-training framework that learns generalizable representations from multi-source time series data. We propose a two-level prototype-based contrastive learning method to effectively utilize various augmentations in multi-source pre-training, which learns representations for TSC that can be generalized to different domains. In addition, considering augmentations within the single time series modality are insufficient to fully address classification problems with distribution shift, we introduce the image modality to supplement structural information and establish a series-image contrastive learning to improve the generalization of the learned representations for TSC tasks. Extensive experiments show that after multi-source pre-training, AimTS achieves good generalization performance, enabling efficient learning and even few-shot learning on various downstream TSC datasets.

cs.LG↗

arXiv2025

Thermodynamic nonequilibrium effects in three-dimensional high-speed compressible flows: Multiscale modeling and simulation via the discrete Boltzmann method

Three-dimensional (3D) high-speed compressible flow is a typical nonlinear, nonequilibrium, and multiscale complex flow. Traditional fluid mechanics models, based on the quasi-continuum assumption and near-equilibrium approximation, are insufficient to capture significant discrete effects and thermodynamic nonequilibrium effects (TNEs) as the Knudsen number increases. To overcome these limitations, a discrete Boltzmann modeling and simulation method, rooted in kinetic and mean-field theories, has been developed. By applying Chapman-Enskog multiscale analysis, the essential kinetic moment relations $\bmΦ$ for characterizing second-order TNEs are determined. These relations are invariants in coarse-grained physical modeling, providing a unique mesoscopic perspective for analyzing TNE behaviors. A discrete Boltzmann model, accurate to the second-order in the Knudsen number, is developed to enable multiscale simulations of 3D supersonic flows. As key TNE measures, nonlinear constitutive relations (NCRs), are theoretically derived for the 3D case, offering a constitutive foundation for improving macroscopic fluid modeling. The NCRs in three dimensions exhibit greater complexity than their two-dimensional counterparts. This complexity arises from increased degrees of freedom, which introduce additional kinds of nonequilibrium driving forces, stronger coupling between these forces, and a significant increase in nonequilibrium components. At the macroscopic level, the model is validated through several classical test cases, ranging from 1D to 3D scenarios, from subsonic to supersonic regimes. At the mesoscopic level, the model accurately captures typical TNEs, such as viscous stress and heat flux, around mesoscale structures, across various scales and orders. This work provides kinetic insights that advance multiscale simulation techniques for 3D high-speed compressible flows.

physics.flu-dyn↗

arXiv2025

Data Driven Decision Making with Time Series and Spatio-temporal Data

Time series data captures properties that change over time. Such data occurs widely, ranging from the scientific and medical domains to the industrial and environmental domains. When the properties in time series exhibit spatial variations, we often call the data spatio-temporal. As part of the continued digitalization of processes throughout society, increasingly large volumes of time series and spatio-temporal data are available. In this tutorial, we focus on data-driven decision making with such data, e.g., enabling greener and more efficient transportation based on traffic time series forecasting. The tutorial adopts the holistic paradigm of ``data-governance-analytics-decision.'' We first introduce the data foundation of time series and spatio-temporal data, which is often heterogeneous. Next, we discuss data governance methods that aim to improve data quality. We then cover data analytics, focusing on the ``AGREE'' principles: Automation, Generalization, Robustness, Explainability, and Efficiency. We finally cover data-driven decision making strategies and briefly discuss promising research directions. We hope that the tutorial will serve as a primary resource for researchers and practitioners who are interested in value creation from time series and spatio-temporal data.

cs.DB↗

arXiv2025

Multiscale thermodynamic nonequilibrium effects in Kelvin-Helmholtz instability and their relative importance

This study investigates the complex kinetics of thermodynamic nonequilibrium effects (TNEs) and their relative importance during the development of Kelvin-Helmholtz instability (KHI) using high-order discrete Boltzmann models (DBMs). First, the capabilities and differences among various discrete velocity sets in capturing TNEs and distribution functions are assessed. Practical guidelines for constructing discrete velocity stencils are proposed to enhance phase-space discretization and improve the robustness of high-order DBM simulation. At different stages of KHI and under varying initial conditions, multiscale TNEs, such as viscous stresses of different orders, emerge with distinct dominant roles. Specifically, three scenarios are identified: (i) regimes dominated by first-order TNEs,(ii) alternation between first- and second-order TNEs, and (iii) states where second-order TNEs govern the system's behavior. To quantitatively capture these transitions, criteria for TNE dominance at different orders in KHI evolution are established based on the relative thermodynamic nonequilibrium intensity ($R_{\text{TNE}}$). In scenarios dominated by second-order TNEs, differences between first-order and second-order models are compared in terms of macroscopic quantities, nonequilibrium effects, and kinetic moments, revealing the physical limitations of low-order models in capturing TNEs. Furthermore, the effectiveness, extensibility, and limitations of a representative high-order model are examined under second-order TNE-dominated conditions. To encapsulate these findings, a nonequilibrium phase diagram that visually maps the multiscale characteristics of KHI is constructed. This diagram not only provides intuitive insights into the dynamic interplay of different nonequilibrium effects but also serves as a kinetic roadmap for selecting suitable models under diverse nonequilibrium conditions.

physics.flu-dyn↗

arXiv2025

Learning Generalizable Skills from Offline Multi-Task Data for Multi-Agent Cooperation

Learning cooperative multi-agent policy from offline multi-task data that can generalize to unseen tasks with varying numbers of agents and targets is an attractive problem in many scenarios. Although aggregating general behavior patterns among multiple tasks as skills to improve policy transfer is a promising approach, two primary challenges hinder the further advancement of skill learning in offline multi-task MARL. Firstly, extracting general cooperative behaviors from various action sequences as common skills lacks bringing cooperative temporal knowledge into them. Secondly, existing works only involve common skills and can not adaptively choose independent knowledge as task-specific skills in each task for fine-grained action execution. To tackle these challenges, we propose Hierarchical and Separate Skill Discovery (HiSSD), a novel approach for generalizable offline multi-task MARL through skill learning. HiSSD leverages a hierarchical framework that jointly learns common and task-specific skills. The common skills learn cooperative temporal knowledge and enable in-sample exploitation for offline multi-task MARL. The task-specific skills represent the priors of each task and achieve a task-guided fine-grained action execution. To verify the advancement of our method, we conduct experiments on multi-agent MuJoCo and SMAC benchmarks. After training the policy using HiSSD on offline multi-task data, the empirical results show that HiSSD assigns effective cooperative behaviors and obtains superior performance in unseen tasks.

cs.LG↗

arXiv2025

A high-resolution microresonator-frequency-comb spectrometer

Spectral analysis is one of the most powerful technologies for studying and understanding matter. As the devices for spectral analysis, spectrometers are widely used in material detection, isotope analysis, trace gas detection, and the study of atomic and molecular hyperfine structures. While high resolution, wide bandwidth and fast speed are essential factors, they are always trade-offs for conventional spectrometers. Here, we present a soliton-microcomb-based spectrometer that overcomes these challenges by integrating dissipative Kerr solitons (DKSs) with double-sideband modulation and parallelized detection. Leveraging a high-quality silicon nitride microresonator, we generate a broadband, fully stabilized soliton microcomb and employ radio-frequency-modulated double sidebands to scan the optical spectrum with the resolution constrained only by the comb-line linewidth. By projecting the comb lines onto a two-dimensional charge-coupled device (CCD) via a virtually imaged phased array (VIPA)-grating system, we enable parallel processing of all spectral components, circumventing sequential scanning delays. The resulting spectrometer achieves 200-kHz resolution across a 4-THz bandwidth with minutes-level processing time while maintaining robustness against environmental fluctuations. Being promising for miniaturization, this work bridges the gap between laboratory-grade performance and field-deployable practicality, unlocking new possibilities for spectroscopy in astronomy, metrology, and integrated photonics.

physics.optics↗

arXiv2025

Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey

Spatio-Temporal (ST) data science, which includes sensing, managing, and mining large-scale data across space and time, is fundamental to understanding complex systems in domains such as urban computing, climate science, and intelligent transportation. Traditional deep learning approaches have significantly advanced this field, particularly in the stage of ST data mining. However, these models remain task-specific and often require extensive labeled data. Inspired by the success of Foundation Models (FM), especially large language models, researchers have begun exploring the concept of Spatio-Temporal Foundation Models (STFMs) to enhance adaptability and generalization across diverse ST tasks. Unlike prior architectures, STFMs empower the entire workflow of ST data science, ranging from data sensing, management, to mining, thereby offering a more holistic and scalable approach. Despite rapid progress, a systematic study of STFMs for ST data science remains lacking. This survey aims to provide a comprehensive review of STFMs, categorizing existing methodologies and identifying key research directions to advance ST general intelligence.

cs.DB↗

arXiv2025

Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models

Latent diffusion models with Transformer architectures excel at generating high-fidelity images. However, recent studies reveal an optimization dilemma in this two-stage design: while increasing the per-token feature dimension in visual tokenizers improves reconstruction quality, it requires substantially larger diffusion models and more training iterations to achieve comparable generation performance. Consequently, existing systems often settle for sub-optimal solutions, either producing visual artifacts due to information loss within tokenizers or failing to converge fully due to expensive computation costs. We argue that this dilemma stems from the inherent difficulty in learning unconstrained high-dimensional latent spaces. To address this, we propose aligning the latent space with pre-trained vision foundation models when training the visual tokenizers. Our proposed VA-VAE (Vision foundation model Aligned Variational AutoEncoder) significantly expands the reconstruction-generation frontier of latent diffusion models, enabling faster convergence of Diffusion Transformers (DiT) in high-dimensional latent spaces. To exploit the full potential of VA-VAE, we build an enhanced DiT baseline with improved training strategies and architecture designs, termed LightningDiT. The integrated system achieves state-of-the-art (SOTA) performance on ImageNet 256x256 generation with an FID score of 1.35 while demonstrating remarkable training efficiency by reaching an FID score of 2.11 in just 64 epochs--representing an over 21 times convergence speedup compared to the original DiT. Models and codes are available at: https://github.com/hustvl/LightningDiT.

cs.CV↗

arXiv2025

Assessing Pre-Trained Models for Transfer Learning Through Distribution of Spectral Components

Pre-trained model assessment for transfer learning aims to identify the optimal candidate for the downstream tasks from a model hub, without the need of time-consuming fine-tuning. Existing advanced works mainly focus on analyzing the intrinsic characteristics of the entire features extracted by each pre-trained model or how well such features fit the target labels. This paper proposes a novel perspective for pre-trained model assessment through the Distribution of Spectral Components (DISCO). Through singular value decomposition of features extracted from pre-trained models, we investigate different spectral components and observe that they possess distinct transferability, contributing diversely to the fine-tuning performance. Inspired by this, we propose an assessment method based on the distribution of spectral components which measures the proportions of their corresponding singular values. Pre-trained models with features concentrating on more transferable components are regarded as better choices for transfer learning. We further leverage the labels of downstream data to better estimate the transferability of each spectral component and derive the final assessment criterion. Our proposed method is flexible and can be applied to both classification and regression tasks. We conducted comprehensive experiments across three benchmarks and two tasks including image classification and object detection, demonstrating that our method achieves state-of-the-art performance in choosing proper pre-trained models from the model hub for transfer learning.

cs.LG↗

arXiv2025

RCRank: Multimodal Ranking of Root Causes of Slow Queries in Cloud Database Systems

With the continued migration of storage to cloud database systems,the impact of slow queries in such systems on services and user experience is increasing. Root-cause diagnosis plays an indispensable role in facilitating slow-query detection and revision. This paper proposes a method capable of both identifying possible root cause types for slow queries and ranking these according to their potential for accelerating slow queries. This enables prioritizing root causes with the highest impact, in turn improving slow-query revision effectiveness. To enable more accurate and detailed diagnoses, we propose the multimodal Ranking for the Root Causes of slow queries (RCRank) framework, which formulates root cause analysis as a multimodal machine learning problem and leverages multimodal information from query statements, execution plans, execution logs, and key performance indicators. To obtain expressive embeddings from its heterogeneous multimodal input, RCRank integrates self-supervised pre-training that enhances cross-modal alignment and task relevance. Next, the framework integrates root-cause-adaptive cross Transformers that enable adaptive fusion of multimodal features with varying characteristics. Finally, the framework offers a unified model that features an impact-aware training objective for identifying and ranking root causes. We report on experiments on real and synthetic datasets, finding that RCRank is capable of consistently outperforming the state-of-the-art methods at root cause identification and ranking according to a range of metrics.

cs.DB↗

arXiv2025

Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model

Multi-modal Large Language Models (MLLMs) integrate visual and linguistic reasoning to address complex tasks such as image captioning and visual question answering. While MLLMs demonstrate remarkable versatility, MLLMs appears limited performance on special applications. But tuning MLLMs for downstream tasks encounters two key challenges: Task-Expert Specialization, where distribution shifts between pre-training and target datasets constrain target performance, and Open-World Stabilization, where catastrophic forgetting erases the model general knowledge. In this work, we systematically review recent advancements in MLLM tuning methodologies, classifying them into three paradigms: (I) Selective Tuning, (II) Additive Tuning, and (III) Reparameterization Tuning. Furthermore, we benchmark these tuning strategies across popular MLLM architectures and diverse downstream tasks to establish standardized evaluation analysis and systematic tuning principles. Finally, we highlight several open challenges in this domain and propose future research directions. To facilitate ongoing progress in this rapidly evolving field, we provide a public repository that continuously tracks developments: https://github.com/WenkeHuang/Awesome-MLLM-Tuning.

cs.CL↗

arXiv2025

Near-infrared spectroscopic characterization of the Pallas family

The Pallas collisional family of asteroids, named after (2) Pallas, is notable for its high orbital inclination and the distinct blue color of Pallas and a few larger B-type family members. While Pallas itself, as one of the largest asteroids, has been studied in detail, most of its smaller family members still remain unexplored. This study aims to characterize the physical properties of medium- to small-sized Pallas family asteroids to investigate the origin of their unusual blueness. Additionally, we explore the relationship between the Pallas family and the near-Earth object (NEO) (3200) Phaethon. We conducted near-infrared (NIR) spectroscopy with the NASA Infrared Telescope Facility (IRTF) to collect reflectance spectra for 22 asteroids, including one from the IRTF Legacy Archive. Spectroscopic and dynamical analyses were carried out to identify outliers, while additional data from NEOWISE and Gaia were incorporated to examine potential correlations among their physical properties. Meteorite analogs were identified through chi-square matching using samples from the RELAB database. The observed Pallas family asteroids exhibit nearly identical spectral profiles, suggesting a homogeneous composition of ejected material. Small variations in spectral slopes are observed, which may result from different levels of alteration experienced by individual asteroids, with some influence from variations in grain size. Most of the observed spectra of the Pallas asteroids, from 0.8 to 2.2 micron, closely resemble those of the CY and CI meteorites. The blueness of asteroid surfaces is likely due to the presence of magnetite, troilite, or phyllosilicates, which are products of aqueous alteration. The striking spectral similarity between (3200) Phaethon and Pallas family members of comparable sizes suggests a potential common origin.

astro-ph.EP↗

arXiv2025

Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders

Time series anomaly detection plays a vital role in a wide range of applications. Existing methods require training one specific model for each dataset, which exhibits limited generalization capability across different target datasets, hindering anomaly detection performance in various scenarios with scarce training data. Aiming at this problem, we propose constructing a general time series anomaly detection model, which is pre-trained on extensive multi-domain datasets and can subsequently apply to a multitude of downstream scenarios. The significant divergence of time series data across different domains presents two primary challenges in building such a general model: (1) meeting the diverse requirements of appropriate information bottlenecks tailored to different datasets in one unified model, and (2) enabling distinguishment between multiple normal and abnormal patterns, both are crucial for effective anomaly detection in various target scenarios. To tackle these two challenges, we propose a General time series anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders (DADA), which enables flexible selection of bottlenecks based on different data and explicitly enhances clear differentiation between normal and abnormal series. We conduct extensive experiments on nine target datasets from different domains. After pre-training on multi-domain data, DADA, serving as a zero-shot anomaly detector for these datasets, still achieves competitive or even superior results compared to those models tailored to each specific dataset. The code is made available at https://github.com/decisionintelligence/DADA.

cs.LG↗