arXiv subjects

Han Zhang

Publications and source records attributed to Han Zhang.

At least 163 records · Page 9Linked to original sources

arXiv2025

S$^2$M-Former: Spiking Symmetric Mixing Branchformer for Brain Auditory Attention Detection

Auditory attention detection (AAD) aims to decode listeners' focus in complex auditory environments from electroencephalography (EEG) recordings, which is crucial for developing neuro-steered hearing devices. Despite recent advancements, EEG-based AAD remains hindered by the absence of synergistic frameworks that can fully leverage complementary EEG features under energy-efficiency constraints. We propose S$^2$M-Former, a novel spiking symmetric mixing framework to address this limitation through two key innovations: i) Presenting a spike-driven symmetric architecture composed of parallel spatial and frequency branches with mirrored modular design, leveraging biologically plausible token-channel mixers to enhance complementary learning across branches; ii) Introducing lightweight 1D token sequences to replace conventional 3D operations, reducing parameters by 14.7$\times$. The brain-inspired spiking architecture further reduces power consumption, achieving a 5.8$\times$ energy reduction compared to recent ANN methods, while also surpassing existing SNN baselines in terms of parameter efficiency and performance. Comprehensive experiments on three AAD benchmarks (KUL, DTU and AV-GC-AAD) across three settings (within-trial, cross-trial and cross-subject) demonstrate that S$^2$M-Former achieves comparable state-of-the-art (SOTA) decoding accuracy, making it a promising low-power, high-performance solution for AAD tasks. Code is available at https://github.com/JackieWang9811/S2M-Former.

cs.LG↗

arXiv2025

Novel Model-Independent Approach to Explore New Physics in Five-body Semileptonic Decays

Substantial contribution of the tensor current in semileptonic decays is regarded as a clear signal for new physics. In this work, we propose a model-independent approach to unambiguously test contribution of the tensor current in semileptonic five-body decays $\bar{D}_{(s)}/\bar{B}_{(s)}\to V\ell\barν_{\ell}\,(\ell=e,μ,τ)$ with $V\to π^0π^+π^-$, where $V$ denotes vector particles. We derive three parameters associated with the angular asymmetry, which are always equal to one in the Standard Model regardless the data of form factor but will deviate if contribution of the tensor current doesn't vanish. The outcomes have potential applications in precisely testing the Standard Model and searching for new physics. Relevant measurements can be performed using data collected by BESIII, Belle~II, and LHCb.

hep-ph↗

arXiv2025

UniRL-Zero: Reinforcement Learning on Unified Models with Joint Language Model and Diffusion Model Experts

We present UniRL-Zero, a unified reinforcement learning (RL) framework that boosts, multimodal language model understanding and reasoning, diffusion model multimedia generation, and their beneficial interaction capabilities within a unified model. Our work defines six scenarios for unified model reinforcement learning, providing systematic baselines for reinforcement learning of unified understanding and generation model. Our code is available at https://github.com/G-U-N/UniRL.

cs.LG↗

arXiv2025

Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks

Adam is a popular and widely used adaptive gradient method in deep learning, which has also received tremendous focus in theoretical research. However, most existing theoretical work primarily analyzes its full-batch version, which differs fundamentally from the stochastic variant used in practice. Unlike SGD, stochastic Adam does not converge to its full-batch counterpart even with infinitesimal learning rates. We present the first theoretical characterization of how batch size affects Adam's generalization, analyzing two-layer over-parameterized CNNs on image data. Our results reveal that while both Adam and AdamW with proper weight decay $λ$ converge to poor test error solutions, their mini-batch variants can achieve near-zero test error. We further prove Adam has a strictly smaller effective weight decay bound than AdamW, theoretically explaining why Adam requires more sensitive $λ$ tuning. Extensive experiments validate our findings, demonstrating the critical role of batch size and weight decay in Adam's generalization performance.

cs.LG↗

arXiv2025

pEBR: A Probabilistic Approach to Embedding Based Retrieval

Embedding-based retrieval aims to learn a shared semantic representation space for both queries and items, enabling efficient and effective item retrieval through approximate nearest neighbor (ANN) algorithms. In current industrial practice, retrieval systems typically retrieve a fixed number of items for each query. However, this fixed-size retrieval often results in insufficient recall for head queries and low precision for tail queries. This limitation largely stems from the dominance of frequentist approaches in loss function design, which fail to address this challenge in industry. In this paper, we propose a novel \textbf{p}robabilistic \textbf{E}mbedding-\textbf{B}ased \textbf{R}etrieval (\textbf{pEBR}) framework. Our method models the item distribution conditioned on each query, enabling the use of a dynamic cosine similarity threshold derived from the cumulative distribution function (CDF) of the probabilistic model. Experimental results demonstrate that pEBR significantly improves both retrieval precision and recall. Furthermore, ablation studies reveal that the probabilistic formulation effectively captures the inherent differences between head-to-tail queries.

cs.IR↗

arXiv2025

Addressing the ID-Matching Challenge in Long Video Captioning

Generating captions for long and complex videos is both critical and challenging, with significant implications for the growing fields of text-to-video generation and multi-modal understanding. One key challenge in long video captioning is accurately recognizing the same individuals who appear in different frames, which we refer to as the ID-Matching problem. Few prior works have focused on this important issue. Those that have, usually suffer from limited generalization and depend on point-wise matching, which limits their overall effectiveness. In this paper, unlike previous approaches, we build upon LVLMs to leverage their powerful priors. We aim to unlock the inherent ID-Matching capabilities within LVLMs themselves to enhance the ID-Matching performance of captions. Specifically, we first introduce a new benchmark for assessing the ID-Matching capabilities of video captions. Using this benchmark, we investigate LVLMs containing GPT-4o, revealing key insights that the performance of ID-Matching can be improved through two methods: 1) enhancing the usage of image information and 2) increasing the quantity of information of individual descriptions. Based on these insights, we propose a novel video captioning method called Recognizing Identities for Captioning Effectively (RICE). Extensive experiments including assessments of caption quality and ID-Matching performance, demonstrate the superiority of our approach. Notably, when implemented on GPT-4o, our RICE improves the precision of ID-Matching from 50% to 90% and improves the recall of ID-Matching from 15% to 80% compared to baseline. RICE makes it possible to continuously track different individuals in the captions of long videos.

cs.CV↗

arXiv2025

A laser with instability reaching $4 \times 10^{-17}$ based on a 10-cm-long silicon cavity at sub-5-K temperatures

The realization of ultra-stable lasers with $10^{-17}$-level frequency stability has enabled a wide range of researches on precision metrology and fundamental science, where cryogenic single-crystalline cavities constitute the heart of such ultra-stable lasers. For further improvements in stability, increasing the cavity length at few-kelvin temperatures provides a promising alternative to utilizing relatively short cavities with novel coating, but has yet to be demonstrated with state-of-the-art stability. Here we report on the realization of a relatively long ultra-stable silicon cavity with a length of 10 cm and sub-5-K operating temperatures. We devise a dynamical protocol of cool-quiet quench measurement that reveals the inherent $10^{-17}$-level frequency instability of the silicon cavity despite the substantially larger frequency noise induced by the cryostat vibration. We further develop a method for suppressing the cryostat-vibration-induced frequency noise under continuous cooling, and observe an average frequency instability of $4.3(2) \times 10^{-17}$ for averaging times of 4 to 12 seconds. Using the measured noise power spectral density, we compute a median linewidth of 9.6(3) mHz for the silicon cavity laser at 1397 nm, which is supported by an empirically determined linewidth of 5.7(3) mHz based on direct optical beat measurements. These results establish a new record for optical cavities within a closed-cycle cryocooler at sub-5-K temperatures and provide a prototypical system for using long cryogenic cavities to enhance frequency stabilities to the low-$10^{-17}$ or better level.

physics.optics↗

arXiv2025

OptPipe: Memory- and Scheduling-Optimized Pipeline Parallelism for LLM Training

Pipeline parallelism (PP) has become a standard technique for scaling large language model (LLM) training across multiple devices. However, despite recent progress in reducing memory consumption through activation offloading, existing approaches remain largely heuristic and coarse-grained, often overlooking the fine-grained trade-offs between memory, computation, and scheduling latency. In this work, we revisit the pipeline scheduling problem from a principled optimization perspective. We observe that prevailing strategies either rely on static rules or aggressively offload activations without fully leveraging the interaction between memory constraints and scheduling efficiency. To address this, we formulate scheduling as a constrained optimization problem that jointly accounts for memory capacity, activation reuse, and pipeline bubble minimization. Solving this model yields fine-grained schedules that reduce pipeline bubbles while adhering to strict memory budgets. Our approach complements existing offloading techniques: whereas prior approaches trade memory for time in a fixed pattern, we dynamically optimize the tradeoff with respect to model structure and hardware configuration. Experimental results demonstrate that our method consistently improves both throughput and memory utilization. In particular, we reduce idle pipeline time by up to 50% under the same per-device memory limit, and in some cases, enable the training of larger models within limited memory budgets.

cs.DC↗

arXiv2025

Push the Limit of Multi-modal Emotion Recognition by Prompting LLMs with Receptive-Field-Aware Attention Weighting

Understanding the emotions in a dialogue usually requires external knowledge to accurately understand the contents. As the LLMs become more and more powerful, we do not want to settle on the limited ability of the pre-trained language model. However, the LLMs either can only process text modality or are too expensive to process the multimedia information. We aim to utilize both the power of LLMs and the supplementary features from the multimedia modalities. In this paper, we present a framework, Lantern, that can improve the performance of a certain vanilla model by prompting large language models with receptive-field-aware attention weighting. This framework trained a multi-task vanilla model to produce probabilities of emotion classes and dimension scores. These predictions are fed into the LLMs as references to adjust the predicted probabilities of each emotion class with its external knowledge and contextual understanding. We slice the dialogue into different receptive fields, and each sample is included in exactly t receptive fields. Finally, the predictions of LLMs are merged with a receptive-field-aware attention-driven weighting module. In the experiments, vanilla models CORECT and SDT are deployed in Lantern with GPT-4 or Llama-3.1-405B. The experiments in IEMOCAP with 4-way and 6-way settings demonstrated that the Lantern can significantly improve the performance of current vanilla models by up to 1.23% and 1.80%.

cs.CL↗

arXiv2025

DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation

We present DexUMI - a data collection and policy learning framework that uses the human hand as the natural interface to transfer dexterous manipulation skills to various robot hands. DexUMI includes hardware and software adaptations to minimize the embodiment gap between the human hand and various robot hands. The hardware adaptation bridges the kinematics gap using a wearable hand exoskeleton. It allows direct haptic feedback in manipulation data collection and adapts human motion to feasible robot hand motion. The software adaptation bridges the visual gap by replacing the human hand in video data with high-fidelity robot hand inpainting. We demonstrate DexUMI's capabilities through comprehensive real-world experiments on two different dexterous robot hand hardware platforms, achieving an average task success rate of 86%.

cs.RO↗

arXiv2025

RAE: A Neural Network Dimensionality Reduction Method for Nearest Neighbors Preservation in Vector Search

While high-dimensional embedding vectors are being increasingly employed in various tasks like Retrieval-Augmented Generation and Recommendation Systems, popular dimensionality reduction (DR) methods such as PCA and UMAP have rarely been adopted for accelerating the retrieval process due to their inability of preserving the nearest neighbor (NN) relationship among vectors. Empowered by neural networks' optimization capability and the bounding effect of Rayleigh quotient, we propose a Regularized Auto-Encoder (RAE) for k-NN preserving dimensionality reduction. RAE constrains the network parameter variation through regularization terms, adjusting singular values to control embedding magnitude changes during reduction, thus preserving k-NN relationships. We provide a rigorous mathematical analysis demonstrating that regularization establishes an upper bound on the norm distortion rate of transformed vectors, thereby offering provable guarantees for k-NN preservation. With modest training overhead, RAE achieves superior k-NN recall compared to existing DR approaches while maintaining fast retrieval efficiency.

cs.IR↗

arXiv2025

Terrain-Awared LiDAR-Inertial Odometry for Legged-Wheel Robots Based on Radial Basis Function Approximation

An accurate odometry is essential for legged-wheel robots operating in unstructured terrains such as bumpy roads and staircases. Existing methods often suffer from pose drift due to their ignorance of terrain geometry. We propose a terrain-awared LiDAR-Inertial odometry (LIO) framework that approximates the terrain using Radial Basis Functions (RBF) whose centers are adaptively selected and weights are recursively updated. The resulting smooth terrain manifold enables ``soft constraints" that regularize the odometry optimization and mitigates the $z$-axis pose drift under abrupt elevation changes during robot's maneuver. To ensure the LIO's real-time performance, we further evaluate the RBF-related terms and calculate the inverse of the sparse kernel matrix with GPU parallelization. Experiments on unstructured terrains demonstrate that our method achieves higher localization accuracy than the state-of-the-art baselines, especially in the scenarios that have continuous height changes or sparse features when abrupt height changes occur.

cs.RO↗

arXiv2025

Adaptive Source-Channel Coding for Multi-User Semantic and Data Communications

This paper considers a multi-user semantic and data communication (MU-SemDaCom) system, where a base station (BS) simultaneously serves users with different semantic and data tasks through a downlink multi-user multiple-input single-output (MU-MISO) channel. The coexistence of heterogeneous communication tasks, diverse channel conditions, and the requirements for digital compatibility poses significant challenges to the efficient design of MU-SemDaCom systems. To address these issues, we propose a multi-user adaptive source-channel coding (MU-ASCC) framework that adaptively optimizes deep neural network (DNN)-based source coding, digital channel coding, and superposition broadcasting. First, we employ a data-regression method to approximate the end-to-end (E2E) semantic and data distortions, for which no closed-form expressions exist. The obtained logistic formulas decompose the E2E distortion as the addition of the source and channel distortion terms, in which the logistic parameter variations are task-dependent and jointly determined by both the DNN and channel parameters. Then, based on the derived formulas, we formulate a weighted-sum E2E distortion minimization problem that jointly optimizes the source-channel coding rates, power allocation, and beamforming vectors for both the data and semantic users. Finally, an alternating optimization (AO) framework is developed, where the adaptive rate optimization is solved using the subgradient descent method, while the joint power and beamforming is addressed via the uplink-downlink duality (UDD) technique. Simulation results demonstrate that, compared with the conventional separate source-channel coding (SSCC) and deep joint source-channel coding (DJSCC) schemes that are designed for a single task, the proposed MU-ASCC scheme achieves simultaneous improvements in both the data recovery and semantic task performance.

eess.IV↗

arXiv2025

Theoretical Performance Limit for Radar Parameter Estimation

In this paper, we employ the thoughts and methodologies of Shannon's information theory to solve the problem of the optimal radar parameter estimation. Based on a general radar system model, the \textit{a posteriori} probability density function of targets' parameters is derived. Range information (RI) and entropy error (EE) are defined to evaluate the performance. It is proved that acquiring 1 bit of the range information is equivalent to reducing estimation deviation by half. The closed-form approximation for the EE is deduced in all signal-to-noise ratio (SNR) regions, which demonstrates that the EE degenerates to the mean square error (MSE) when the SNR is tending to infinity. Parameter estimation theorem is then proved, which claims that the theoretical RI is achievable. The converse claims that there exists no unbiased estimator whose empirical RI is larger than the theoretical RI. Simulation result demonstrates that the theoretical EE is tighter than the commonly used Cramér-Rao bound and the ZivZakai bound.

cs.IT↗

arXiv2025

CHURRO: Making History Readable with an Open-Weight Large Vision-Language Model for High-Accuracy, Low-Cost Historical Text Recognition

Accurate text recognition for historical documents can greatly advance the study and preservation of cultural heritage. Existing vision-language models (VLMs), however, are designed for modern, standardized texts and are not equipped to read the diverse languages and scripts, irregular layouts, and frequent degradation found in historical materials. This paper presents CHURRO, a 3B-parameter open-weight VLM specialized for historical text recognition. The model is trained on CHURRO-DS, the largest historical text recognition dataset to date. CHURRO-DS unifies 155 historical corpora comprising 99,491 pages, spanning 22 centuries of textual heritage across 46 language clusters, including historical variants and dead languages. We evaluate several open-weight and closed VLMs and optical character recognition (OCR) systems on CHURRO-DS and find that CHURRO outperforms all other VLMs. On the CHURRO-DS test set, CHURRO achieves 82.3% (printed) and 70.1% (handwritten) normalized Levenshtein similarity, surpassing the second-best model, Gemini 2.5 Pro, by 1.4% and 6.5%, respectively, while being 15.5 times more cost-effective. By releasing the model and dataset, we aim to enable community-driven research to improve the readability of historical texts and accelerate scholarship.

cs.CL↗

arXiv2025

ColorBlindnessEval: Can Vision-Language Models Pass Color Blindness Tests?

This paper presents ColorBlindnessEval, a novel benchmark designed to evaluate the robustness of Vision-Language Models (VLMs) in visually adversarial scenarios inspired by the Ishihara color blindness test. Our dataset comprises 500 Ishihara-like images featuring numbers from 0 to 99 with varying color combinations, challenging VLMs to accurately recognize numerical information embedded in complex visual patterns. We assess 9 VLMs using Yes/No and open-ended prompts and compare their performance with human participants. Our experiments reveal limitations in the models' ability to interpret numbers in adversarial contexts, highlighting prevalent hallucination issues. These findings underscore the need to improve the robustness of VLMs in complex visual environments. ColorBlindnessEval serves as a valuable tool for benchmarking and improving the reliability of VLMs in real-world applications where accuracy is critical.

cs.CV↗

arXiv2025

BEAM: Brainwave Empathy Assessment Model for Early Childhood

Empathy in young children is crucial for their social and emotional development, yet predicting it remains challenging. Traditional methods often only rely on self-reports or observer-based labeling, which are susceptible to bias and fail to objectively capture the process of empathy formation. EEG offers an objective alternative; however, current approaches primarily extract static patterns, neglecting temporal dynamics. To overcome these limitations, we propose a novel deep learning framework, the Brainwave Empathy Assessment Model (BEAM), to predict empathy levels in children aged 4-6 years. BEAM leverages multi-view EEG signals to capture both cognitive and emotional dimensions of empathy. The framework comprises three key components: 1) a LaBraM-based encoder for effective spatio-temporal feature extraction, 2) a feature fusion module to integrate complementary information from multi-view signals, and 3) a contrastive learning module to enhance class separation. Validated on the CBCP dataset, BEAM outperforms state-of-the-art methods across multiple metrics, demonstrating its potential for objective empathy assessment and providing a preliminary insight into early interventions in children's prosocial development.

cs.LG↗

arXiv2025

DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model

Annotation variability remains a substantial challenge in medical image segmentation, stemming from ambiguous imaging boundaries and diverse clinical expertise. Traditional deep learning methods producing single deterministic segmentation predictions often fail to capture these annotator biases. Although recent studies have explored multi-rater segmentation, existing methods typically focus on a single perspective -- either generating a probabilistic ``gold standard'' consensus or preserving expert-specific preferences -- thus struggling to provide a more omni view. In this study, we propose DiffOSeg, a two-stage diffusion-based framework, which aims to simultaneously achieve both consensus-driven (combining all experts' opinions) and preference-driven (reflecting experts' individual assessments) segmentation. Stage I establishes population consensus through a probabilistic consensus strategy, while Stage II captures expert-specific preference via adaptive prompts. Demonstrated on two public datasets (LIDC-IDRI and NPC-170), our model outperforms existing state-of-the-art methods across all evaluated metrics. Source code is available at https://github.com/string-ellipses/DiffOSeg .

cs.CV↗