Search arXiv⌕ Search

arXiv subjects

Li Zhang

Publications and source records attributed to Li Zhang.

At least 397 records · Page 22Linked to original sources
arXiv2025

OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academia

Large Language Models (LLMs) have made significant progress in various downstream tasks, inspiring the development of Speech Understanding Language Models (SULMs) to enable comprehensive speech-based interactions. However, most advanced SULMs are developed by the industry, leveraging large-scale datasets and computational resources that are not readily available to the academic community. Moreover, the lack of transparency in training details creates additional barriers to further innovation. In this study, we present OSUM, an Open Speech Understanding Model designed to explore the potential of training SLUMs under constrained academic resources. The OSUM model combines a Whisper encoder with a Qwen2 LLM and supports a wide range of speech tasks, including speech recognition (ASR), speech recognition with timestamps (SRWT), vocal event detection (VED), speech emotion recognition (SER), speaking style recognition (SSR), speaker gender classification (SGC), speaker age prediction (SAP), and speech-to-text chat (STTC). By employing an ASR+X training strategy, OSUM achieves efficient and stable multi-task training by simultaneously optimizing ASR alongside target tasks. Beyond delivering strong performance, OSUM emphasizes transparency by providing openly available data preparation and training methodologies, offering valuable insights and practical guidance for the academic community. By doing so, we aim to accelerate research and innovation in advanced SULM technologies.

cs.SD↗
arXiv2025

DLBayesian: An Alternative Bayesian Reconstruction of Limited-view CT by Optimizing Deep Learning Parameters

Limited-view computed tomography (CT) presents significant potential for reducing radiation exposure and expediting the scanning process. While deep learning (DL) methods have exhibited promising results in mitigating streaking artifacts caused by a reduced number of projection views, their generalization remains challenging. In this work, we proposed a DL-driven alternative Bayesian reconstruction method (DLBayesian) that efficiently integrates data-driven priors and data consistency constraints. DLBayesian comprises three stages: group-level embedding, significance evaluation, and individual-level consistency adaptation. Firstly, DL network parameters are optimized to learn how to eliminate the general limited-view artifacts on a large-scale paired dataset. Then, we introduced a significance score to quantitatively evaluate the contribution of parameters in DL models as a guide for the subsequent individual-level adaptation. Finally, in the Bayesian adaptation stage, an alternative Bayesian reconstruction further optimizes the DL network parameters precisely according to the projection data of the target case. We validated DLBayesian with sparse-view (90 views) projections from a circular trajectory CT and a special data missing case from a multi-segment linear trajectory CT. The results underscore DLBayesian's superior generalization capabilities across variations in patients, anatomic structures, and data distribution, as well as excelling in contextual structure recovery compared to networks solely trained via supervised loss. Real experiments on a dead rat demonstrate its capability in practical CT scans.

physics.med-ph↗
arXiv2025

CANeRV: Content Adaptive Neural Representation for Video Compression

Recent advances in video compression introduce implicit neural representation (INR) based methods, which effectively capture global dependencies and characteristics of entire video sequences. Unlike traditional and deep learning based approaches, INR-based methods optimize network parameters from a global perspective, resulting in superior compression potential. However, most current INR methods utilize a fixed and uniform network architecture across all frames, limiting their adaptability to dynamic variations within and between video sequences. This often leads to suboptimal compression outcomes as these methods struggle to capture the distinct nuances and transitions in video content. To overcome these challenges, we propose Content Adaptive Neural Representation for Video Compression (CANeRV), an innovative INR-based video compression network that adaptively conducts structure optimisation based on the specific content of each video sequence. To better capture dynamic information across video sequences, we propose a dynamic sequence-level adjustment (DSA). Furthermore, to enhance the capture of dynamics between frames within a sequence, we implement a dynamic frame-level adjustment (DFA). {Finally, to effectively capture spatial structural information within video frames, thereby enhancing the detail restoration capabilities of CANeRV, we devise a structure level hierarchical structural adaptation (HSA).} Experimental results demonstrate that CANeRV can outperform both H.266/VVC and state-of-the-art INR-based video compression techniques across diverse video datasets.

cs.CV↗
arXiv2025

Noise Controlled CT Super-Resolution with Conditional Diffusion Model

Improving the spatial resolution of CT images is a meaningful yet challenging task, often accompanied by the issue of noise amplification. This article introduces an innovative framework for noise-controlled CT super-resolution utilizing the conditional diffusion model. The model is trained on hybrid datasets, combining noise-matched simulation data with segmented details from real data. Experimental results with real CT images validate the effectiveness of our proposed framework, showing its potential for practical applications in CT imaging.

cs.CV↗
arXiv2025

SeaFormer++: Squeeze-enhanced Axial Transformer for Mobile Visual Recognition

Since the introduction of Vision Transformers, the landscape of many computer vision tasks (e.g., semantic segmentation), which has been overwhelmingly dominated by CNNs, recently has significantly revolutionized. However, the computational cost and memory requirement renders these methods unsuitable on the mobile device. In this paper, we introduce a new method squeeze-enhanced Axial Transformer (SeaFormer) for mobile visual recognition. Specifically, we design a generic attention block characterized by the formulation of squeeze Axial and detail enhancement. It can be further used to create a family of backbone architectures with superior cost-effectiveness. Coupled with a light segmentation head, we achieve the best trade-off between segmentation accuracy and latency on the ARM-based mobile devices on the ADE20K, Cityscapes, Pascal Context and COCO-Stuff datasets. Critically, we beat both the mobilefriendly rivals and Transformer-based counterparts with better performance and lower latency without bells and whistles. Furthermore, we incorporate a feature upsampling-based multi-resolution distillation technique, further reducing the inference latency of the proposed framework. Beyond semantic segmentation, we further apply the proposed SeaFormer architecture to image classification and object detection problems, demonstrating the potential of serving as a versatile mobile-friendly backbone. Our code and models are made publicly available at https://github.com/fudan-zvg/SeaFormer.

cs.CV↗
arXiv2025

GS-LiDAR: Generating Realistic LiDAR Point Clouds with Panoramic Gaussian Splatting

LiDAR novel view synthesis (NVS) has emerged as a novel task within LiDAR simulation, offering valuable simulated point cloud data from novel viewpoints to aid in autonomous driving systems. However, existing LiDAR NVS methods typically rely on neural radiance fields (NeRF) as their 3D representation, which incurs significant computational costs in both training and rendering. Moreover, NeRF and its variants are designed for symmetrical scenes, making them ill-suited for driving scenarios. To address these challenges, we propose GS-LiDAR, a novel framework for generating realistic LiDAR point clouds with panoramic Gaussian splatting. Our approach employs 2D Gaussian primitives with periodic vibration properties, allowing for precise geometric reconstruction of both static and dynamic elements in driving scenarios. We further introduce a novel panoramic rendering technique with explicit ray-splat intersection, guided by panoramic LiDAR supervision. By incorporating intensity and ray-drop spherical harmonic (SH) coefficients into the Gaussian primitives, we enhance the realism of the rendered point clouds. Extensive experiments on KITTI-360 and nuScenes demonstrate the superiority of our method in terms of quantitative metrics, visual quality, as well as training and rendering efficiency.

cs.CV↗
arXiv2025

MATCNN: Infrared and Visible Image Fusion Method Based on Multi-scale CNN with Attention Transformer

While attention-based approaches have shown considerable progress in enhancing image fusion and addressing the challenges posed by long-range feature dependencies, their efficacy in capturing local features is compromised by the lack of diverse receptive field extraction techniques. To overcome the shortcomings of existing fusion methods in extracting multi-scale local features and preserving global features, this paper proposes a novel cross-modal image fusion approach based on a multi-scale convolutional neural network with attention Transformer (MATCNN). MATCNN utilizes the multi-scale fusion module (MSFM) to extract local features at different scales and employs the global feature extraction module (GFEM) to extract global features. Combining the two reduces the loss of detail features and improves the ability of global feature representation. Simultaneously, an information mask is used to label pertinent details within the images, aiming to enhance the proportion of preserving significant information in infrared images and background textures in visible images in fused images. Subsequently, a novel optimization algorithm is developed, leveraging the mask to guide feature extraction through the integration of content, structural similarity index measurement, and global feature loss. Quantitative and qualitative evaluations are conducted across various datasets, revealing that MATCNN effectively highlights infrared salient targets, preserves additional details in visible images, and achieves better fusion results for cross-modal images. The code of MATCNN will be available at https://github.com/zhang3849/MATCNN.git.

cs.CV↗
arXiv2025

Reflective Gaussian Splatting

Novel view synthesis has experienced significant advancements owing to increasingly capable NeRF- and 3DGS-based methods. However, reflective object reconstruction remains challenging, lacking a proper solution to achieve real-time, high-quality rendering while accommodating inter-reflection. To fill this gap, we introduce a Reflective Gaussian splatting (Ref-Gaussian) framework characterized with two components: (I) Physically based deferred rendering that empowers the rendering equation with pixel-level material properties via formulating split-sum approximation; (II) Gaussian-grounded inter-reflection that realizes the desired inter-reflection function within a Gaussian splatting paradigm for the first time. To enhance geometry modeling, we further introduce material-aware normal propagation and an initial per-Gaussian shading stage, along with 2D Gaussian primitives. Extensive experiments on standard datasets demonstrate that Ref-Gaussian surpasses existing approaches in terms of quantitative metrics, visual quality, and compute efficiency. Further, we show that our method serves as a unified solution for both reflective and non-reflective scenes, going beyond the previous alternatives focusing on only reflective scenes. Also, we illustrate that Ref-Gaussian supports more applications such as relighting and editing.

cs.CV↗
arXiv2025

Evidence for Magneto-gravitational Processes in Supermassive Black Hole Binary PG 1553+113

PG1553+113 has drawn significant attention for its quasi-periodic oscillation (QPO) in gamma-ray variability, though the origin of its variability remains uncertain. In this study, we propose a physical mechanism to explain the observed gamma-ray variability within the framework of a supermassive black hole binary (SMBHB) system, supported by a newly identified component hidden in the light curve. A detailed analysis for its about 16-year light curve obtained from Fermi-LAT observations is performed by Gaussian process (GP). As anticipated, the QPO of 2.1 years is effectively captured by the stochastically-driven damped simple harmonic oscillator (SHO) kernel within the under-damped regime, and the overall stochastic nature of the variability is described by the damped random walk (DRW) kernel albeit with an unconstrained damping timescale. Additionally, our results reveal a previously unrecognized component in active galactic nuclei variability, characterized by the Matérn-3/2 kernel, which is typically associated with systems undergoing abrupt energy release. These findings can be consistently interpreted within the SMBHB framework. The QPO of about 2.1 years is likely attributed to the orbital motion in a SMBHB system. The Matérn-3/2 component is interpreted as resulting from magnetic reconnection events triggered by gravitational perturbations of the magnetic field within the jet, occurring as one black hole approaches the other. Meanwhile, in this case, the damping timescale of the common DRW kernel remains unconstrained due to the influence of new perturbations within the system.

astro-ph.HE↗
arXiv2025

ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents

Recent advancements in integrating large language models (LLMs) with application programming interfaces (APIs) have gained significant interest in both academia and industry. Recent work demonstrates that these API-based agents exhibit relatively strong autonomy and planning capabilities. However, their ability to handle multi-dimensional difficulty levels, diverse task types, and real-world demands remains unknown. In this paper, we introduce \textsc{ShortcutsBench}, a large-scale benchmark for the comprehensive evaluation of API-based agents in solving real-world complex tasks. \textsc{ShortcutsBench} includes a wealth of real APIs from Apple Inc., refined user queries, human-annotated high-quality action sequences, detailed parameter filling values, and parameters requesting necessary input from the system or user. We revealed how existing benchmarks~/~datasets struggle to accommodate the advanced reasoning capabilities of existing more intelligent LLMs. Moreover, our extensive evaluation of agents built with $5$ leading open-source (size $\geq$ 57B) and $5$ closed-source LLMs (e.g. Gemini-1.5-Pro and GPT-4o-mini) with varying intelligence level reveals significant limitations of existing API-based agents in the whole process of handling complex queries related to API selection, parameter filling, and requesting necessary input from the system and the user. These findings highlight the great challenges that API-based agents face in effectively fulfilling real and complex user queries. All datasets, code, experimental logs, and results are available at \url{https://github.com/EachSheep/ShortcutsBench}.

cs.SE↗
arXiv2025

ComptoNet: An End-to-End Deep Learning Framework for Scatter Estimation in Multi-Source Stationary CT

Multi-source stationary computed tomography (MSS-CT) offers significant advantages in medical and industrial applications due to its gantry-less scan architecture and/or capability of simultaneous multi-source emission. However, the lack of anti-scatter grid deployment in MSS-CT results in severe forward and/or cross scatter contamination, presenting a critical challenge that necessitates an accurate and efficient scatter correction. In this work, ComptoNet, an innovative end-to-end deep learning framework for scatter estimation in MSS-CT, is proposed, which integrates Compton-scattering physics with deep learning techniques to address the challenges of scatter estimation effectively. Central to ComptoNet is the Compton-map, a novel concept that captures the distribution of scatter signals outside the scan field of view, primarily consisting of large-angle Compton scatter. In ComptoNet, a reference Compton-map and/or spare detector data are used to guide the physics-driven deep estimation of scatter from simultaneous emissions by multiple sources. Additionally, a frequency attention module is employed for enhancing the low-frequency smoothness. Such a multi-source deep scatter estimation framework decouples the cross and forward scatter. It reduces network complexity and ensures a consistent low-frequency signature with different photon numbers of simulations, as evidenced by mean absolute percentage errors (MAPEs) that are less than $1.26\%$. Conducted by using data generated from Monte Carlo simulations with various phantoms, experiments demonstrate the effectiveness of ComptoNet, with significant improvements in scatter estimation accuracy (a MAPE of $0.84\%$). After scatter correction, nearly artifact-free CT images are obtained, further validating the capability of our proposed ComptoNet in mitigating scatter-induced errors.

physics.med-ph↗
arXiv2025

Adaptive Data Augmentation with NaturalSpeech3 for Far-field Speaker Verification

The scarcity of speaker-annotated far-field speech presents a significant challenge in developing high-performance far-field speaker verification (SV) systems. While data augmentation using large-scale near-field speech has been a common strategy to address this limitation, the mismatch in acoustic environments between near-field and far-field speech significantly hinders the improvement of far-field SV effectiveness. In this paper, we propose an adaptive speech augmentation approach leveraging NaturalSpeech3, a pre-trained foundation text-to-speech (TTS) model, to convert near-field speech into far-field speech by incorporating far-field acoustic ambient noise for data augmentation. Specifically, we utilize FACodec from NaturalSpeech3 to decompose the speech waveform into distinct embedding subspaces-content, prosody, speaker, and residual (acoustic details) embeddings-and reconstruct the speech waveform from these disentangled representations. In our method, the prosody, content, and residual embeddings of far-field speech are combined with speaker embeddings from near-field speech to generate augmented pseudo far-field speech that maintains the speaker identity from the out-domain near-field speech while preserving the acoustic environment of the in-domain far-field speech. This approach not only serves as an effective strategy for augmenting training data for far-field speaker verification but also extends to cross-data augmentation for enrollment and test speech in evaluation trials.Experimental results on FFSVC demonstrate that the adaptive data augmentation method significantly outperforms traditional approaches, such as random noise addition and reverberation, as well as other competitive data augmentation strategies.

cs.SD↗
arXiv2025

Subject Disentanglement Neural Network for Speech Envelope Reconstruction from EEG

Reconstructing speech envelopes from EEG signals is essential for exploring neural mechanisms underlying speech perception. Yet, EEG variability across subjects and physiological artifacts complicate accurate reconstruction. To address this problem, we introduce Subject Disentangling Neural Network (SDN-Net), which disentangles subject identity information from reconstructed speech envelopes to enhance cross-subject reconstruction accuracy. SDN-Net integrates three key components: MLA-Codec, MPN-MI, and CTA-MTDNN. The MLA-Codec, a fully convolutional neural network, decodes EEG signals into speech envelopes. The CTA-MTDNN module, a multi-scale time-delay neural network with channel and temporal attention, extracts subject identity features from EEG signals. Lastly, the MPN-MI module, a mutual information estimator with a multi-layer perceptron, supervises the removal of subject identity information from the reconstructed speech envelope. Experiments on the Auditory EEG Decoding Dataset demonstrate that SDN-Net achieves superior performance in inner- and cross-subject speech envelope reconstruction compared to recent state-of-the-art methods.

eess.SP↗
arXiv2025

Photonic antiferromagnetic topological insulator with a single surface Dirac cone

Antiferromagnetism, characterized by magnetic moments aligned in alternating directions with a vanished ensemble average, has garnered renewed interest for its potential applications in spintronics and axion dynamics. The synergy between antiferromagnetism and topology can lead to the emergence of an exotic topological phase unique to certain magnetic order, termed antiferromagnetic topological insulators (AF TIs). A hallmark signature of AF TIs is the presence of a single surface Dirac cone--a feature typically associated with strong three-dimensional (3D) topological insulators--only on certain symmetry-preserving crystal terminations. However, the direct observation of this phenomenon poses a significant challenge. Here, we have theoretically and experimentally discovered a 3D photonic AF TI hosting a single surface Dirac cone protected by the combined symmetry of time reversal and half-lattice translation. Conceptually, our setup can be viewed as a z-directional stack of two-dimensional Chern insulators, with adjacent layers oppositely magnetized to form a 3D type-A AF configuration. By measuring both bulk and surface states, we have directly observed the symmetry-protected gapless single-Dirac-cone surface state, which shows remarkable robustness against random magnetic disorders. Our work constitutes the first realization of photonic AF TIs and photonic analogs of strong topological insulators, opening a new chapter for exploring novel topological photonic devices and phenomena that incorporate additional magnetic degrees of freedom.

physics.optics↗
arXiv2025

DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theory

Quantizing the weights of a neural network has two steps: (1) Finding a good low bit-complexity representation for weights (which we call the quantization grid) and (2) Rounding the original weights to values in the quantization grid. In this paper, we study the problem of rounding optimally given any quantization grid. The simplest and most commonly used way to round is Round-to-Nearest (RTN). By rounding in a data-dependent way instead, one can improve the quality of the quantized model significantly. We study the rounding problem from the lens of \emph{discrepancy theory}, which studies how well we can round a continuous solution to a discrete solution without affecting solution quality too much. We prove that given $m=\mathrm{poly}(1/ε)$ samples from the data distribution, we can round all but $O(m)$ model weights such that the expected approximation error of the quantized model on the true data distribution is $\le ε$ as long as the space of gradients of the original model is approximately low rank (which we empirically validate). Our proof, which is algorithmic, inspired a simple and practical rounding algorithm called \emph{DiscQuant}. In our experiments, we demonstrate that DiscQuant significantly improves over the prior state-of-the-art rounding method called GPTQ and the baseline RTN over a range of benchmarks on Phi3mini-3.8B and Llama3.1-8B. For example, rounding Phi3mini-3.8B to a fixed quantization grid with 3.25 bits per parameter using DiscQuant gets 64\% accuracy on the GSM8k dataset, whereas GPTQ achieves 54\% and RTN achieves 31\% (the original model achieves 84\%). We make our code available at https://github.com/jerry-chee/DiscQuant.

cs.LG↗
arXiv2025

Photon-recycling dielectric laser accelerator

We propose a photon-recycling dielectric laser accelerator (DLA) system based on silicon photonic device. Our DLA system employs guided electromagnetic waves as a primary energy source, modulated to inject into the electron-light interaction region to accelerate or modulate electron beams and recycled the energy for the next round-trip. Long-distance acceleration takes place as electrons interact with the pre-modulated light field. Our loop recycles post-interaction light field, enabling photons reuse across successive cycles. To optimize the interaction process, we developed an adaptive algorithm to refine waveguide structures, and identified an "optimal waveguide accelerator" with superior performance on our dataset. We find that the optimized DLA loop only requires low-power light injection to sufficiently sustain high acceleration gradients for continuous electron beams. Under optimal electron beam intensity, the system achieves exceptionally high photon utilization, ensuring that nearly all injected light power transferred to electrons. Using spectral analysis, we demonstrate that the optimal waveguide also operates as an electron energy filter, selecting and manipulating phase-matched electrons over a broad energy range, even for quantum electron wavefunction shaping. Our photon-recycling DLA setup is not only suitable for low-energy beam accelerators, but also offers versatility as a beam filter or a narrow energy selection combined with other optical elements, the total setup can be further applied to explore free electron quantum optics engaging with the advancing field of photonic integrated circuits.

physics.optics↗
arXiv2025

Wireless Channel Measurements and Characterization in Industrial IoT Scenarios

Wireless Fidelity (Wi-Fi) communication technologies hold significant potential for realizing the Industrial Internet of Things (IIoT). In this paper, both Single-Input Single-Output (SISO) and polarized Multiple-Input Multiple-Output (MIMO) channel measurements are conducted in an IIoT scenario at the less congested Wi-Fi band, i.e., 5.5~GHz. The purpose is to investigate wireless characteristics of communications between access points and terminals mounted on automated guided vehicles as well as those surrounding manufacturing areas. For SISO channel measurements, statistical properties including the delay Power Spectral Density (PSD), path loss, shadowing fading, delay spread, excess delay, K-factor, and amplitude distribution of small-scale fading are analyzed and compared with those observed in an office scenario. For MIMO channel measurements, results show that there are multiple Dense Multipath Component (DMC) processes in the delay PSD. An estimation algorithm based on the algorithm for a single DMC process is proposed to effectively process the multi-processes data. Moreover, delay, angular, power, and polarization properties of DMCs are investigated and compared with those of specular multipath components. Furthermore, effects of DMCs on Singular Values (SVs) and channel capacities are explored. Ignoring DMCs can overestimate SVs and underestimate channel capacities.

eess.SY↗