arXiv subjects

Lei Yu

Publications and source records attributed to Lei Yu.

At least 91 records · Page 5Linked to original sources

arXiv2025

Large Deviation Analysis for the Reverse Shannon Theorem

Channel simulation is to simulate a noisy channel using noiseless channels with unlimited shared randomness. This can be interpreted as the reverse problem to Shannon's noisy coding theorem. In contrast to previous works, our approach employs Rényi divergence (with the parameter $α\in(0,\infty)$) to measure the level of approximation. Specifically, we obtain the reverse Shannon theorem under the Rényi divergence, which characterizes the Rényi simulation rate, the minimum communication cost rate required for the Rényi divergence vanishing asymptotically. We also investigate the behaviors of the Rényi divergence when the communication cost rate is above or below the Rényi simulation rate. When the communication cost rate is above the Rényi simulation rate, we provide a complete characterization of the convergence exponent, called the reliability function. When the communication cost rate is below the Rényi simulation rate, we determine the linear increasing rate for the Rényi divergence with parameter $α\in(0,\infty]$, which implies the strong converse exponent for the $α$-order fidelity.

cs.IT↗

arXiv2025

Holistic Large-Scale Scene Reconstruction via Mixed Gaussian Splatting

Recent advances in 3D Gaussian Splatting have shown remarkable potential for novel view synthesis. However, most existing large-scale scene reconstruction methods rely on the divide-and-conquer paradigm, which often leads to the loss of global scene information and requires complex parameter tuning due to scene partitioning and local optimization. To address these limitations, we propose MixGS, a novel holistic optimization framework for large-scale 3D scene reconstruction. MixGS models the entire scene holistically by integrating camera pose and Gaussian attributes into a view-aware representation, which is decoded into fine-detailed Gaussians. Furthermore, a novel mixing operation combines decoded and original Gaussians to jointly preserve global coherence and local fidelity. Extensive experiments on large-scale scenes demonstrate that MixGS achieves state-of-the-art rendering quality and competitive speed, while significantly reducing computational requirements, enabling large-scale scene reconstruction training on a single 24GB VRAM GPU. The code will be released at https://github.com/azhuantou/MixGS.

cs.CV↗

arXiv2025

Spa-VLM: Stealthy Poisoning Attacks on RAG-based VLM

With the rapid development of the Vision-Language Model (VLM), significant progress has been made in Visual Question Answering (VQA) tasks. However, existing VLM often generate inaccurate answers due to a lack of up-to-date knowledge. To address this issue, recent research has introduced Retrieval-Augmented Generation (RAG) techniques, commonly used in Large Language Models (LLM), into VLM, incorporating external multi-modal knowledge to enhance the accuracy and practicality of VLM systems. Nevertheless, the RAG in LLM may be susceptible to data poisoning attacks. RAG-based VLM may also face the threat of this attack. This paper first reveals the vulnerabilities of the RAG-based large model under poisoning attack, showing that existing single-modal RAG poisoning attacks have a 100\% failure rate in multi-modal RAG scenarios. To address this gap, we propose Spa-VLM (Stealthy Poisoning Attack on RAG-based VLM), a new paradigm for poisoning attacks on large models. We carefully craft malicious multi-modal knowledge entries, including adversarial images and misleading text, which are then injected into the RAG's knowledge base. When users access the VLM service, the system may generate misleading outputs. We evaluate Spa-VLM on two Wikipedia datasets and across two different RAGs. Results demonstrate that our method achieves highly stealthy poisoning, with the attack success rate exceeding 0.8 after injecting just 5 malicious entries into knowledge bases with 100K and 2M entries, outperforming state-of-the-art poisoning attacks designed for RAG-based LLMs. Additionally, we evaluated several defense mechanisms, all of which ultimately proved ineffective against Spa-VLM, underscoring the effectiveness and robustness of our attack.

cs.CR↗

arXiv2025

An Unsupervised Learning Method for Radio Interferometry Deconvolution

Given the incomplete sampling of spatial frequencies by radio interferometers, achieving precise restoration of astrophysical information remains challenging. To address this ill-posed problem, compressive sensing(CS) provides a robust framework for stable and unique recovery of sky brightness distributions in noisy environments, contingent upon satisfying specific conditions. We explore the applicability of CS theory and find that for radio interferometric telescopes, the conditions can be simplified to sparse representation. {Building on this insight, we develop a deep dictionary (realized through a convolutional neural network), which is designed to be multi-resolution and overcomplete, to achieve sparse representation and integrate it within the CS framework. The resulting method is a novel, fully interpretable unsupervised learning approach that combines} the mathematical rigor of CS with the expressive power of deep neural networks, effectively bridging the gap between deep learning and classical dictionary methods. {During the deconvolution process, the model image and the deep dictionary are updated alternatively.} This approach enables efficient and accurate recovery of extended sources with complex morphologies from noisy measurements. Comparative analyses with state-of-the-art algorithms demonstrate the outstanding performance of our method, i.e., achieving a dynamic range (DR) nearly 45 to 100 times higher than that of multiscale CLEAN (MS-CLEAN).

astro-ph.IM↗

arXiv2025

Rate-Distortion-Perception Theory for the Quadratic Wasserstein Space

We establish a single-letter characterization of the fundamental distortion-rate-perception tradeoff with limited common randomness under the squared error distortion measure and the squared Wasserstein-2 perception measure. Moreover, it is shown that this single-letter characterization can be explicitly evaluated for the Gaussian source. Various notions of universal representation are also clarified.

cs.IT↗

arXiv2025

Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations

LLMs often adopt an assertive language style also when making false claims. Such ``overconfident hallucinations'' mislead users and erode trust. Achieving the ability to express in language the actual degree of uncertainty around a claim is therefore of great importance. We find that ``verbal uncertainty'' is governed by a single linear feature in the representation space of LLMs, and show that this has only moderate correlation with the actual ``semantic uncertainty'' of the model. We apply this insight and show that (1) the mismatch between semantic and verbal uncertainty is a better predictor of hallucinations than semantic uncertainty alone and (2) we can intervene on verbal uncertainty at inference time and reduce confident hallucinations on short-form answers, achieving an average relative reduction of ~30%.

cs.CL↗

arXiv2025

Designing optimal subsidy schemes and recycling plans for sustainable treatment of construction and demolition waste

More than 10 billion tons of construction and demolition waste (CW) are generated globally each year, exerting a significant impact on the environment. In the CW recycling process, the government and the carrier are the two primary stakeholders. The carrier is responsible for transporting CW from production sites to backfill sites or processing facilities, with a primary focus on transport efficiency and revenue. Meanwhile, the government aims to minimize pollution from the recycling system, which is influenced by transport modes, shipment distances, and the processing methods used for CW. This paper develops a bi-objective, bi-level optimization model to address these challenges. The upper-level model is a linear programming model that optimizes the government's subsidy scheme, while the lower-level model is a minimum-cost flow model that optimizes the carrier's recycling plan. A hybrid heuristic solution method is proposed to tackle the problem's complexity. A case study in Chengdu, China, demonstrates the computational efficiency of the model and its small solution gap. With an optimized subsidy scheme and recycling plan, pollution can be reduced by over 29.29% through a relatively small investment in subsidies.

math.OC↗

arXiv2025

MOS: Towards Effective Smart Contract Vulnerability Detection through Mixture-of-Experts Tuning of Large Language Models

Smart contract vulnerabilities pose significant security risks to blockchain systems, potentially leading to severe financial losses. Existing methods face several limitations: (1) Program analysis-based approaches rely on predefined patterns, lacking flexibility for new vulnerability types; (2) Deep learning-based methods lack explanations; (3) Large language model-based approaches suffer from high false positives. We propose MOS, a smart contract vulnerability detection framework based on mixture-of-experts tuning (MOE-Tuning) of large language models. First, we conduct continual pre-training on a large-scale smart contract dataset to provide domain-enhanced initialization. Second, we construct a high-quality MOE-Tuning dataset through a multi-stage pipeline combining LLM generation and expert verification for reliable explanations. Third, we design a vulnerability-aware routing mechanism that activates the most relevant expert networks by analyzing code features and their matching degree with experts. Finally, we extend the feed-forward layers into multiple parallel expert networks, each specializing in specific vulnerability patterns. We employ a dual-objective loss function: one for optimizing detection and explanation performance, and another for ensuring reasonable distribution of vulnerability types to experts through entropy calculation. Experiments show that MOS significantly outperforms existing methods with average improvements of 6.32% in F1 score and 4.80% in accuracy. The vulnerability explanations achieve positive ratings (scores of 3-4 on a 4-point scale) of 82.96%, 85.21% and 94.58% for correctness, completeness, and conciseness through human and LLM evaluation.

cs.SE↗

arXiv2025

NTIRE 2025 Challenge on Event-Based Image Deblurring: Methods and Results

This paper presents an overview of NTIRE 2025 the First Challenge on Event-Based Image Deblurring, detailing the proposed methodologies and corresponding results. The primary goal of the challenge is to design an event-based method that achieves high-quality image deblurring, with performance quantitatively assessed using Peak Signal-to-Noise Ratio (PSNR). Notably, there are no restrictions on computational complexity or model size. The task focuses on leveraging both events and images as inputs for single-image deblurring. A total of 199 participants registered, among whom 15 teams successfully submitted valid results, offering valuable insights into the current state of event-based image deblurring. We anticipate that this challenge will drive further advancements in event-based vision research.

cs.CV↗

arXiv2025

Exploring Scene Affinity for Semi-Supervised LiDAR Semantic Segmentation

This paper explores scene affinity (AIScene), namely intra-scene consistency and inter-scene correlation, for semi-supervised LiDAR semantic segmentation in driving scenes. Adopting teacher-student training, AIScene employs a teacher network to generate pseudo-labeled scenes from unlabeled data, which then supervise the student network's learning. Unlike most methods that include all points in pseudo-labeled scenes for forward propagation but only pseudo-labeled points for backpropagation, AIScene removes points without pseudo-labels, ensuring consistency in both forward and backward propagation within the scene. This simple point erasure strategy effectively prevents unsupervised, semantically ambiguous points (excluded in backpropagation) from affecting the learning of pseudo-labeled points. Moreover, AIScene incorporates patch-based data augmentation, mixing multiple scenes at both scene and instance levels. Compared to existing augmentation techniques that typically perform scene-level mixing between two scenes, our method enhances the semantic diversity of labeled (or pseudo-labeled) scenes, thereby improving the semi-supervised performance of segmentation models. Experiments show that AIScene outperforms previous methods on two popular benchmarks across four settings, achieving notable improvements of 1.9% and 2.1% in the most challenging 1% labeled data.

cs.CV↗

arXiv2025

Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots

Multimodal Language Models (MMLMs) typically undergo post-training alignment to prevent harmful content generation. However, these alignment stages focus primarily on the assistant role, leaving the user role unaligned, and stick to a fixed input prompt structure of special tokens, leaving the model vulnerable when inputs deviate from these expectations. We introduce Role-Modality Attacks (RMA), a novel class of adversarial attacks that exploit role confusion between the user and assistant and alter the position of the image token to elicit harmful outputs. Unlike existing attacks that modify query content, RMAs manipulate the input structure without altering the query itself. We systematically evaluate these attacks across multiple Vision Language Models (VLMs) on eight distinct settings, showing that they can be composed to create stronger adversarial prompts, as also evidenced by their increased projection in the negative refusal direction in the residual stream, a property observed in prior successful attacks. Finally, for mitigation, we propose an adversarial training approach that makes the model robust against input prompt perturbations. By training the model on a range of harmful and benign prompts all perturbed with different RMA settings, it loses its sensitivity to Role Confusion and Modality Manipulation attacks and is trained to only pay attention to the content of the query in the input prompt structure, effectively reducing Attack Success Rate (ASR) while preserving the model's general utility.

cs.CR↗

arXiv2025

The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction

Large language models (LLMs) excel on a variety of reasoning benchmarks, but previous studies suggest they sometimes struggle to generalize to unseen questions, potentially due to over-reliance on memorized training examples. However, the precise conditions under which LLMs switch between reasoning and memorization during text generation remain unclear. In this work, we provide a mechanistic understanding of LLMs' reasoning-memorization dynamics by identifying a set of linear features in the model's residual stream that govern the balance between genuine reasoning and memory recall. These features not only distinguish reasoning tasks from memory-intensive ones but can also be manipulated to causally influence model performance on reasoning tasks. Additionally, we show that intervening in these reasoning features helps the model more accurately activate the most relevant problem-solving capabilities during answer generation. Our findings offer new insights into the underlying mechanisms of reasoning and memory in LLMs and pave the way for the development of more robust and interpretable generative AI systems.

cs.CL↗

arXiv2025

Robust LLM safeguarding via refusal feature adversarial training

Large language models (LLMs) are vulnerable to adversarial attacks that can elicit harmful responses. Defending against such attacks remains challenging due to the opacity of jailbreaking mechanisms and the high computational cost of training LLMs robustly. We demonstrate that adversarial attacks share a universal mechanism for circumventing LLM safeguards that works by ablating a dimension in the residual stream embedding space called the refusal feature. We further show that the operation of refusal feature ablation (RFA) approximates the worst-case perturbation of offsetting model safety. Based on these findings, we propose Refusal Feature Adversarial Training (ReFAT), a novel algorithm that efficiently performs LLM adversarial training by simulating the effect of input-level attacks via RFA. Experiment results show that ReFAT significantly improves the robustness of three popular LLMs against a wide range of adversarial attacks, with considerably less computational overhead compared to existing adversarial training methods.

cs.LG↗

arXiv2025

Leveraging Large Language Models for Enhanced Digital Twin Modeling: Trends, Methods, and Challenges

Digital twin technology is a transformative innovation driving the digital transformation and intelligent optimization of manufacturing systems. By integrating real-time data with computational models, digital twins enable continuous monitoring, simulation, prediction, and optimization, effectively bridging the gap between the physical and digital worlds. Recent advancements in communication, computing, and control technologies have accelerated the development and adoption of digital twins across various industries. However, significant challenges remain, including limited data for accurate system modeling, inefficiencies in system analysis, and a lack of explainability in the interactions between physical and digital systems. The rise of large language models (LLMs) offers new avenues to address these challenges. LLMs have shown exceptional capabilities across diverse domains, exhibiting strong generalization and emergent abilities that hold great potential for enhancing digital twins. This paper provides a comprehensive review of recent developments in LLMs and their applications to digital twin modeling. We propose a unified description-prediction-prescription framework to integrate digital twin modeling technologies and introduce a structured taxonomy to categorize LLM functionalities in these contexts. For each stage of application, we summarize the methodologies, identify key challenges, and explore potential future directions. To demonstrate the effectiveness of LLM-enhanced digital twins, we present an LLM-enhanced enterprise digital twin system, which enables automatic modeling and optimization of an enterprise. Finally, we discuss future opportunities and challenges in advancing LLM-enhanced digital twins, offering valuable insights for researchers and practitioners in related fields.

cs.ET↗

arXiv2025

Privacy and Accuracy-Aware AI/ML Model Deduplication

With the growing adoption of privacy-preserving machine learning algorithms, such as Differentially Private Stochastic Gradient Descent (DP-SGD), training or fine-tuning models on private datasets has become increasingly prevalent. This shift has led to the need for models offering varying privacy guarantees and utility levels to satisfy diverse user requirements. However, managing numerous versions of large models introduces significant operational challenges, including increased inference latency, higher resource consumption, and elevated costs. Model deduplication is a technique widely used by many model serving and database systems to support high-performance and low-cost inference queries and model diagnosis queries. However, none of the existing model deduplication works has considered privacy, leading to unbounded aggregation of privacy costs for certain deduplicated models and inefficiencies when applied to deduplicate DP-trained models. We formalize the problems of deduplicating DP-trained models for the first time and propose a novel privacy- and accuracy-aware deduplication mechanism to address the problems. We developed a greedy strategy to select and assign base models to target models to minimize storage and privacy costs. When deduplicating a target model, we dynamically schedule accuracy validations and apply the Sparse Vector Technique to reduce the privacy costs associated with private validation data. Compared to baselines that do not provide privacy guarantees, our approach improved the compression ratio by up to $35\times$ for individual models (including large language models and vision transformers). We also observed up to $43\times$ inference speedup due to the reduction of I/O operations.

cs.CR↗

arXiv2025

Frequency-Adaptive Low-Latency Object Detection Using Events and Frames

Fusing Events and RGB images for object detection leverages the robustness of Event cameras in adverse environments and the rich semantic information provided by RGB cameras. However, two critical mismatches: low-latency Events \textit{vs.}~high-latency RGB frames; temporally sparse labels in training \textit{vs.}~continuous flow in inference, significantly hinder the high-frequency fusion-based object detection. To address these challenges, we propose the \textbf{F}requency-\textbf{A}daptive Low-Latency \textbf{O}bject \textbf{D}etector (FAOD). FAOD aligns low-frequency RGB frames with high-frequency Events through an Align Module, which reinforces cross-modal style and spatial proximity to address the Event-RGB Mismatch. We further propose a training strategy, Time Shift, which enforces the module to align the prediction from temporally shifted Event-RGB pairs and their original representation, that is, consistent with Event-aligned annotations. This strategy enables the network to use high-frequency Event data as the primary reference while treating low-frequency RGB images as supplementary information, retaining the low-latency nature of the Event stream toward high-frequency detection. Furthermore, we observe that these corrected Event-RGB pairs demonstrate better generalization from low training frequency to higher inference frequencies compared to using Event data alone. Extensive experiments on the PKU-DAVIS-SOD and DSEC-Detection datasets demonstrate that our FAOD achieves SOTA performance. Specifically, in the PKU-DAVIS-SOD Dataset, FAOD achieves 9.8 points improvement in terms of the mAP in fully paired Event-RGB data with only a quarter of the parameters compared to SODFormer, and even maintains robust performance (only a 3 points drop in mAP) under 80$\times$ Event-RGB frequency mismatch.

cs.CV↗

arXiv2025

Sparse Brains are Also Adaptive Brains: Cognitive-Load-Aware Dynamic Activation for LLMs

Dense large language models(LLMs) face critical efficiency bottlenecks as they rigidly activate all parameters regardless of input complexity. While existing sparsity methods(static pruning or dynamic activation) address this partially, they either lack adaptivity to contextual or model structural demands or incur prohibitive computational overhead. Inspired by human brain's dual-process mechanisms - predictive coding (N400) for backbone sparsity and structural reanalysis (P600) for complex context - we propose CLADA, a \textit{\textbf{C}ognitive-\textbf{L}oad-\textbf{A}ware \textbf{D}ynamic \textbf{A}ctivation} framework that synergizes statistical sparsity with semantic adaptability. Our key insight is that LLM activations exhibit two complementary patterns: 1) \textit{Global statistical sparsity} driven by sequence-level prefix information, and 2) \textit{Local semantic adaptability} modulated by cognitive load metrics(e.g., surprisal and entropy). CLADA employs a hierarchical thresholding strategy: a baseline from offline error-controlled optimization ensures 40\%+ sparsity, dynamically adjusted by real-time cognitive signals. Evaluations across six mainstream LLMs and nine benchmarks demonstrate that CLADA achieves \textbf{~20\% average speedup with <2\% accuracy drop}, outperforming Griffin (5\%+ degradation) and TT (negligible speedup). Crucially, we establish the first formal connection between neurolinguistic event-related potential (ERP) components and LLM efficiency mechanisms through multi-level regression analysis ($R^2=0.17$ for sparsity-adaptation synergy). Requiring no retraining or architectural changes, CLADA offers a deployable solution for resource-aware LLM inference while advancing biologically-inspired AI design. Our code is available at \href{https://github.com/Oldify/CLADA}{CLADA}.

cs.CL↗

arXiv2025

On the Fundamental Limits of Integrated Sensing and Communications Under Logarithmic Loss

We study a unified information-theoretic framework for integrated sensing and communications (ISAC), applicable to both monostatic and bistatic sensing scenarios. Special attention is given to the case where the sensing receiver (Rx) is required to produce a "soft" estimate of the state sequence, with logarithmic loss serving as the performance metric. We derive lower and upper bounds on the capacity-distortion function, which delineates the fundamental tradeoff between communication rate and sensing distortion. These bounds coincide when the channel between the ISAC transmitter (Tx) and the communication Rx is degraded with respect to the channel between the ISAC Tx and the sensing Rx, or vice versa. Furthermore, we provide a complete characterization of the capacity-distortion function for an ISAC system that simultaneously transmits information over a binary-symmetric channel and senses additive Bernoulli states through another binary-symmetric channel. The Gaussian counterpart of this problem is also explored, which, together with a state-splitting trick, fully determines the capacity-distortion-power function under the squared error distortion measure.

cs.IT↗