arXiv subjects

Yang Yu

Publications and source records attributed to Yang Yu.

At least 145 records · Page 8Linked to original sources

arXiv2025

Pairing boost from enhanced spin-fermion coupling in the pseudogap regime

We perform a fluctuation analysis of the pairing interaction in the hole-doped Hubbard model within the dynamical cluster approximation. Our analysis reveals that spin-fluctuation-mediated pairing differs qualitatively in the over- and underdoped regimes. In the underdoped regime, spin fluctuations open a pseudogap. We show that in this regime the spin-fermion coupling mediates a giant attraction between antinodal fermions. This explains why superconductivity survives at underdoping in the Hubbard model and cuprates, despite the lack of coherent fermionic quasiparticles in the pseudogap regime.

cond-mat.str-el↗

arXiv2025

Generalist Reward Models: Found Inside Large Language Models

The alignment of Large Language Models (LLMs) is critically dependent on reward models trained on costly human preference data. While recent work explores bypassing this cost with AI feedback, these methods often lack a rigorous theoretical foundation. In this paper, we discover that a powerful generalist reward model is already latently present within any LLM trained via standard next-token prediction. We prove that this endogenous reward is not a heuristic, but is theoretically equivalent to a reward function learned through offline inverse reinforcement learning. This connection allows us to directly elicit a high-quality reward signal from a base (pre-trained or supervised fine-tuned) model without any further training. Critically, we also prove that subsequent reinforcement learning using this endogenous reward leads to a policy with a provably superior error bound compared to the base model. To our best knowledge, this is the first theoretical proof of the effectiveness of reinforcement learning for LLMs. Our experiments validate this theory, demonstrating that our method not only outperforms existing LLM-as-a-judge approaches but can also surpass explicitly trained reward models. These findings suggest that the reward modeling stage can be replaced by a principled method of eliciting the knowledge already captured during pre-training, heralding a more efficient, powerful, and scalable paradigm for LLMs alignment as well as multi-modal models.

cs.CL↗

arXiv2025

Multi-agent Embodied AI: Advances and Future Directions

Embodied artificial intelligence (Embodied AI) plays a pivotal role in the application of advanced technologies in the intelligent era, where AI systems are integrated with physical bodies that enable them to perceive, reason, and interact with their environments. Through the use of sensors for input and actuators for action, these systems can learn and adapt based on real-world feedback, allowing them to perform tasks effectively in dynamic and unpredictable environments. As techniques such as deep learning (DL), reinforcement learning (RL), and large language models (LLMs) mature, embodied AI has become a leading field in both academia and industry, with applications spanning robotics, healthcare, transportation, and manufacturing. However, most research has focused on single-agent systems that often assume static, closed environments, whereas real-world embodied AI must navigate far more complex scenarios. In such settings, agents must not only interact with their surroundings but also collaborate with other agents, necessitating sophisticated mechanisms for adaptation, real-time learning, and collaborative problem-solving. Despite increasing interest in multi-agent systems, existing research remains narrow in scope, often relying on simplified models that fail to capture the full complexity of dynamic, open environments for multi-agent embodied AI. Moreover, no comprehensive survey has systematically reviewed the advancements in this area. As embodied AI rapidly evolves, it is crucial to deepen our understanding of multi-agent embodied AI to address the challenges presented by real-world applications. To fill this gap and foster further development in the field, this paper reviews the current state of research, analyzes key contributions, and identifies challenges and future directions, providing insights to guide innovation and progress in this field.

cs.AI↗

arXiv2025

Fabrication of airbridges with gradient exposure

In superconducting quantum circuits, airbridges are critical for eliminating parasitic slotline modes of coplanar waveguide circuits and reducing crosstalks between direct current magnetic flux biases. Here, we present a technique for fabricating superconducting airbridges. With this technique, a single layer of photoresist is employed, and the gradient exposure process is used to define the profile of airbridges. In order to properly obtain the bridge profile, we design exposure dosage based on residual photoresist thickness and laser power calibrations. Compared with other airbridge fabrication techniques, the gradient exposure fabrication technique provides the ability to produce lossless superconducting airbridges with flexible size and, thus, is more suitable for large-scale superconducting quantum circuits. Furthermore, this method reduces the complexity of the fabrication process and provides a high fabrication yield.

quant-ph↗

arXiv2025

Broadband merged-element Josephson parametric amplifier

Broadband quantum-limited amplifiers are essential for quantum information processing, yet challenges in design and fabrication continue to hinder their widespread applications. Here, we introduce the broadband merged-element Josephson parametric amplifier in which the discrete parallel capacitor is directly integrated with the Josephson junctions. This merged-element design eliminates the shortcomings of discrete capacitors, simplifying the fabrication process, reducing the need for high-precision lithography tools, and ensuring compatibility with standard superconducting qubit fabrication procedures. Experimental results demonstrate a gain of 15 dB over a 500 MHz bandwidth, a mean saturation power of -116 dBm and near-quantum-limited noise performance. This robust readily implemented parametric amplifier holds significant promise for broader applications in superconducting quantum information and the advancement of quantum computation.

quant-ph↗

arXiv2025

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference

Learning reward models from human preference datasets and subsequently optimizing language models via reinforcement learning has emerged as a fundamental paradigm for aligning LLMs with human preferences. The performance of the reward model plays a crucial role in the effectiveness of alignment. Previous reward models operate at a coarse-grained level, requiring the generation of a complete response to obtain a reward value. The sparse reward may present challenges for downstream reinforcement learning. While recent efforts have attempted to learn token-level reward models, the lack of explicit semantic information makes it difficult to model the credit of every individual token. In this paper, we propose assigning scores to every sentence, introducing an intermediate-grained reward model. By segmenting the complete response into sentences and applying differential operations to reward output at the start and end positions of each sentence, we can effectively model the rewards of sentences. Moreover, a novel attention mechanism is introduced to aggregate the scores of all sentences into a response-level score, which allows it to be trained using the Bradley-Terry model. On common benchmarks, our method outperforms the response-level reward model by 2.7% on RewardBench (for reward modeling evaluation) and surpasses all baselines on AlpacaEval (for alignment evaluation).

cs.CL↗

arXiv2025

A Survey on Large Language Models for Mathematical Reasoning

Mathematical reasoning has long represented one of the most fundamental and challenging frontiers in artificial intelligence research. In recent years, large language models (LLMs) have achieved significant advances in this area. This survey examines the development of mathematical reasoning abilities in LLMs through two high-level cognitive phases: comprehension, where models gain mathematical understanding via diverse pretraining strategies, and answer generation, which has progressed from direct prediction to step-by-step Chain-of-Thought (CoT) reasoning. We review methods for enhancing mathematical reasoning, ranging from training-free prompting to fine-tuning approaches such as supervised fine-tuning and reinforcement learning, and discuss recent work on extended CoT and "test-time scaling". Despite notable progress, fundamental challenges remain in terms of capacity, efficiency, and generalization. To address these issues, we highlight promising research directions, including advanced pretraining and knowledge augmentation techniques, formal reasoning frameworks, and meta-generalization through principled learning paradigms. This survey tries to provide some insights for researchers interested in enhancing reasoning capabilities of LLMs and for those seeking to apply these techniques to other domains.

cs.AI↗

arXiv2025

DTVM: Revolutionizing Smart Contract Execution with Determinism and Compatibility

We introduce the DeTerministic Virtual Machine (DTVM) Stack, a next-generation smart contract execution framework designed to address critical performance, determinism, and ecosystem compatibility challenges in blockchain networks. Building upon WebAssembly (Wasm) while maintaining full Ethereum Virtual Machine (EVM) ABI compatibility, DTVM introduces a Deterministic Middle Intermediate Representation (dMIR) and a hybrid lazy-JIT compilation engine to balance compilation speed and execution efficiency. DTVM further accommodates diverse instruction set architectures (e.g., EVM, RISC-V) through modular adaptation layers. This enables seamless integration with DTVM's hybrid lazy-JIT compilation engine, which dynamically optimizes performance while preserving deterministic execution guarantees across heterogeneous environments. The key contributions including: 1). The framework achieves up to 2$\times$ acceleration over evmone in dominant Ethereum contract (e.g. ERC20/721/1155) execution and reduces fibonacci computation latency by 11.8$\sim$40.5% compared to Wasm based VMs. 2). A novel trampoline hot-switch mechanism enables sub-millisecond (0.95ms) post-deployment invocation times, outperforming up to about 23$\times$ in compilation and invocation efficiency. 3). It supports multi-language development (Solidity, C++, Rust, Java, Go, and AssemblyScript) through unified bytecode conversion while maintaining EVM ABI compatibility for seamless invocation. It reduces machine code object sizes by 30.0$\sim$72.6%, coupled with a minimized Trusted Computing Base. 4). It offers SmartCogent, an AI-driven full-stack development experience, leveraging fine-tuned LLMs and retrieval-augmented generation to automate tasks across the smart contract lifecycle: development, debugging, security auditing, and deployment. DTVM Stack has been open-sourced (https://github.com/DTVMStack).

cs.DC↗

arXiv2025

CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large Language Models

We introduce CCI4.0, a large-scale bilingual pre-training dataset engineered for superior data quality and diverse human-like reasoning trajectory. CCI4.0 occupies roughly $35$ TB of disk space and comprises two sub-datasets: CCI4.0-M2-Base and CCI4.0-M2-CoT. CCI4.0-M2-Base combines a $5.2$ TB carefully curated Chinese web corpus, a $22.5$ TB English subset from Nemotron-CC, and diverse sources from math, wiki, arxiv, and code. Although these data are mostly sourced from well-processed datasets, the quality standards of various domains are dynamic and require extensive expert experience and labor to process. So, we propose a novel pipeline justifying data quality mainly based on models through two-stage deduplication, multiclassifier quality scoring, and domain-aware fluency filtering. We extract $4.5$ billion pieces of CoT(Chain-of-Thought) templates, named CCI4.0-M2-CoT. Differing from the distillation of CoT from larger models, our proposed staged CoT extraction exemplifies diverse reasoning patterns and significantly decreases the possibility of hallucination. Empirical evaluations demonstrate that LLMs pre-trained in CCI4.0 benefit from cleaner, more reliable training signals, yielding consistent improvements in downstream tasks, especially in math and code reflection tasks. Our results underscore the critical role of rigorous data curation and human thinking templates in advancing LLM performance, shedding some light on automatically processing pretraining corpora.

cs.CL↗

arXiv2025

Variational quantum algorithms with invariant probabilistic error cancellation on noisy quantum processors

In the noisy intermediate-scale quantum era, emerging classical-quantum hybrid optimization algorithms, such as variational quantum algorithms (VQAs), can leverage the unique characteristics of quantum devices to accelerate computations tailored to specific problems with shallow circuits. However, these algorithms encounter biases and iteration difficulties due to significant noise in quantum processors. These difficulties can only be partially addressed without error correction by optimizing hardware, reducing circuit complexity, or fitting and extrapolation. A compelling solution is applying probabilistic error cancellation (PEC), a quantum error mitigation technique that enables unbiased results without full error correction. Traditional PEC is challenging to apply in VQAs due to its variance amplification, contradicting iterative process assumptions. This paper proposes a novel noise-adaptable strategy that combines PEC with the quantum approximate optimization algorithm (QAOA). It is implemented through invariant sampling circuits (invariant-PEC, or IPEC) and substantially reduces iteration variance. This strategy marks the first successful integration of PEC and QAOA, resulting in efficient convergence. Moreover, we introduce adaptive partial PEC (APPEC), which modulates the error cancellation proportion of IPEC during iteration. We experimentally validated this technique on a superconducting quantum processor, cutting sampling cost by 90.1\%. Notably, we find that dynamic adjustments of error levels via APPEC can enhance escape from local minima and reduce sampling costs. These results open promising avenues for executing VQAs with large-scale, low-noise quantum circuits, paving the way for practical quantum computing advancements.

quant-ph↗

arXiv2025

Minimal pole representation for spectral functions

Representing spectral densities, real-frequency, and real-time Green's functions of continuous systems by a small discrete set of complex poles is an ubiquitous problem in condensed matter physics, with applications ranging from quantum transport simulations to the simulation of strongly correlated electron systems. This paper introduces a method for obtaining a compact, approximate representation of these functions, based on their parameterization on the real axis and a given approximate precision. We show applications to typical spectral functions and results for structured and unstructured correlation functions of model systems.

cond-mat.str-el↗

arXiv2025

Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning

Behavior regularization, which constrains the policy to stay close to some behavior policy, is widely used in offline reinforcement learning (RL) to manage the risk of hazardous exploitation of unseen actions. Nevertheless, existing literature on behavior-regularized RL primarily focuses on explicit policy parameterizations, such as Gaussian policies. Consequently, it remains unclear how to extend this framework to more advanced policy parameterizations, such as diffusion models. In this paper, we introduce BDPO, a principled behavior-regularized RL framework tailored for diffusion-based policies, thereby combining the expressive power of diffusion policies and the robustness provided by regularization. The key ingredient of our method is to calculate the Kullback-Leibler (KL) regularization analytically as the accumulated discrepancies in reverse-time transition kernels along the diffusion trajectory. By integrating the regularization, we develop an efficient two-time-scale actor-critic RL algorithm that produces the optimal policy while respecting the behavior constraint. Comprehensive evaluations conducted on synthetic 2D tasks and continuous control tasks from the D4RL benchmark validate its effectiveness and superior performance.

cs.LG↗

arXiv2025

Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistency

We present a novel statistical framework for analyzing stereotypes in large language models (LLMs) by systematically estimating the bias and variation in their generation. Current alignment evaluation metrics often overlook stereotypes' randomness caused by LLMs' inconsistent generative behavior. For instance, LLMs may display contradictory stereotypes, such as those related to gender or race, for identical professions in different contexts. Ignoring this inconsistency risks misleading conclusions in alignment assessments and undermines efforts to evaluate the potential of LLMs to perpetuate or amplify social biases and unfairness. To address this, we propose the Bias-Volatility Framework (BVF), which estimates the probability distribution of stereotypes in LLM outputs. By capturing the variation in generative behavior, BVF assesses both the likelihood and degree to which LLM outputs negatively impact vulnerable groups, enabling a quantification of aggregated discrimination risk. Additionally, we introduce a mathematical framework to decompose this risk into bias risk (from the mean of the stereotype distribution) and volatility risk (from its variation). Applying BVF to 12 widely used LLMs, we find: i) Bias risk is the dominant contributor to discrimination; ii) Most LLMs exhibit substantial pro-male stereotypes across nearly all professions; iii) Reinforcement learning from human feedback reduces bias but increases volatility; iv) Discrimination risk correlates with socio-economic factors, such as professional salaries. Finally, we highlight BVF's broader applicability for assessing how generation inconsistencies in LLMs impact behavior beyond stereotypes.

cs.CL↗

arXiv2025

SceneTracker: Long-term Scene Flow Estimation Network

Considering that scene flow estimation has the capability of the spatial domain to focus but lacks the coherence of the temporal domain, this study proposes long-term scene flow estimation (LSFE), a comprehensive task that can simultaneously capture the fine-grained and long-term 3D motion in an online manner. We introduce SceneTracker, the first LSFE network that adopts an iterative approach to approximate the optimal 3D trajectory. The network dynamically and simultaneously indexes and constructs appearance correlation and depth residual features. Transformers are then employed to explore and utilize long-range connections within and between trajectories. With detailed experiments, SceneTracker shows superior capabilities in addressing 3D spatial occlusion and depth noise interference, highly tailored to the needs of the LSFE task. We build a real-world evaluation dataset, LSFDriving, for the LSFE field and use it in experiments to further demonstrate the advantage of SceneTracker in generalization abilities. The code and data are available at https://github.com/wwsource/SceneTracker.

cs.CV↗

arXiv2025

Learnware of Language Models: Specialized Small Language Models Can Do Big

The learnware paradigm offers a novel approach to machine learning by enabling users to reuse a set of well-trained models for tasks beyond the models' original purposes. It eliminates the need to build models from scratch, instead relying on specifications (representations of a model's capabilities) to identify and leverage the most suitable models for new tasks. While learnware has proven effective in many scenarios, its application to language models has remained largely unexplored. At the same time, large language models (LLMs) have demonstrated remarkable universal question-answering abilities, yet they face challenges in specialized scenarios due to data scarcity, privacy concerns, and high computational costs, thus more and more specialized small language models (SLMs) are being trained for specific domains. To address these limitations systematically, the learnware paradigm provides a promising solution by enabling maximum utilization of specialized SLMs, and allowing users to identify and reuse them in a collaborative and privacy-preserving manner. This paper presents a preliminary attempt to apply the learnware paradigm to language models. We simulated a learnware system comprising approximately 100 learnwares of specialized SLMs with 8B parameters, fine-tuned across finance, healthcare, and mathematics domains. Each learnware contains an SLM and a specification, which enables users to identify the most relevant models without exposing their own data. Experimental results demonstrate promising performance: by selecting one suitable learnware for each task-specific inference, the system outperforms the base SLMs on all benchmarks. Compared to LLMs, the system outperforms Qwen1.5-110B, Qwen2.5-72B, and Llama3.1-70B-Instruct by at least 14% in finance domain tasks, and surpasses Flan-PaLM-540B (ranked 7th on the Open Medical LLM Leaderboard) in medical domain tasks.

cs.LG↗

arXiv2025

ImagineBench: Evaluating Reinforcement Learning with Large Language Model Rollouts

A central challenge in reinforcement learning (RL) is its dependence on extensive real-world interaction data to learn task-specific policies. While recent work demonstrates that large language models (LLMs) can mitigate this limitation by generating synthetic experience (noted as imaginary rollouts) for mastering novel tasks, progress in this emerging field is hindered due to the lack of a standard benchmark. To bridge this gap, we introduce ImagineBench, the first comprehensive benchmark for evaluating offline RL algorithms that leverage both real rollouts and LLM-imaginary rollouts. The key features of ImagineBench include: (1) datasets comprising environment-collected and LLM-imaginary rollouts; (2) diverse domains of environments covering locomotion, robotic manipulation, and navigation tasks; and (3) natural language task instructions with varying complexity levels to facilitate language-conditioned policy learning. Through systematic evaluation of state-of-the-art offline RL algorithms, we observe that simply applying existing offline RL algorithms leads to suboptimal performance on unseen tasks, achieving 35.44% success rate in hard tasks in contrast to 64.37% of method training on real rollouts for hard tasks. This result highlights the need for algorithm advancements to better leverage LLM-imaginary rollouts. Additionally, we identify key opportunities for future research: including better utilization of imaginary rollouts, fast online adaptation and continual learning, and extension to multi-modal tasks. Our code is publicly available at https://github.com/LAMDA-RL/ImagineBench.

cs.LG↗

arXiv2025

Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy

As a key component to intuitive cognition and reasoning solutions in human intelligence, causal knowledge provides great potential for reinforcement learning (RL) agents' interpretability towards decision-making by helping reduce the searching space. However, there is still a considerable gap in discovering and incorporating causality into RL, which hinders the rapid development of causal RL. In this paper, we consider explicitly modeling the generation process of states with the causal graphical model, based on which we augment the policy. We formulate the causal structure updating into the RL interaction process with active intervention learning of the environment. To optimize the derived objective, we propose a framework with theoretical performance guarantees that alternates between two steps: using interventions for causal structure learning during exploration and using the learned causal structure for policy guidance during exploitation. Due to the lack of public benchmarks that allow direct intervention in the state space, we design the root cause localization task in our simulated fault alarm environment and then empirically show the effectiveness and robustness of the proposed method against state-of-the-art baselines. Theoretical analysis shows that our performance improvement attributes to the virtuous cycle of causal-guided policy learning and causal structure learning, which aligns with our experimental results. Codes are available at https://github.com/DMIRLAB-Group/FaultAlarm_RL.

cs.LG↗

arXiv2025

New constraints on singlet scalar dark matter model with LZ, invisible Higgs decay and gamma-ray line observations

The singlet scalar dark matter (DM) model is a minimal extension of the Standard Model (SM). This model features only two free parameters: the singlet scalar mass $m_S$ and the quartic coupling $a_2$ between the singlet scalar and the SM Higgs doublet. Previous studies have suggested that only the resonant region remains viable under combined constraints. In this work, we revisit and refine the constraints using data from the direct detection experiment LZ, invisible Higgs decay measurements, and gamma-ray line observations conducted by Fermi-LAT and DAMPE. Our findings indicate that the latest results from LZ have significantly strengthened the constraints on the model parameters. We demonstrate that only a narrow parameter region remains viable, specifically $60.5\,\text{GeV}< m_S < 62.5\,\text{GeV}$ and $1.7\times10^{-4}<a_2< 4.7\times10^{-4}$. This suggests that a three-fold improvement in the current LZ results would thoroughly examine the parameter space below $1\,\rm{TeV}$ for the singlet scalar DM model.

hep-ph↗