arXiv subjects

Yang Li

Publications and source records attributed to Yang Li.

At least 685 records · Page 38Linked to original sources

arXiv2024

CollaMamba: Efficient Collaborative Perception with Cross-Agent Spatial-Temporal State Space Model

By sharing complementary perceptual information, multi-agent collaborative perception fosters a deeper understanding of the environment. Recent studies on collaborative perception mostly utilize CNNs or Transformers to learn feature representation and fusion in the spatial dimension, which struggle to handle long-range spatial-temporal features under limited computing and communication resources. Holistically modeling the dependencies over extensive spatial areas and extended temporal frames is crucial to enhancing feature quality. To this end, we propose a resource efficient cross-agent spatial-temporal collaborative state space model (SSM), named CollaMamba. Initially, we construct a foundational backbone network based on spatial SSM. This backbone adeptly captures positional causal dependencies from both single-agent and cross-agent views, yielding compact and comprehensive intermediate features while maintaining linear complexity. Furthermore, we devise a history-aware feature boosting module based on temporal SSM, extracting contextual cues from extended historical frames to refine vague features while preserving low overhead. Extensive experiments across several datasets demonstrate that CollaMamba outperforms state-of-the-art methods, achieving higher model accuracy while reducing computational and communication overhead by up to 71.9% and 1/64, respectively. This work pioneers the exploration of the Mamba's potential in collaborative perception. The source code will be made available.

cs.CV↗

arXiv2024

Atomistic modeling of diffusion processes at Al(Si)/Si(111) interphase boundaries obtained by vapor deposition

Molecular dynamics and parallel-replica dynamics simulations are applied to investigate the atomic structures and diffusion processes at {\text{Al}\{111\}}\parallel{\text{Si}}\{111\} interphase boundaries constructed by simulated vapor deposition of Al(Si) alloy on Si(111) substrates. Different orientation relationships and interface structures are obtained for different pre-deposition Si (111) surface reconstructions. Diffusion of both Al and Si atoms at the interfaces is calculated and compared with diffusion along grain boundaries, triple junctions, contact lines, and threading dislocations in the Al-Si system. It is found that {\text{Al}\{111\}}\parallel{\text{Si}}\{111\} interphase boundaries exhibit the lowest diffusivity among these structures and are closest to the lattice diffusivity. In most cases (except for the Si substrate), Si atoms are more mobile than Al atoms. The diffusion processes are typically mediated by Al vacancies and Si interstitial atoms migrating by either direct or indirect interstitial mechanisms.

cond-mat.mtrl-sci↗

arXiv2024

Enhancing Cyber-Resilience in Integrated Energy System Scheduling with Demand Response Using Deep Reinforcement Learning

Optimally scheduling multi-energy flow is an effective method to utilize renewable energy sources (RES) and improve the stability and economy of integrated energy systems (IES). However, the stable demand-supply of IES faces challenges from uncertainties that arise from RES and loads, as well as the increasing impact of cyber-attacks with advanced information and communication technologies adoption. To address these challenges, this paper proposes an innovative model-free resilience scheduling method based on state-adversarial deep reinforcement learning (DRL) for integrated demand response (IDR)-enabled IES. The proposed method designs an IDR program to explore the interaction ability of electricity-gas-heat flexible loads. Additionally, the state-adversarial Markov decision process (SA-MDP) model characterizes the energy scheduling problem of IES under cyber-attack, incorporating cyber-attacks as adversaries directly into the scheduling process. The state-adversarial soft actor-critic (SA-SAC) algorithm is proposed to mitigate the impact of cyber-attacks on the scheduling strategy, integrating adversarial training into the learning process to against cyber-attacks. Simulation results demonstrate that our method is capable of adequately addressing the uncertainties resulting from RES and loads, mitigating the impact of cyber-attacks on the scheduling strategy, and ensuring a stable demand supply for various energy sources. Moreover, the proposed method demonstrates resilience against cyber-attacks. Compared to the original soft actor-critic (SAC) algorithm, it achieves a 10% improvement in economic performance under cyber-attack scenarios.

eess.SY↗

arXiv2024

UniAR: A Unified model for predicting human Attention and Responses on visual content

Progress in human behavior modeling involves understanding both implicit, early-stage perceptual behavior, such as human attention, and explicit, later-stage behavior, such as subjective preferences or likes. Yet most prior research has focused on modeling implicit and explicit human behavior in isolation; and often limited to a specific type of visual content. We propose UniAR -- a unified model of human attention and preference behavior across diverse visual content. UniAR leverages a multimodal transformer to predict subjective feedback, such as satisfaction or aesthetic quality, along with the underlying human attention or interaction heatmaps and viewing order. We train UniAR on diverse public datasets spanning natural images, webpages, and graphic designs, and achieve SOTA performance on multiple benchmarks across various image domains and behavior modeling tasks. Potential applications include providing instant feedback on the effectiveness of UIs/visual content, and enabling designers and content-creation models to optimize their creation for human-centric improvements.

cs.CV↗

arXiv2024

VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks

As the adoption of large language models increases and the need for per-user or per-task model customization grows, the parameter-efficient fine-tuning (PEFT) methods, such as low-rank adaptation (LoRA) and its variants, incur substantial storage and transmission costs. To further reduce stored parameters, we introduce a "divide-and-share" paradigm that breaks the barriers of low-rank decomposition across matrix dimensions, modules, and layers by sharing parameters globally via a vector bank. As an instantiation of the paradigm to LoRA, our proposed VB-LoRA composites all the low-rank matrices of LoRA from a shared vector bank with a differentiable top-k admixture module. VB-LoRA achieves extreme parameter efficiency while maintaining comparable or better performance compared to state-of-the-art PEFT methods. Extensive experiments demonstrate the effectiveness of VB-LoRA on natural language understanding, natural language generation, instruction tuning, and mathematical reasoning tasks. When fine-tuning the Llama2-13B model, VB-LoRA only uses 0.4% of LoRA's stored parameters, yet achieves superior results. Our source code is available at https://github.com/leo-yangli/VB-LoRA. This method has been merged into the Hugging Face PEFT package.

cs.CL↗

arXiv2024

Singularity formations in Lagrangian mean curvature flow

We study singularities along the Lagrangian mean curvature flow with tangent flows given by multiplicity one special Lagrangian cones that are smooth away from the origin. Some results are: uniqueness of all such tangent flows in dimension two; uniqueness in any dimension when the link of the cone is connected; the existence of nontrivial special Lagrangian blowup limits. We also prove a singular version of Imagi-Joyce-dos Santos's uniqueness result of the Lawlor neck. As an application we prove that in any dimension, singularities that admit a tangent flow given by the union of two transverse planes is modeled on shrinking Lawlor necks at suitable scales.

math.DG↗

arXiv2024

Similarity-based context aware continual learning for spiking neural networks

Biological brains have the capability to adaptively coordinate relevant neuronal populations based on the task context to learn continuously changing tasks in real-world environments. However, existing spiking neural network-based continual learning algorithms treat each task equally, ignoring the guiding role of different task similarity associations for network learning, which limits knowledge utilization efficiency. Inspired by the context-dependent plasticity mechanism of the brain, we propose a Similarity-based Context Aware Spiking Neural Network (SCA-SNN) continual learning algorithm to efficiently accomplish task incremental learning and class incremental learning. Based on contextual similarity across tasks, the SCA-SNN model can adaptively reuse neurons from previous tasks that are beneficial for new tasks (the more similar, the more neurons are reused) and flexibly expand new neurons for the new task (the more similar, the fewer neurons are expanded). Selective reuse and discriminative expansion significantly improve the utilization of previous knowledge and reduce energy consumption. Extensive experimental results on CIFAR100, ImageNet generalized datasets, and FMNIST-MNIST, SVHN-CIFAR100 mixed datasets show that our SCA-SNN model achieves superior performance compared to both SNN-based and DNN-based continual learning algorithms. Additionally, our algorithm has the capability to adaptively select similar groups of neurons for related tasks, offering a promising approach to enhancing the biological interpretability of efficient continual learning.

cs.NE↗

arXiv2024

Aligning Individual and Collective Objectives in Multi-Agent Cooperation

Among the research topics in multi-agent learning, mixed-motive cooperation is one of the most prominent challenges, primarily due to the mismatch between individual and collective goals. The cutting-edge research is focused on incorporating domain knowledge into rewards and introducing additional mechanisms to incentivize cooperation. However, these approaches often face shortcomings such as the effort on manual design and the absence of theoretical groundings. To close this gap, we model the mixed-motive game as a differentiable game for the ease of illuminating the learning dynamics towards cooperation. More detailed, we introduce a novel optimization method named \textbf{\textit{A}}ltruistic \textbf{\textit{G}}radient \textbf{\textit{A}}djustment (\textbf{\textit{AgA}}) that employs gradient adjustments to progressively align individual and collective objectives. Furthermore, we theoretically prove that AgA effectively attracts gradients to stable fixed points of the collective objective while considering individual interests, and we validate these claims with empirical evidence. We evaluate the effectiveness of our algorithm AgA through benchmark environments for testing mixed-motive collaboration with small-scale agents such as the two-player public good game and the sequential social dilemma games, Cleanup and Harvest, as well as our self-developed large-scale environment in the game StarCraft II.

cs.MA↗

arXiv2024

Influential Language Data Selection via Gradient Trajectory Pursuit

Curating a desirable dataset for training has been the core of building highly capable large language models (Touvron et al., 2023; Achiam et al., 2023; Team et al.,2024). Gradient influence scores (Pruthi et al., 2020; Xia et al., 2024) are shown to be correlated with model performance and are commonly used as the criterion for data selection. However, existing methods are built upon either individual sample rankings or inefficient matching process, leading to suboptimal performance or scaling up issues.In this paper, we propose Gradient Trajectory Pursuit (GTP), an algorithm that performs pursuit of gradient trajectories via jointly selecting data points under an L0-norm regularized objective. The proposed algorithm highlights: (1) joint selection instead of independent top-k selection, which automatically de-duplicates samples; (2) higher efficiency with compressive sampling processes, which can be further sped up using a distributed framework. In the experiments, we demonstrate the algorithm in both in-domain and target-domain selection benchmarks and show that it outperforms top-k selection and competitive algorithms consistently, for example, our algorithm chooses as low as 0.5% data to achieve full performance on the targeted instruction tuning tasks

cs.AI↗

arXiv2024

Estimating Individual Dose-Response Curves under Unobserved Confounders from Observational Data

Estimating an individual's potential response to continuously varied treatments is crucial for addressing causal questions across diverse domains, from healthcare to social sciences. However, existing methods are limited either to estimating causal effects of binary treatments, or scenarios where all confounding variables are measurable. In this work, we present ContiVAE, a novel framework for estimating causal effects of continuous treatments, measured by individual dose-response curves, considering the presence of unobserved confounders using observational data. Leveraging a variational auto-encoder with a Tilted Gaussian prior distribution, ContiVAE models the hidden confounders as latent variables, and is able to predict the potential outcome of any treatment level for each individual while effectively capture the heterogeneity among individuals. Experiments on semi-synthetic datasets show that ContiVAE outperforms existing methods by up to 62%, demonstrating its robustness and flexibility. Application on a real-world dataset illustrates its practical utility.

cs.LG↗

arXiv2024

SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent

Public scarce resource allocation plays a crucial role in economics as it directly influences the efficiency and equity in society. Traditional studies including theoretical model-based, empirical study-based and simulation-based methods encounter limitations due to the idealized assumption of complete information and individual rationality, as well as constraints posed by limited available data. In this work, we propose an innovative framework, SRAP-Agent (Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent), which integrates Large Language Models (LLMs) into economic simulations, aiming to bridge the gap between theoretical models and real-world dynamics. Using public housing allocation scenarios as a case study, we conduct extensive policy simulation experiments to verify the feasibility and effectiveness of the SRAP-Agent and employ the Policy Optimization Algorithm with certain optimization objectives. The source code can be found in https://github.com/jijiarui-cather/SRAPAgent_Framework

cs.CL↗

arXiv2024

A further study on the mass formula for linear codes with prescribed hull dimension

Finding a mass formula for a given class of linear codes is a fundamental problem in combinatorics and coding theory. In this paper, we consider the action of the unitary (resp. symplectic) group on the set of all Hermitian (resp. symplectic) linear complementary dual (LCD) codes, prove that all Hermitian (resp. symplectic) LCD codes are on a unique orbit under this action, and determine the formula for the size of the orbit. Based on this, we develop a general technique to obtain a closed mass formula for linear codes with prescribed Hermitian (resp. symplectic) hull dimension, and further obtain some asymptotic results.

cs.IT↗

arXiv2024

Physical Informed-Inspired Deep Reinforcement Learning Based Bi-Level Programming for Microgrid Scheduling

To coordinate the interests of operator and users in a microgrid under complex and changeable operating conditions, this paper proposes a microgrid scheduling model considering the thermal flexibility of thermostatically controlled loads and demand response by leveraging physical informed-inspired deep reinforcement learning (DRL) based bi-level programming. To overcome the non-convex limitations of karush-kuhn-tucker (KKT)-based methods, a novel optimization solution method based on DRL theory is proposed to handle the bi-level programming through alternate iterations between levels. Specifically, by combining a DRL algorithm named asynchronous advantage actor-critic (A3C) and automated machine learning-prioritized experience replay (AutoML-PER) strategy to improve the generalization performance of A3C to address the above problems, an improved A3C algorithm, called AutoML-PER-A3C, is designed to solve the upper-level problem; while the DOCPLEX optimizer is adopted to address the lower-level problem. In this solution process, AutoML is used to automatically optimize hyperparameters and PER improves learning efficiency and quality by extracting the most valuable samples. The test results demonstrate that the presented approach manages to reconcile the interests between multiple stakeholders in MG by fully exploiting various flexibility resources. Furthermore, in terms of economic viability and computational efficiency, the proposal vastly exceeds other advanced reinforcement learning methods.

eess.SY↗

arXiv2024

SEED-Story: Multimodal Long Story Generation with Large Language Model

With the remarkable advancements in image generation and open-form text generation, the creation of interleaved image-text content has become an increasingly intriguing field. Multimodal story generation, characterized by producing narrative texts and vivid images in an interleaved manner, has emerged as a valuable and practical task with broad applications. However, this task poses significant challenges, as it necessitates the comprehension of the complex interplay between texts and images, and the ability to generate long sequences of coherent, contextually relevant texts and visuals. In this work, we propose SEED-Story, a novel method that leverages a Multimodal Large Language Model (MLLM) to generate extended multimodal stories. Our model, built upon the powerful comprehension capability of MLLM, predicts text tokens as well as visual tokens, which are subsequently processed with an adapted visual de-tokenizer to produce images with consistent characters and styles. We further propose multimodal attention sink mechanism to enable the generation of stories with up to 25 sequences (only 10 for training) in a highly efficient autoregressive manner. Additionally, we present a large-scale and high-resolution dataset named StoryStream for training our model and quantitatively evaluating the task of multimodal story generation in various aspects.

cs.CV↗

arXiv2024

Low Mach number limit for the global large solutions to the $2$D Navier--Stokes--Korteweg system in the critical $\widehat{L^p}$ framework

In the present paper, we consider the compressible Navier--Stokes--Korteweg system on the $2$D whole plane and show that a unique global solution exists in the scaling critical Fourier--Besov spaces for arbitrary large initial data provided that the Mach number is sufficiently small. Moreover, we also show that the global solution converges to the $2$D incompressible Navier--Stokes flow in the singular limit of zero Mach number. The key ingredient of the proof lies in the nonlinear stability estimates around the large incompressible flow via the Strichartz estimate for the linearized equations in Fourier--Besov spaces.

math.AP↗

arXiv2024

Linear and nonlinear stability for the $3$D stratified Boussinesq equations with the horizontal viscosity and diffusivity

In this manuscript, we consider the $3$D Boussinesq equations for stably stratified fluids with the horizontal viscosity and thermal diffusivity and investigate the large time behavior of the solutions. Making use of the anisotropic Littlewood--Paley theory, we obtain their precise $L^1$-$L^p$ decay estimates, which provide us information on both the anisotropic and dispersive structure of the system. More precisely, we reveal that the dispersion from the skew symmetric terms of stratification makes the decay rates of some portions of the solutions faster and furthermore the third component of the velocity field exhibit the enhanced dissipative effect, which provides the additional fast decay rate.

math.AP↗

arXiv2024

Cosmic Simulations of Axion String-Wall Networks: Probing Dark Matter and Gravitational Waves for Discovery

We simultaneously study gravitational waves (GWs) and free axions emitted from axionic string-wall networks in the early universe using advanced 3D lattice simulations. Our simulations start before the Peccei-Quinn phase transition and end with the destruction of string-wall networks below the QCD scale. The axion dark matter (DM) relic abundance radiated from string-wall networks are updated and refined for the scenarios of $N_{\rm DW}>1$. In this scenario, we observe that the GW spectrum is almost independent of the bias term and $N_{\rm DW}$, and $Ω_{\rm GW}h^2\propto f^{1.29}(f^{-0.43})$ in the IR and middle-frequency regions. After considering the constraints from DM relic abundance, we found that the QCD axion model predicts undetectable GW emissions, and the axion-like particles model allows for a detectable GW signal in the nano-Hertz to the milli-Hertz frequency range corresponding to axion masses range from KeV to TeV. For $N_{\rm DW}=1$, the GW energy density appears undetectable for QCD axions and axion-like particles.

astro-ph.CO↗

arXiv2024

Selecting the Best Sequential Transfer Path for Medical Image Segmentation with Limited Labeled Data

The medical image processing field often encounters the critical issue of scarce annotated data. Transfer learning has emerged as a solution, yet how to select an adequate source task and effectively transfer the knowledge to the target task remains challenging. To address this, we propose a novel sequential transfer scheme with a task affinity metric tailored for medical images. Considering the characteristics of medical image segmentation tasks, we analyze the image and label similarity between tasks and compute the task affinity scores, which assess the relatedness among tasks. Based on this, we select appropriate source tasks and develop an effective sequential transfer strategy by incorporating intermediate source tasks to gradually narrow the domain discrepancy and minimize the transfer cost. Thereby we identify the best sequential transfer path for the given target task. Extensive experiments on three MRI medical datasets, FeTS 2022, iSeg-2019, and WMH, demonstrate the efficacy of our method in finding the best source sequence. Compared with directly transferring from a single source task, the sequential transfer results underline a significant improvement in target task performance, achieving an average of 2.58% gain in terms of segmentation Dice score, notably, 6.00% for FeTS 2022. Code is available at the git repository.

eess.IV↗