arXiv subjects

Derong Xu

Publications and source records attributed to Derong Xu.

54 records · Page 3Linked to original sources

arXiv2025

Measure Domain's Gap: A Similar Domain Selection Principle for Multi-Domain Recommendation

Multi-Domain Recommendation (MDR) achieves the desirable recommendation performance by effectively utilizing the transfer information across different domains. Despite the great success, most existing MDR methods adopt a single structure to transfer complex domain-shared knowledge. However, the beneficial transferring information should vary across different domains. When there is knowledge conflict between domains or a domain is of poor quality, unselectively leveraging information from all domains will lead to a serious Negative Transfer Problem (NTP). Therefore, how to effectively model the complex transfer relationships between domains to avoid NTP is still a direction worth exploring. To address these issues, we propose a simple and dynamic Similar Domain Selection Principle (SDSP) for multi-domain recommendation in this paper. SDSP presents the initial exploration of selecting suitable domain knowledge for each domain to alleviate NTP. Specifically, we propose a novel prototype-based domain distance measure to effectively model the complexity relationship between domains. Thereafter, the proposed SDSP can dynamically find similar domains for each domain based on the supervised signals of the domain metrics and the unsupervised distance measure from the learned domain prototype. We emphasize that SDSP is a lightweight method that can be incorporated with existing MDR methods for better performance while not introducing excessive time overheads. To the best of our knowledge, it is the first solution that can explicitly measure domain-level gaps and dynamically select appropriate domains in the MDR field. Extensive experiments on three datasets demonstrate the effectiveness of our proposed method.

cs.IR↗

arXiv2025

Beam Optics Ramping in Under-Constrained Lattice Design: Application to Electron-Ion Collider Hadron Storage Ring Cooling Section

This paper presents the lattice design and optics ramping strategy for the cooling section of the Hadron Storage Ring (HSR) at the Electron-Ion Collider (EIC). The main challenge is that available tuning knobs exceed beam-optics constraints. Independently optimized injection and top-energy optics often yield disconnected solutions, making interpolation impossible. To address this, we propose two new methods. The first is a midpoint-penalty scheme that ensures ramping path continuity by penalizing constraint violations at intermediate state. The second is a top-down approach that adapts high-energy optics to low energy, guided by an adaptive weighting scheme to balance injection and ramping constraints. The solutions meet all beam dynamics and hardware limits. The two methods offer a general strategy for ramping in systems where the solution space is under-constrained and the starting and target configurations are far apart.

physics.acc-ph↗

arXiv2025

A Survey of Personalization: From RAG to Agent

Personalization has become an essential capability in modern AI systems, enabling customized interactions that align with individual user preferences, contexts, and goals. Recent research has increasingly concentrated on Retrieval-Augmented Generation (RAG) frameworks and their evolution into more advanced agent-based architectures within personalized settings to enhance user satisfaction. Building on this foundation, this survey systematically examines personalization across the three core stages of RAG: pre-retrieval, retrieval, and generation. Beyond RAG, we further extend its capabilities into the realm of Personalized LLM-based Agents, which enhance traditional RAG systems with agentic functionalities, including user understanding, personalized planning and execution, and dynamic generation. For both personalization in RAG and agent-based personalization, we provide formal definitions, conduct a comprehensive review of recent literature, and summarize key datasets and evaluation metrics. Additionally, we discuss fundamental challenges, limitations, and promising research directions in this evolving field. Relevant papers and resources are continuously updated at https://github.com/Applied-Machine-Learning-Lab/Awesome-Personalized-RAG-Agent.

cs.IR↗

arXiv2025

Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation

Large Language Models (LLMs) demonstrate remarkable capabilities, yet struggle with hallucination and outdated knowledge when tasked with complex knowledge reasoning, resulting in factually incorrect outputs. Previous studies have attempted to mitigate it by retrieving factual knowledge from large-scale knowledge graphs (KGs) to assist LLMs in logical reasoning and prediction of answers. However, this kind of approach often introduces noise and irrelevant data, especially in situations with extensive context from multiple knowledge aspects. In this way, LLM attention can be potentially mislead from question and relevant information. In our study, we introduce an Adaptive Multi-Aspect Retrieval-augmented over KGs (Amar) framework. This method retrieves knowledge including entities, relations, and subgraphs, and converts each piece of retrieved text into prompt embeddings. The Amar framework comprises two key sub-components: 1) a self-alignment module that aligns commonalities among entities, relations, and subgraphs to enhance retrieved text, thereby reducing noise interference; 2) a relevance gating module that employs a soft gate to learn the relevance score between question and multi-aspect retrieved data, to determine which information should be used to enhance LLMs' output, or even filtered altogether. Our method has achieved state-of-the-art performance on two common datasets, WebQSP and CWQ, showing a 1.9\% improvement in accuracy over its best competitor and a 6.6\% improvement in logical form generation over a method that directly uses retrieved text as context prompts. These results demonstrate the effectiveness of Amar in improving the reasoning of LLMs.

cs.CL↗

arXiv2024

Large Language Models for Generative Information Extraction: A Survey

Information extraction (IE) aims to extract structural knowledge from plain natural language texts. Recently, generative Large Language Models (LLMs) have demonstrated remarkable capabilities in text understanding and generation. As a result, numerous works have been proposed to integrate LLMs for IE tasks based on a generative paradigm. To conduct a comprehensive systematic review and exploration of LLM efforts for IE tasks, in this study, we survey the most recent advancements in this field. We first present an extensive overview by categorizing these works in terms of various IE subtasks and techniques, and then we empirically analyze the most advanced methods and discover the emerging trend of IE tasks with LLMs. Based on a thorough review conducted, we identify several insights in technique and promising research directions that deserve further exploration in future studies. We maintain a public repository and consistently update related works and resources on GitHub (\href{https://github.com/quqxui/Awesome-LLM4IE-Papers}{LLM4IE repository})

cs.CL↗

arXiv2024

Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding

The impressive capabilities of large language models (LLMs) have attracted extensive interests of applying LLMs to medical field. However, the complex nature of clinical environments presents significant hallucination challenges for LLMs, hindering their widespread adoption. In this paper, we address these hallucination issues in the context of Medical Information Extraction (MIE) tasks by introducing ALternate Contrastive Decoding (ALCD). We begin by redefining MIE tasks as an identify-and-classify process. We then separate the identification and classification functions of LLMs by selectively masking the optimization of tokens during fine-tuning. During the inference stage, we alternately contrast output distributions derived from sub-task models. This approach aims to selectively enhance the identification and classification capabilities while minimizing the influence of other inherent abilities in LLMs. Additionally, we propose an alternate adaptive constraint strategy to more effectively adjust the scale and scope of contrastive tokens. Through comprehensive experiments on two different backbones and six diverse medical information extraction tasks, ALCD demonstrates significant improvements in resolving hallucination issues compared to conventional decoding methods.

cs.CL↗

arXiv2024

Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models

Model editing aims to precisely alter the behaviors of large language models (LLMs) in relation to specific knowledge, while leaving unrelated knowledge intact. This approach has proven effective in addressing issues of hallucination and outdated information in LLMs. However, the potential of using model editing to modify knowledge in the medical field remains largely unexplored, even though resolving hallucination is a pressing need in this area. Our observations indicate that current methods face significant challenges in dealing with specialized and complex knowledge in medical domain. Therefore, we propose MedLaSA, a novel Layer-wise Scalable Adapter strategy for medical model editing. MedLaSA harnesses the strengths of both adding extra parameters and locate-then-edit methods for medical model editing. We utilize causal tracing to identify the association of knowledge in neurons across different layers, and generate a corresponding scale set from the association value for each piece of knowledge. Subsequently, we incorporate scalable adapters into the dense layers of LLMs. These adapters are assigned scaling values based on the corresponding specific knowledge, which allows for the adjustment of the adapter's weight and rank. The more similar the content, the more consistent the scale between them. This ensures precise editing of semantically identical knowledge while avoiding impact on unrelated knowledge. To evaluate the editing impact on the behaviours of LLMs, we propose two model editing studies for medical domain: (1) editing factual knowledge for medical specialization and (2) editing the explanatory ability for complex knowledge. We build two novel medical benchmarking datasets and introduce a series of challenging and comprehensive metrics. Extensive experiments on medical LLMs demonstrate the editing efficiency of MedLaSA, without affecting unrelated knowledge.

cs.CL↗

arXiv2024

Online regularization of Poincaré map of storage rings with Shannon entropy

Shannon entropy, as a chaos indicator, is used for online Poincaré map regularization and dynamic aperture optimization in the National Synchrotron Light Source-II (NSLS-II) ring. Although various chaos indicators are widely used in studying nonlinear dynamical systems, including modern particle accelerators, it is the first time to use a measurable one in a real-world machine for online nonlinear optimization. Poincaré maps, constructed with the turn-by-turn beam trajectory readings from beam position monitors, are commonly used to observe the nonlinearity in ring-based accelerators. However, such observations typically only provide a qualitative interpretation. We analyze their entropy to quantify the chaos in measured Poincaré maps. After some canonical transformations on the Poincaré maps, not only can the commonly used nonlinear characterizations be extracted, but more importantly, the chaos can be quantitatively calibrated with Shannon entropy, and then used as the online optimization objectives.

physics.acc-ph↗

arXiv2024

Enhanced beam-beam modeling to include longitudinal variation during weak-strong simulation

Beam-beam interactions pose substantial challenges in the design and operation of circular colliders, significantly affecting their performance. In particular, the weak-strong simulation approach is pivotal for investigating single-particle dynamics during the collider design phase. This paper evaluates the limitations of existing models in weak-strong simulations, noting that while they accurately account for energy changes due to slingshot effects, they fail to incorporate longitudinal coordinate changes ($z$-variation). To address this gap, we introduce two novel transformations that enhance Hirata's original framework by including both $z$-variation and slingshot effect-induced energy changes. Through rigorous mathematical analysis and extensive weak-strong simulation studies, we validate the efficacy of these enhancements in achieving a more precise simulation of beam-beam interactions. Our results reveal that although $z$-variation constitutes a higher-order effect and does not substantially affect the emittance growth rate within the specific design parameters of the Electron-Ion Collider (EIC), the refined model offers improved accuracy, particularly in scenarios involving the interaction between beam-beam effects and other random diffusion processes, as well as in simulations incorporating realistic lattice models.

physics.acc-ph↗

arXiv2024

When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications

The recent surge in Large Language Models (LLMs) has garnered significant attention across numerous fields. Fine-tuning is often required to fit general LLMs for a specific domain, like the web-based healthcare system. However, two problems arise during fine-tuning LLMs for medical applications. One is the task variety problem, which involves distinct tasks in real-world medical scenarios. The variety often leads to sub-optimal fine-tuning for data imbalance and seesaw problems. Besides, the large amount of parameters in LLMs leads to huge time and computation consumption by fine-tuning. To address these two problems, we propose a novel parameter efficient fine-tuning framework for multi-task medical applications, dubbed as MOELoRA. The designed framework aims to absorb both the benefits of mixture-of-expert (MOE) for multi-task learning and low-rank adaptation (LoRA) for parameter efficient fine-tuning. For unifying MOE and LoRA, we devise multiple experts as the trainable parameters, where each expert consists of a pair of low-rank matrices to retain the small size of trainable parameters. Then, a task-motivated gate function for all MOELoRA layers is proposed, which can control the contributions of each expert and produce distinct parameters for various tasks. We conduct experiments on a multi-task medical dataset, indicating MOELoRA outperforms the existing parameter efficient fine-tuning methods. The code is available online.

cs.CL↗

arXiv2024

Assessing global crabbing scheme feasibility for Electron-Ion Collider

The Electron-Ion Collider (EIC) plans to utilize the local crabbing crossing scheme. This paper explores the feasibility of adopting a single crab cavity with adjusted voltage, inspired by the successful global crabbing scheme in KEKB, to restore effective head-on collisions. Using weak-strong simulations, the study assesses the potential of this global crabbing scheme for the EIC while emphasizing the need for adiabatic cavity ramping to prevent luminosity loss. Additionally, the research outlines potential risks associated with beam dynamics in implementing this scheme.

physics.acc-ph↗

arXiv2024

Advancing Electron Injection Dynamics and Mitigation Approaches in the Electron-Ion Collider Swap-out Injection Scheme

The Electron-Ion Collider (EIC) will use swap-out injection scheme for the Electron Storage Ring (ESR) to overcome limitations in polarization lifetime. However, the pursuit of highest luminosity with the required 28 nC electron bunches encounters stability challenges in the Rapid Cycling Synchrotron (RCS). One method is to inject multiple RCS bunches into a same ESR bucket. In this paper we perform simulation studies investigating proton emittance growth and electron emittance blowup in this injection scheme. Mitigation strategies are explored. These findings promise enhanced EIC stability and performance, shaping potential future operational improvements.

physics.acc-ph↗

arXiv2024

Large Language Model based Long-tail Query Rewriting in Taobao Search

In the realm of e-commerce search, the significance of semantic matching cannot be overstated, as it directly impacts both user experience and company revenue. Along this line, query rewriting, serving as an important technique to bridge the semantic gaps inherent in the semantic matching process, has attached wide attention from the industry and academia. However, existing query rewriting methods often struggle to effectively optimize long-tail queries and alleviate the phenomenon of "few-recall" caused by semantic gap. In this paper, we present BEQUE, a comprehensive framework that Bridges the sEmantic gap for long-tail QUEries. In detail, BEQUE comprises three stages: multi-instruction supervised fine tuning (SFT), offline feedback, and objective alignment. We first construct a rewriting dataset based on rejection sampling and auxiliary tasks mixing to fine-tune our large language model (LLM) in a supervised fashion. Subsequently, with the well-trained LLM, we employ beam search to generate multiple candidate rewrites, and feed them into Taobao offline system to obtain the partial order. Leveraging the partial order of rewrites, we introduce a contrastive learning method to highlight the distinctions between rewrites, and align the model with the Taobao online objectives. Offline experiments prove the effectiveness of our method in bridging semantic gap. Online A/B tests reveal that our method can significantly boost gross merchandise volume (GMV), number of transaction (#Trans) and unique visitor (UV) for long-tail queries. BEQUE has been deployed on Taobao, one of most popular online shopping platforms in China, since October 2023.

cs.IR↗

arXiv2024

Multi-perspective Improvement of Knowledge Graph Completion with Large Language Models

Knowledge graph completion (KGC) is a widely used method to tackle incompleteness in knowledge graphs (KGs) by making predictions for missing links. Description-based KGC leverages pre-trained language models to learn entity and relation representations with their names or descriptions, which shows promising results. However, the performance of description-based KGC is still limited by the quality of text and the incomplete structure, as it lacks sufficient entity descriptions and relies solely on relation names, leading to sub-optimal results. To address this issue, we propose MPIKGC, a general framework to compensate for the deficiency of contextualized knowledge and improve KGC by querying large language models (LLMs) from various perspectives, which involves leveraging the reasoning, explanation, and summarization capabilities of LLMs to expand entity descriptions, understand relations, and extract structures, respectively. We conducted extensive evaluation of the effectiveness and improvement of our framework based on four description-based KGC models and four datasets, for both link prediction and triplet classification tasks.

cs.CL↗

arXiv2023

Social Context-aware GCN for Video Character Search via Scene-prior Enhancement

With the increasing demand for intelligent services of online video platforms, video character search task has attracted wide attention to support downstream applications like fine-grained retrieval and summarization. However, traditional solutions only focus on visual or coarse-grained social information and thus cannot perform well when facing complex scenes, such as changing camera view or character posture. Along this line, we leverage social information and scene context as prior knowledge to solve the problem of character search in complex scenes. Specifically, we propose a scene-prior-enhanced framework, named SoCoSearch. We first integrate multimodal clues for scene context to estimate the prior probability of social relationships, and then capture characters' co-occurrence to generate an enhanced social context graph. Afterwards, we design a social context-aware GCN framework to achieve feature passing between characters to obtain robust representation for the character search task. Extensive experiments have validated the effectiveness of SoCoSearch in various metrics.

cs.MM↗

arXiv2023

Nonlinear optics from hybrid dispersive orbits

In this paper we present an expansion of the technique of characterizing nonlinear optics from off-energy orbits (NOECO) to cover harmonic sextupoles in storage rings. The existing NOECO technique has been successfully used to correct the chromatic sextupole errors on the MAX-IV machine, however, it doesn't account for harmonic sextupoles, which are widely used on many other machines. Through generating vertical dispersion with chromatic skew quadrupoles, a measurable dependency of nonlinear optics on harmonic sextupoles can be observed from hybrid horizontal and vertical dispersive orbits. Proof of concept of our expanded technique was accomplished by simulations and beam measurements on the National Synchrotron Light Source II (NSLS-II) storage ring.

physics.acc-ph↗

arXiv2022

Synchro-betatron Resonance of Crab Crossing Scheme with Large Crossing Angle and Finite Bunch Length

Crab crossing scheme is an essential collision scheme to achieve high luminosity for the future colliders with large crossing angles. However, when bunch length of one or both colliding beams is comparable with the wavelength of the crab cavity voltage, the nonlinear dependence of the crabbing kick may present a challenge to the beam dynamics of the colliding beams and impact the beam quality as well as the luminosity lifetime. In this paper, the results of nonlinear dynamics in the crab crossing scheme are presented, using both analytical and numerical studies. The result indicates that higher-order synchro-betatron resonances may be excited in the crab crossing scheme with large crossing angle, which causes the beam quality deterioration and luminosity degradation. The studies also reveal possible countermeasures to suppress the synchro-beta resonance, hence mitigate the degradation of beam quality and luminosity.

physics.acc-ph↗

arXiv2022

Combined effects of Crab Dispersion and Momentum Dispersion in Colliders with Local Crab Crossing Scheme

In this paper, we present the effects of linear transverse-longitudinal coupling on beam size at Interaction Point (IP) of a collider with local crab crossing scheme, when time dependent transverse deflection (crab kicks) and dispersive orbit intertwine near IP. The analytic propagation formula and the closed orbit form of the crab dispersion and momentum dispersion are derived. The non-zero momentum dispersion at crab cavities and the non-ideal phase from crab cavities to IP are detailed with the derived propagation formula to predict the beam size distortion at IP with or without the beam-beam interaction. The linear results are compared with nonlinear simulation using the weak-strong beam-beam code.

physics.acc-ph↗