arXiv subjects

Yan Liu

Publications and source records attributed to Yan Liu.

At least 235 records · Page 13Linked to original sources

arXiv2024

Improving Long Text Understanding with Knowledge Distilled from Summarization Model

Long text understanding is important yet challenging for natural language processing. A long article or document usually contains many redundant words that are not pertinent to its gist and sometimes can be regarded as noise. With recent advances of abstractive summarization, we propose our \emph{Gist Detector} to leverage the gist detection ability of a summarization model and integrate the extracted gist into downstream models to enhance their long text understanding ability. Specifically, Gist Detector first learns the gist detection knowledge distilled from a summarization model, and then produces gist-aware representations to augment downstream models. We evaluate our method on three different tasks: long document classification, distantly supervised open-domain question answering, and non-parallel text style transfer. The experimental results show that our method can significantly improve the performance of baseline models on all tasks.

cs.CL↗

arXiv2024

CC2Vec: Combining Typed Tokens with Contrastive Learning for Effective Code Clone Detection

With the development of the open source community, the code is often copied, spread, and evolved in multiple software systems, which brings uncertainty and risk to the software system (e.g., bug propagation and copyright infringement). Therefore, it is important to conduct code clone detection to discover similar code pairs. Many approaches have been proposed to detect code clones where token-based tools can scale to big code. However, due to the lack of program details, they cannot handle more complicated code clones, i.e., semantic code clones. In this paper, we introduce CC2Vec, a novel code encoding method designed to swiftly identify simple code clones while also enhancing the capability for semantic code clone detection. To retain the program details between tokens, CC2Vec divides them into different categories (i.e., typed tokens) according to the syntactic types and then applies two self-attention mechanism layers to encode them. To resist changes in the code structure of semantic code clones, CC2Vec performs contrastive learning to reduce the differences introduced by different code implementations. We evaluate CC2Vec on two widely used datasets (i.e., BigCloneBench and Google Code Jam) and the results report that our method can effectively detect simple code clones. In addition, CC2Vec not only attains comparable performance to widely used semantic code clone detection systems such as ASTNN, SCDetector, and FCCA by simply fine-tuning, but also significantly surpasses these methods in both detection efficiency.

cs.SE↗

arXiv2024

MetaRM: Shifted Distributions Alignment via Meta-Learning

The success of Reinforcement Learning from Human Feedback (RLHF) in language model alignment is critically dependent on the capability of the reward model (RM). However, as the training process progresses, the output distribution of the policy model shifts, leading to the RM's reduced ability to distinguish between responses. This issue is further compounded when the RM, trained on a specific data distribution, struggles to generalize to examples outside of that distribution. These two issues can be united as a challenge posed by the shifted distribution of the environment. To surmount this challenge, we introduce MetaRM, a method leveraging meta-learning to align the RM with the shifted environment distribution. MetaRM is designed to train the RM by minimizing data loss, particularly for data that can improve the differentiation ability to examples of the shifted target distribution. Extensive experiments demonstrate that MetaRM significantly improves the RM's distinguishing ability in iterative RLHF optimization, and also provides the capacity to identify subtle differences in out-of-distribution samples.

cs.LG↗

arXiv2024

Large time behavior for the hyperbolic-parabolic coupled system with the regularity-loss structure

This paper considers the hyperbolic-parabolic coupled system, arising from the generalized thermoelastic coupled system, in the whole space $\mathbb{R}^n$. We study some qualitative properties for an energy term by diagonalization procedures, and for the solution by the WKB analysis. Particularly, we derive new large time asymptotic profiles with the regularity-loss structure (from the biharmonic parabolic equation and the diffusion wave equation with the Riesz potential operator) and optimal decay estimates with suitable higher regularities for the Cauchy data. Finally, we discover that the wave equation with the Riesz potential dissipation is a large time approximated model of our hyperbolic-parabolic coupled system.

math.AP↗

arXiv2024

Highly Squeezed States in Ring Resonators: Beyond the Undepleted Pump Approximation

We present a multimode theory of squeezed state generation in resonant systems valid for arbitrary pump power and including pump depletion. The Hamiltonian is written in terms of asymptotic-in and -out fields from scattering theory, capable of describing a general interaction. As an example we consider the lossy generation of a highly squeezed state by an effective second-order interaction in a silicon nitride ring resonator point-coupled to a waveguide. We calculate the photon number, Schmidt number, and the second-order correlation function of the generated state in the waveguide. The treatment we present provides a path forward to study the deterministic generation of non-Gaussian states in resonant systems.

quant-ph↗

arXiv2024

Microscopic Insights into Fatigue Mechanism in Wurtzite Ferroelectric Al$_{0.65}$Sc$_{0.35}$N: Oxygen Infiltration Enabled Grain Amorphization Spanning Boundary to Bulk

For the first time, the fatigue behavior involving external oxygen in highly Sc-doped AlN ferroelectric film was observed using transmission electron microscope techniques. Despite increasing the Sc composition in AlScN film contributes to reducing the device operation voltage, the inherent affinity of Sc for oxygen introduces instability in device performance. In this study, oxygen incorporation at top electrode edges and grain boundaries accompanied with an increase in current leakage and the disappearance of ferroelectric properties, was observed in nanoscale after long-term field cycling. This observation indicates the emergence of non-ferroelectric and even amorphous states. This presented work revealed solid experimental evidence of an oxygen-involved fatigue mechanism, providing valuable insights into the physical nature of the ferroelectric properties of AlScN films.

cond-mat.mtrl-sci↗

arXiv2024

Interpretable Detection of Out-of-Context Misinformation with Neural-Symbolic-Enhanced Large Multimodal Model

Recent years have witnessed the sustained evolution of misinformation that aims at manipulating public opinions. Unlike traditional rumors or fake news editors who mainly rely on generated and/or counterfeited images, text and videos, current misinformation creators now more tend to use out-of-context multimedia contents (e.g. mismatched images and captions) to deceive the public and fake news detection systems. This new type of misinformation increases the difficulty of not only detection but also clarification, because every individual modality is close enough to true information. To address this challenge, in this paper we explore how to achieve interpretable cross-modal de-contextualization detection that simultaneously identifies the mismatched pairs and the cross-modal contradictions, which is helpful for fact-check websites to document clarifications. The proposed model first symbolically disassembles the text-modality information to a set of fact queries based on the Abstract Meaning Representation of the caption and then forwards the query-image pairs into a pre-trained large vision-language model select the ``evidences" that are helpful for us to detect misinformation. Extensive experiments indicate that the proposed methodology can provide us with much more interpretable predictions while maintaining the accuracy same as the state-of-the-art model on this task.

cs.CL↗

arXiv2024

TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting

The past decade has witnessed significant advances in time series modeling with deep learning. While achieving state-of-the-art results, the best-performing architectures vary highly across applications and domains. Meanwhile, for natural language processing, the Generative Pre-trained Transformer (GPT) has demonstrated impressive performance via training one general-purpose model across various textual datasets. It is intriguing to explore whether GPT-type architectures can be effective for time series, capturing the intrinsic dynamic attributes and leading to significant accuracy improvements. In this paper, we propose a novel framework, TEMPO, that can effectively learn time series representations. We focus on utilizing two essential inductive biases of the time series task for pre-trained models: (i) decomposition of the complex interaction between trend, seasonal and residual components; and (ii) introducing the design of prompts to facilitate distribution adaptation in different types of time series. TEMPO expands the capability for dynamically modeling real-world temporal phenomena from data within diverse domains. Our experiments demonstrate the superior performance of TEMPO over state-of-the-art methods on zero shot setting for a number of time series benchmark datasets. This performance gain is observed not only in scenarios involving previously unseen datasets but also in scenarios with multi-modal inputs. This compelling finding highlights TEMPO's potential to constitute a foundational model-building framework.

cs.LG↗

arXiv2024

Chiral magnetic waves in strongly coupled Weyl semimetals

Propagating chiral magnetic waves (CMW) are expected to exist in chiral plasmas due to the interplay between the chiral magnetic and chiral separation effects induced by the presence of a chiral anomaly. Unfortunately, it was pointed out that, because of the effects of electric conductivity and dissipation, CMW are overdamped and therefore their signatures are unlikely to be seen in heavy-ion collision experiments and in the quark gluon plasma. Nonetheless, the chiral anomaly plays a fundamental role in Weyl semimetals and their anomalous transport properties as well. Hence, CMW could be potentially observed in topological semimetals using table-top experiments. By using a holographic model for strongly coupled Weyl semimetals, we investigate in detail the nature of CMW in presence of Coulomb interactions and axial charge relaxation and estimate whether, and in which regimes, CMW could be observed as underdamped collective excitations in topological materials.

hep-th↗

arXiv2024

How accurately can quantitative imaging methods be ranked without ground truth: An upper bound on no-gold-standard evaluation

Objective evaluation of quantitative imaging (QI) methods with patient data, while important, is typically hindered by the lack of gold standards. To address this challenge, no-gold-standard evaluation (NGSE) techniques have been proposed. These techniques have demonstrated efficacy in accurately ranking QI methods without access to gold standards. The development of NGSE methods has raised an important question: how accurately can QI methods be ranked without ground truth. To answer this question, we propose a Cramer-Rao bound (CRB)-based framework that quantifies the upper bound in ranking QI methods without any ground truth. We present the application of this framework in guiding the use of a well-known NGSE technique, namely the regression-without-truth (RWT) technique. Our results show the utility of this framework in quantifying the performance of this NGSE technique for different patient numbers. These results provide motivation towards studying other applications of this upper bound.

physics.med-ph↗

arXiv2024

XAIport: A Service Framework for the Early Adoption of XAI in AI Model Development

In this study, we propose the early adoption of Explainable AI (XAI) with a focus on three properties: Quality of explanation, the explanation summaries should be consistent across multiple XAI methods; Architectural Compatibility, for effective integration in XAI, the architecture styles of both the XAI methods and the models to be explained must be compatible with the framework; Configurable operations, XAI explanations are operable, akin to machine learning operations. Thus, an explanation for AI models should be reproducible and tractable to be trustworthy. We present XAIport, a framework of XAI microservices encapsulated into Open APIs to deliver early explanations as observation for learning model quality assurance. XAIport enables configurable XAI operations along with machine learning development. We quantify the operational costs of incorporating XAI with three cloud computer vision services on Microsoft Azure Cognitive Services, Google Cloud Vertex AI, and Amazon Rekognition. Our findings show comparable operational costs between XAI and traditional machine learning, with XAIport significantly improving both cloud AI model performance and explanation stability.

cs.AI↗

arXiv2024

LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin

Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks. Increasing instruction data substantially is a direct solution to align the model with a broader range of downstream tasks or notably improve its performance on a specific task. However, we find that large-scale increases in instruction data can damage the world knowledge previously stored in LLMs. To address this challenge, we propose LoRAMoE, a novelty framework that introduces several low-rank adapters (LoRA) and integrates them by using a router network, like a plugin version of Mixture of Experts (MoE). It freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge-edge forgetting. Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.

cs.CL↗

arXiv2024

Resonant Quantum Magnetodielectric Effect in Multiferroic Metal-Organic Framework [CH3NH3]Co(HCOO)3

We report the observation of both resonant quantum tunneling of magnetization (RQTM) and resonant quantum magnetodielectric (RQMD) effect in the perovskite multiferroic metal-organic framework [CH3NH3]Co(HCOO)3. An intrinsic magnetic phase separation emerges at low temperatures due to hydrogen-bond-modified long range super-exchange interaction, leading to the coexistence of canted antiferromagnetic order and single-ion magnet. Subsequently, a stair-shaped magnetic hysteresis loop along the [101] direction characterizing the RQTM appears below the magnetic blocking temperature. More interestingly, the magnetic field dependence of dielectric permittivity exhibits pronounced negative peaks at the critical fields corresponding to the RQTM, a phenomenon termed the RQMD effect which enables electrical detection of the RQTM. These intriguing properties make the multiferroic metal-organic framework a promising candidate for solid-state quantum computing.

cond-mat.mtrl-sci↗

arXiv2024

Validity of energy conditions of matter in traversable wormholes under the $f(Q)$ modified gravity theory

In the framework of the theory of general relativity, in order to obtain stable traversable wormholes, matter needs to violate the null energy condition. It is well known that the violation of the energy condition (EC) of matter leads to various physical problems. To address this issue, researchers have turned their attention to exploring modified theories of gravity, aiming to avoid the violation of ECs by introducing geometric terms. In this paper, within the framework of the $f(Q)$ modified gravitational theory, we investigate the effectiveness of ECs for matter in traversable wormholes. We examine the compliance of four types of energy conditions (weak energy condition, null energy condition, dominant energy condition, and strong energy condition) in the model by selecting a power-law model for $f(Q)$ and considering different shape functions $b(r)$. Our study reveals that for traversable wormholes realized through the $f(Q)$ modified gravity theory using the power-law model $f(Q)=a(-Q)^n$, all four types of ECs for matter can be satisfied. There is no need to introduce exotic matter (violating the null energy condition) or special matter (violating other energy conditions) artificially in the physics of wormholes.

gr-qc↗

arXiv2024

COSTAR: Improved Temporal Counterfactual Estimation with Self-Supervised Learning

Estimation of temporal counterfactual outcomes from observed history is crucial for decision-making in many domains such as healthcare and e-commerce, particularly when randomized controlled trials (RCTs) suffer from high cost or impracticality. For real-world datasets, modeling time-dependent confounders is challenging due to complex dynamics, long-range dependencies and both past treatments and covariates affecting the future outcomes. In this paper, we introduce Counterfactual Self-Supervised Transformer (COSTAR), a novel approach that integrates self-supervised learning for improved historical representations. We propose a component-wise contrastive loss tailored for temporal treatment outcome observations and explain its effectiveness from the view of unsupervised domain adaptation. COSTAR yields superior performance in estimation accuracy and generalization to out-of-distribution data compared to existing models, as validated by empirical results on both synthetic and real-world datasets.

cs.LG↗

arXiv2024

A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective

Vertical Federated Learning (VFL) is a federated learning paradigm where multiple participants, who share the same set of samples but hold different features, jointly train machine learning models. Although VFL enables collaborative machine learning without sharing raw data, it is still susceptible to various privacy threats. In this paper, we conduct the first comprehensive survey of the state-of-the-art in privacy attacks and defenses in VFL. We provide taxonomies for both attacks and defenses, based on their characterizations, and discuss open challenges and future research directions. Specifically, our discussion is structured around the model's life cycle, by delving into the privacy threats encountered during different stages of machine learning and their corresponding countermeasures. This survey not only serves as a resource for the research community but also offers clear guidance and actionable insights for practitioners to safeguard data privacy throughout the model's life cycle.

cs.CR↗

arXiv2024

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback

The advancement of large language models (LLMs) has significantly propelled the field of code generation. Previous work integrated reinforcement learning (RL) with compiler feedback for exploring the output space of LLMs to enhance code generation quality. However, the lengthy code generated by LLMs in response to complex human requirements makes RL exploration a challenge. Also, since the unit tests may not cover the complicated code, optimizing LLMs by using these unexecuted code snippets is ineffective. To tackle these challenges, we introduce StepCoder, a novel RL framework for code generation, consisting of two main components: CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks, while FGO only optimizes the model by masking the unexecuted code segments to provide Fine-Grained Optimization. In addition, we furthermore construct the APPS+ dataset for RL training, which is manually verified to ensure the correctness of unit tests. Experimental results show that our method improves the ability to explore the output space and outperforms state-of-the-art approaches in corresponding benchmarks. Our dataset APPS+ and StepCoder are available online.

cs.SE↗

arXiv2024

The Analytical Solutions of Equatorial Geodesic Motion in Kerr Spacetime

The study of Kerr geodesics has a long history, particularly for those occurring within the equatorial plane, which is generally well-understood. However, upon comparison with the classification introduced by one of us https://journals.aps.org/prd/abstract/10.1103/PhysRevD.105.024075 , it becomes apparent that certain classes of geodesics, such as trapped orbits, are still lacking analytical solutions. Thus, in this study, we provide explicit analytical solutions for equatorial timelike geodesics in Kerr spacetime, including solutions of trapped orbits, which capture the characteristics of special geodesics, such as the positions and conserved quantities of circular orbits, bound orbits, and deflecting orbits. Specifically, we determine the precise location at which retrograde orbits undergo a transition from counter-rotating to prograde motion due to the strong gravitational effects near the rotating black hole. Interestingly, we observe that for orbits with negative energy, the trajectory remains prograde despite the negative angular momentum. Furthermore, we investigate the intriguing phenomenon of deflecting orbits exhibiting an increased number of revolutions around the black hole as the turning point approaches the turning point of the trapped orbit. Additionally, we find that only prograde marginal deflecting geodesics are capable of traversing through the ergoregion. In summary, our findings present explicit solutions for equatorial timelike geodesics and offer insights into the dynamics of particle motion in the vicinity of a rotating black hole.

gr-qc↗