arXiv subjects

Yue Yu

Publications and source records attributed to Yue Yu.

At least 307 records · Page 17Linked to original sources

arXiv2023

Higher Order Gauge Equivariant CNNs on Riemannian Manifolds and Applications

With the advent of group equivariant convolutions in deep networks literature, spherical CNNs with $\mathsf{SO}(3)$-equivariant layers have been developed to cope with data that are samples of signals on the sphere $S^2$. One can implicitly obtain $\mathsf{SO}(3)$-equivariant convolutions on $S^2$ with significant efficiency gains by explicitly requiring gauge equivariance w.r.t. $\mathsf{SO}(2)$. In this paper, we build on this fact by introducing a higher order generalization of the gauge equivariant convolution, whose implementation is dubbed a gauge equivariant Volterra network (GEVNet). This allows us to model spatially extended nonlinear interactions within a given receptive field while still maintaining equivariance to global isometries. We prove theoretical results regarding the equivariance and construction of higher order gauge equivariant convolutions. Then, we empirically demonstrate the parameter efficiency of our model, first on computer vision benchmark data (e.g. spherical MNIST), and then in combination with a convolutional kernel network (CKN) on neuroimaging data. In the neuroimaging data experiments, the resulting two-part architecture (CKN + GEVNet) is used to automatically discriminate between patients with Lewy Body Disease (DLB), Alzheimer's Disease (AD) and Parkinson's Disease (PD) from diffusion magnetic resonance images (dMRI). The GEVNet extracts micro-architectural features within each voxel, while the CKN extracts macro-architectural features across voxels. This compound architecture is uniquely poised to exploit the intra- and inter-voxel information contained in the dMRI data, leading to improved performance over the classification results obtained from either of the individual components.

cs.CV↗

arXiv2023

Cost Design in Atomic Routing Games

An atomic routing game is a multiplayer game on a directed graph. Each player in the game chooses a path -- a sequence of links that connect its origin node to its destination node -- with the lowest cost, where the cost of each link is a function of all players' choices. We develop a novel numerical method to design the link cost function in atomic routing games such that the players' choices at the Nash equilibrium minimize a given smooth performance function. This method first approximates the nonsmooth Nash equilibrium conditions with smooth ones, then iteratively improves the link cost function via implicit differentiation. We demonstrate the application of this method to atomic routing games that model noncooperative agents navigating in grid worlds.

cs.GT↗

arXiv2023

ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval

With the development of large language models (LLMs), zero-shot learning has attracted much attention for various NLP tasks. Different from prior works that generate training data with billion-scale natural language generation (NLG) models, we propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. To realize this, we first conduct contrastive pretraining to learn an unsupervised dense retriever for extracting the most relevant documents using class-descriptive verbalizers. We then further propose two simple strategies, namely Verbalizer Augmentation with Demonstrations and Self-consistency Guided Filtering to improve the topic coverage of the dataset while removing noisy examples. Experiments on nine datasets demonstrate that REGEN achieves 4.3% gain over the strongest baselines and saves around 70% of the time compared to baselines using large NLG models. Besides, REGEN can be naturally integrated with recently proposed large language models to boost performance.

cs.CL↗

arXiv2023

Dynamic Routing in Stochastic Urban Air Mobility Networks: A Markov Decision Process Approach

Urban air mobility (UAM) is an emerging concept in short-range aviation transportation, where the aircraft will take off, land, and charge their batteries at a set of vertistops, and travel only through a set of flight corridors connecting these vertistops. We study the problem of routing an electric aircraft from its origin vertistop to its destination vertistop with the minimal expected total travel time. We first introduce a UAM network model that accounts for the limited battery capacity of aircraft, stochastic travel times of flight corridors, stochastic queueing delays, and a limited number of battery-charging stations at vertistops. Based on this model, we provide a sufficient condition for the existence of a routing strategy that avoids battery exhaustion. Furthermore, we show how to compute such a strategy by computing the optimal policy in a Markov decision process, a mathematical framework for decision-making in a stochastic dynamic environment. We illustrate our results using a case study with 29 vertistops and 137 flight corridors.

eess.SY↗

arXiv2023

Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approach

Large Language Models have demonstrated remarkable few-shot performance, but the performance can be sensitive to the selection of few-shot instances. We propose PATRON, a new method that uses prompt-based uncertainty estimation for data selection for pre-trained language model fine-tuning under cold-start scenarios, i.e., no initial labeled data are available. In PATRON, we design (1) a prompt-based uncertainty propagation approach to estimate the importance of data points and (2) a partition-then-rewrite (PTR) strategy to promote sample diversity when querying for annotations. Experiments on six text classification datasets show that PATRON outperforms the strongest cold-start data selection baselines by up to 6.9%. Besides, with 128 labels only, PATRON achieves 91.0% and 92.1% of the fully supervised performance based on vanilla fine-tuning and prompt-based learning respectively. Our implementation of PATRON is available at \url{https://github.com/yueyu1030/Patron}.

cs.CL↗

arXiv2023

Quantum Simulation for Partial Differential Equations with Physical Boundary or Interface Conditions

This paper explores the feasibility of quantum simulation for partial differential equations (PDEs) with physical boundary or interface conditions. Semi-discretisation of such problems does not necessarily yield Hamiltonian dynamics and even alters the Hamiltonian structure of the dynamics when boundary and interface conditions are included. This seemingly intractable issue can be resolved by using a recently introduced Schrödingerisation method (Jin et al. 2022) -- it converts any linear PDEs and ODEs with non-Hermitian dynamics to a system of Schrödinger equations, via the so-called warped phase transformation that maps the equation into one higher dimension. We implement this method for several typical problems, including the linear convection equation with inflow boundary conditions and the heat equation with Dirichlet and Neumann boundary conditions. For interface problems, we study the (parabolic) Stefan problem, linear convection, and linear Liouville equations with discontinuous and even measure-valued coefficients. We perform numerical experiments to demonstrate the validity of this approach, which helps to bridge the gap between available quantum algorithms and computational models for classical and quantum dynamics with boundary and interface conditions.

quant-ph↗

arXiv2023

Quantum Simulation for Quantum Dynamics with Artificial Boundary Conditions

Quantum dynamics, typically expressed in the form of a time-dependent Schrödinger equation with a Hermitian Hamiltonian, is a natural application for quantum computing. However, when simulating quantum dynamics that involves the emission of electrons, it is necessary to use artificial boundary conditions (ABC) to confine the computation within a fixed domain. The introduction of ABCs alters the Hamiltonian structure of the dynamics, and existing quantum algorithms can not be directly applied since the evolution is no longer unitary. The current paper utilizes a recently introduced Schrödingerisation method (Jin et al. arXiv:2212.13969 and arXiv:2212.14703) that converts non-Hermitian dynamics to a Schrödinger form, for the artificial boundary problems. We implement this method for three types of ABCs, including the complex absorbing potential technique, perfectly matched layer methods, and Dirichlet-to-Neumann approach. We analyze the query complexity of these algorithms, and perform numerical experiments to demonstrate the validity of this approach. This helps to bridge the gap between available quantum algorithms and computational models for quantum dynamics in unbounded domains.

quant-ph↗

arXiv2023

Theory of the low- and high-field superconducting phases of UTe$_2$

Recent nuclear magnetic resonance (NMR) and calorimetric experiments have observed that UTe$_2$ exhibits a transition between two distinct superconducting phases as a function of magnetic field strength for a field applied along the crystalline $b$-axis. To determine the nature of these phases, we employ a microscopic two-band minimal Hamiltonian with the essential crystal symmetries and structural details. We also adopt anisotropic ferromagnetic exchange terms. We study the resulting pairing symmetries and properties of these low- and high-field phases in mean field theory.

cond-mat.supr-con↗

arXiv2023

Transferability Estimation Based On Principal Gradient Expectation

Transfer learning aims to improve the performance of target tasks by transferring knowledge acquired in source tasks. The standard approach is pre-training followed by fine-tuning or linear probing. Especially, selecting a proper source domain for a specific target domain under predefined tasks is crucial for improving efficiency and effectiveness. It is conventional to solve this problem via estimating transferability. However, existing methods can not reach a trade-off between performance and cost. To comprehensively evaluate estimation methods, we summarize three properties: stability, reliability and efficiency. Building upon them, we propose Principal Gradient Expectation(PGE), a simple yet effective method for assessing transferability. Specifically, we calculate the gradient over each weight unit multiple times with a restart scheme, and then we compute the expectation of all gradients. Finally, the transferability between the source and target is estimated by computing the gap of normalized principal gradients. Extensive experiments show that the proposed metric is superior to state-of-the-art methods on all properties.

cs.CV↗

arXiv2023

Material Properties of Organic Liquids, Ices, and Hazes on Titan

Titan has a diverse range of materials in its atmosphere and on its surface: the simple organics that reside in various phases (gas, liquid, ice) and the solid complex refractory organics that form Titan's haze layers. These materials all actively participate in various physical processes on Titan, and many material properties are found to be important in shaping these processes. Future in-situ exploration on Titan would likely encounter a range of materials, and a comprehensive database to archive the material properties of all possible material candidates will be needed. Here we summarize several important material properties of the organic liquids, ices, and the refractory hazes on Titan that are available in the literature and/or that we have computed. These properties include thermodynamic properties (phase change points, sublimation and vaporization saturation vapor pressure, and latent heat), physical property (density), and surface properties (liquid surface tensions and solid surface energies). We have developed a new database to provide a repository for these data and make them available to the science community. These data can be used as inputs for various theoretical models to interpret current and future remote sensing and in-situ atmospheric and surface measurements on Titan. The material properties of the simple organics may also be applicable to giant planets and icy bodies in the outer solar system, interstellar medium, protoplanetary disks, and exoplanets.

astro-ph.EP↗

arXiv2023

The Solar Upper Transition Region Imager (SUTRI) onboard the SATech-01 satellite

The Solar Upper Transition Region Imager (SUTRI) onboard the Space Advanced Technology demonstration satellite (SATech-01), which was launched to a sun-synchronous orbit at a height of 500 km in July 2022, aims to test the on-orbit performance of our newly developed Sc-Si multi-layer reflecting mirror and the 2kx2k EUV CMOS imaging camera and to take full-disk solar images at the Ne VII 46.5 nm spectral line with a filter width of 3 nm. SUTRI employs a Ritchey-Chretien optical system with an aperture of 18 cm. The on-orbit observations show that SUTRI images have a field of view of 41.6'x41.6' and a moderate spatial resolution of 8" without an image stabilization system. The normal cadence of SUTRI images is 30 s and the solar observation time is about 16 hours each day because the earth eclipse time accounts for about 1/3 of SATech-01's orbit period. Approximately 15 GB data is acquired each day and made available online after processing. SUTRI images are valuable as the Ne VII 46.5 nm line is formed at a temperature regime of 0.5 MK in the solar atmosphere, which has rarely been sampled by existing solar imagers. SUTRI observations will establish connections between structures in the lower solar atmosphere and corona, and advance our understanding of various types of solar activity such as flares, filament eruptions, coronal jets and coronal mass ejections.

astro-ph.SR↗

arXiv2023

Neighborhood-Regularized Self-Training for Learning with Few Labels

Training deep neural networks (DNNs) with limited supervision has been a popular research topic as it can significantly alleviate the annotation burden. Self-training has been successfully applied in semi-supervised learning tasks, but one drawback of self-training is that it is vulnerable to the label noise from incorrect pseudo labels. Inspired by the fact that samples with similar labels tend to share similar representations, we develop a neighborhood-based sample selection approach to tackle the issue of noisy pseudo labels. We further stabilize self-training via aggregating the predictions from different rounds during sample selection. Experiments on eight tasks show that our proposed method outperforms the strongest self-training baseline with 1.83% and 2.51% performance gain for text and graph datasets on average. Our further analysis demonstrates that our proposed data selection strategy reduces the noise of pseudo labels by 36.8% and saves 57.3% of the time when compared with the best baseline. Our code and appendices will be uploaded to https://github.com/ritaranx/NeST.

cs.LG↗

arXiv2023

Two Sides of the Same Coin: Exploiting the Impact of Identifiers in Neural Code Comprehension

Previous studies have demonstrated that neural code comprehension models are vulnerable to identifier naming. By renaming as few as one identifier in the source code, the models would output completely irrelevant results, indicating that identifiers can be misleading for model prediction. However, identifiers are not completely detrimental to code comprehension, since the semantics of identifier names can be related to the program semantics. Well exploiting the two opposite impacts of identifiers is essential for enhancing the robustness and accuracy of neural code comprehension, and still remains under-explored. In this work, we propose to model the impact of identifiers from a novel causal perspective, and propose a counterfactual reasoning-based framework named CREAM. CREAM explicitly captures the misleading information of identifiers through multi-task learning in the training stage, and reduces the misleading impact by counterfactual inference in the inference stage. We evaluate CREAM on three popular neural code comprehension tasks, including function naming, defect detection and code classification. Experiment results show that CREAM not only significantly outperforms baselines in terms of robustness (e.g., +37.9% on the function naming task at F1 score), but also achieve improved results on the original datasets (e.g., +0.5% on the function naming task at F1 score).

cs.SE↗

arXiv2023

MetaNO: How to Transfer Your Knowledge on Learning Hidden Physics

Gradient-based meta-learning methods have primarily been applied to classical machine learning tasks such as image classification. Recently, PDE-solving deep learning methods, such as neural operators, are starting to make an important impact on learning and predicting the response of a complex physical system directly from observational data. Since the data acquisition in this context is commonly challenging and costly, the call of utilization and transfer of existing knowledge to new and unseen physical systems is even more acute. Herein, we propose a novel meta-learning approach for neural operators, which can be seen as transferring the knowledge of solution operators between governing (unknown) PDEs with varying parameter fields. Our approach is a provably universal solution operator for multiple PDE solving tasks, with a key theoretical observation that underlying parameter fields can be captured in the first layer of neural operator models, in contrast to typical final-layer transfer in existing meta-learning methods. As applications, we demonstrate the efficacy of our proposed approach on PDE-based datasets and a real-world material modeling problem, illustrating that our method can handle complex and nonlinear physical response learning tasks while greatly improving the sampling efficiency in unseen tasks.

cs.LG↗

arXiv2023

INO: Invariant Neural Operators for Learning Complex Physical Systems with Momentum Conservation

Neural operators, which emerge as implicit solution operators of hidden governing equations, have recently become popular tools for learning responses of complex real-world physical systems. Nevertheless, the majority of neural operator applications has thus far been data-driven, which neglects the intrinsic preservation of fundamental physical laws in data. In this paper, we introduce a novel integral neural operator architecture, to learn physical models with fundamental conservation laws automatically guaranteed. In particular, by replacing the frame-dependent position information with its invariant counterpart in the kernel space, the proposed neural operator is by design translation- and rotation-invariant, and consequently abides by the conservation laws of linear and angular momentums. As applications, we demonstrate the expressivity and efficacy of our model in learning complex material behaviors from both synthetic and experimental datasets, and show that, by automatically satisfying these essential physical laws, our learned neural operator is not only generalizable in handling translated and rotated datasets, but also achieves state-of-the-art accuracy and efficiency as compared to baseline neural operator models.

cs.LG↗

arXiv2023

Towards a unified nonlocal, peridynamics framework for the coarse-graining of molecular dynamics data with fractures

Molecular dynamics (MD) has served as a powerful tool for designing materials with reduced reliance on laboratory testing. However, the use of MD directly to treat the deformation and failure of materials at the mesoscale is still largely beyond reach. Herein, we propose a learning framework to extract a peridynamic model as a mesoscale continuum surrogate from MD simulated material fracture datasets. Firstly, we develop a novel coarse-graining method, to automatically handle the material fracture and its corresponding discontinuities in MD displacement dataset. Inspired by the Weighted Essentially Non-Oscillatory scheme, the key idea lies at an adaptive procedure to automatically choose the locally smoothest stencil, then reconstruct the coarse-grained material displacement field as piecewise smooth solutions containing discontinuities. Then, based on the coarse-grained MD data, a two-phase optimization-based learning approach is proposed to infer the optimal peridynamics model with damage criterion. In the first phase, we identify the optimal nonlocal kernel function from datasets without material damage, to capture the material stiffness properties. Then, in the second phase, the material damage criterion is learnt as a smoothed step function from the data with fractures. As a result, a peridynamics surrogate is obtained. Our peridynamics surrogate model can be employed in further prediction tasks with different grid resolutions from training, and hence allows for substantial reductions in computational cost compared with MD. We illustrate the efficacy of the proposed approach with several numerical tests for single layer graphene. Our tests show that the proposed data-driven model is robust and generalizable: it is capable in modeling the initialization and growth of fractures under discretization and loading settings that are different from the ones used during training.

cond-mat.mtrl-sci↗

arXiv2022

Quantum simulation of partial differential equations via Schrodingerisation: technical details

We study a new method - called Schrodingerisation introduced in [Jin, Liu, Yu, arXiv: 2212.13969] - for solving general linear partial differential equations with quantum simulation. This method converts linear partial differential equations into a `Schrodingerised' or Hamiltonian system, using a new and simple transformation called the warped phase transformation. Here we provide more in-depth technical discussions and expand on this approach in a more detailed and pedagogical way. We apply this to more examples of partial differential equations, including heat, convection, Fokker-Planck, linear Boltzmann and Black-Scholes equations. This approach can also be extended to Schrodingerise general linear partial differential equations, including the Vlasov-Fokker-Planck equation and the Liouville representation equation for nonlinear ordinary differential equations.

quant-ph↗

arXiv2022

Quantum simulation of partial differential equations via Schrodingerisation

We present a simple new way - called Schrodingerisation - to simulate general linear partial differential equations via quantum simulation. Using a simple new transform, referred to as the warped phase transformation, any linear partial differential equation can be recast into a system of Schrodinger's equations - in real time - in a straightforward way. This can be seen directly on the level of the dynamical equations without more sophisticated methods. This approach is not only applicable to PDEs for classical problems but also those for quantum problems - like the preparation of quantum ground states, Gibbs states and the simulation of quantum states in random media in the semiclassical limit.

quant-ph↗