arXiv subjects

Yu Wang

Publications and source records attributed to Yu Wang.

At least 1,045 records · Page 58Linked to original sources

arXiv2022

Learning Monotone Dynamics by Neural Networks

Feed-forward neural networks (FNNs) work as standard building blocks in applying artificial intelligence (AI) to the physical world. They allow learning the dynamics of unknown physical systems (e.g., biological and chemical) {to predict their future behavior}. However, they are likely to violate the physical constraints of those systems without proper treatment. This work focuses on imposing two important physical constraints: monotonicity (i.e., a partial order of system states is preserved over time) and stability (i.e., the system states converge over time) when using FNNs to learn physical dynamics. For monotonicity constraints, we propose to use nonnegative neural networks and batch normalization. For both monotonicity and stability constraints, we propose to learn the system dynamics and corresponding Lyapunov function simultaneously. As demonstrated by case studies, our methods can preserve the stability and monotonicity of FNNs and significantly reduce their prediction errors.

cs.LG↗

arXiv2022

Improving Out-of-Distribution Robustness via Selective Augmentation

Machine learning algorithms typically assume that training and test examples are drawn from the same distribution. However, distribution shift is a common problem in real-world applications and can cause models to perform dramatically worse at test time. In this paper, we specifically consider the problems of subpopulation shifts (e.g., imbalanced data) and domain shifts. While prior works often seek to explicitly regularize internal representations or predictors of the model to be domain invariant, we instead aim to learn invariant predictors without restricting the model's internal representations or predictors. This leads to a simple mixup-based technique which learns invariant predictors via selective augmentation called LISA. LISA selectively interpolates samples either with the same labels but different domains or with the same domain but different labels. Empirically, we study the effectiveness of LISA on nine benchmarks ranging from subpopulation shifts to domain shifts, and we find that LISA consistently outperforms other state-of-the-art methods and leads to more invariant predictors. We further analyze a linear setting and theoretically show how LISA leads to a smaller worst-group error.

cs.LG↗

arXiv2022

Spatial-temporal Concept based Explanation of 3D ConvNets

Recent studies have achieved outstanding success in explaining 2D image recognition ConvNets. On the other hand, due to the computation cost and complexity of video data, the explanation of 3D video recognition ConvNets is relatively less studied. In this paper, we present a 3D ACE (Automatic Concept-based Explanation) framework for interpreting 3D ConvNets. In our approach: (1) videos are represented using high-level supervoxels, which is straightforward for human to understand; and (2) the interpreting framework estimates a score for each voxel, which reflects its importance in the decision procedure. Experiments show that our method can discover spatial-temporal concepts of different importance-levels, and thus can explore the influence of the concepts on a target task, such as action classification, in-depth. The codes are publicly available.

cs.CV↗

arXiv2022

Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakage

Graph Neural Networks (GNNs) have shown great power in learning node representations on graphs. However, they may inherit historical prejudices from training data, leading to discriminatory bias in predictions. Although some work has developed fair GNNs, most of them directly borrow fair representation learning techniques from non-graph domains without considering the potential problem of sensitive attribute leakage caused by feature propagation in GNNs. However, we empirically observe that feature propagation could vary the correlation of previously innocuous non-sensitive features to the sensitive ones. This can be viewed as a leakage of sensitive information which could further exacerbate discrimination in predictions. Thus, we design two feature masking strategies according to feature correlations to highlight the importance of considering feature propagation and correlation variation in alleviating discrimination. Motivated by our analysis, we propose Fair View Graph Neural Network (FairVGNN) to generate fair views of features by automatically identifying and masking sensitive-correlated features considering correlation variation after feature propagation. Given the learned fair views, we adaptively clamp weights of the encoder to avoid using sensitive-related features. Experiments on real-world datasets demonstrate that FairVGNN enjoys a better trade-off between model utility and fairness. Our code is publicly available at https://github.com/YuWVandy/FairVGNN.

cs.LG↗

arXiv2022

Chemical reactions in imperfect cavities: enhancement, suppression, and resonance

The use of optical cavities to control chemical reactions has been of great interest recently, following demonstrations of enhancement, suppression, and negligible effects on chemical reaction rates depending on the specific reaction and cavity frequency. In this work, we study the reaction rate inside imperfect cavities, where we introduce a broadening parameter in the spectral density to mimic Fabry-Pérot cavities. We investigate cavity modifications to reaction rates using non-Markovian Langevin dynamics with frictional and random forces to account for the presence of imperfect optical cavities. We demonstrate that in the regime of weak solvent and cavity friction, the cavity can enhance chemical reaction rates. On the other hand, in the high friction regime, cavities can suppress chemical reactions. Furthermore, we find that the broadening of the cavity spectral density gives rise to blue shifts of the resonance conditions and, surprisingly, increases the sharpness of the resonance effect.

physics.chem-ph↗

arXiv2022

Testing the consistency of proapgation between light and heavy cosmic ray nuclei

One of the fundamental issues in cosmic ray physics is to explain the nature of cosmic ray acceleration and propagation mechanisms. Thanks to the precise cosmic ray data measured by recent space experiments, we are able to investigate the cosmic ray acceleration and propagation models more comprehensively and reliably. In this paper, we combine the secondary-to-primary ratios and the primary spectra measured by PAMELA, AMS02, ACE-CRIS and Voyager-1 to constrain the cosmic ray source and transport parameters. The study shows that the $Z>2$ data yield a medium-energy diffusion slope $δ_{2}\sim\left(0.42, 0.48\right)$ and a high-energy slope $δ_{3}\sim\left(0.22, 0.34\right)$. The $Z\leq2$ species obtain a looser constraint on $δ_{2}\sim\left(0.38, 0.47\right)$, but a tighter constraint on $δ_{3}\sim\left(0.21, 0.30\right)$. The overlaps infer that the heavy and light particles can give compatible results at medium to high energies. Besides, both the light and heavy nuclei indicate a consistent diffusion slope variation $Δδ_{H}$ around $200\sim300$~GV. At low energies, significant disagreements exist between the heavy and light elements. The B/C ratio requires a much larger diffusion slope shift $Δδ_{L}$ around 4 GV or a stronger \textit{Alfv$\acute{e}$n velocity} $v_{A}$ than the low-mass data. This indicates that the heavy and light particles may suffer different low-energy transport behaviors in Galaxy. However, better understanding on the consistency/inconsistency between the heavy and light cosmic rays relies on more precise cross-sections, better constraints on correlations in systematic errors of data, more accurate estimation on Galaxy halo size and more robust description for Solar modulation during the reversal period of HMF.

astro-ph.HE↗

arXiv2022

ChemicalX: A Deep Learning Library for Drug Pair Scoring

In this paper, we introduce ChemicalX, a PyTorch-based deep learning library designed for providing a range of state of the art models to solve the drug pair scoring task. The primary objective of the library is to make deep drug pair scoring models accessible to machine learning researchers and practitioners in a streamlined framework.The design of ChemicalX reuses existing high level model training utilities, geometric deep learning, and deep chemistry layers from the PyTorch ecosystem. Our system provides neural network layers, custom pair scoring architectures, data loaders, and batch iterators for end users. We showcase these features with example code snippets and case studies to highlight the characteristics of ChemicalX. A range of experiments on real world drug-drug interaction, polypharmacy side effect, and combination synergy prediction tasks demonstrate that the models available in ChemicalX are effective at solving the pair scoring task. Finally, we show that ChemicalX could be used to train and score machine learning models on large drug pair datasets with hundreds of thousands of compounds on commodity hardware.

cs.LG↗

arXiv2022

Efficient Reinforcement Learning from Demonstration Using Local Ensemble and Reparameterization with Split and Merge of Expert Policies

The current work on reinforcement learning (RL) from demonstrations often assumes the demonstrations are samples from an optimal policy, an unrealistic assumption in practice. When demonstrations are generated by sub-optimal policies or have sparse state-action pairs, policy learned from sub-optimal demonstrations may mislead an agent with incorrect or non-local action decisions. We propose a new method called Local Ensemble and Reparameterization with Split and Merge of expert policies (LEARN-SAM) to improve efficiency and make better use of the sub-optimal demonstrations. First, LEARN-SAM employs a new concept, the lambda-function, based on a discrepancy measure between the current state to demonstrated states to "localize" the weights of the expert policies during learning. Second, LEARN-SAM employs a split-and-merge (SAM) mechanism by separating the helpful parts in each expert demonstration and regrouping them into new expert policies to use the demonstrations selectively. Both the lambda-function and SAM mechanism help boost the learning speed. Theoretically, we prove the invariant property of reparameterized policy before and after the SAM mechanism, providing theoretical guarantees for the convergence of the employed policy gradient method. We demonstrate the superiority of the LEARN-SAM method and its robustness with varying demonstration quality and sparsity in six experiments on complex continuous control problems of low to high dimensions, compared to existing methods on RL from demonstration.

cs.LG↗

arXiv2022

Brachial Plexus Nerve Trunk Segmentation Using Deep Learning: A Comparative Study with Doctors' Manual Segmentation

Ultrasound-guided nerve block anesthesia (UGNB) is a high-tech visual nerve block anesthesia method that can observe the target nerve and its surrounding structures, the puncture needle's advancement, and local anesthetics spread in real-time. The key in UGNB is nerve identification. With the help of deep learning methods, the automatic identification or segmentation of nerves can be realized, assisting doctors in completing nerve block anesthesia accurately and efficiently. Here, we establish a public dataset containing 320 ultrasound images of brachial plexus (BP). Three experienced doctors jointly produce the BP segmentation ground truth and label brachial plexus trunks. We design a brachial plexus segmentation system (BPSegSys) based on deep learning. BPSegSys achieves experienced-doctor-level nerve identification performance in various experiments. We evaluate BPSegSys' performance in terms of intersection-over-union (IoU), a commonly used performance measure for segmentation experiments. Considering three dataset groups in our established public dataset, the IoU of BPSegSys are 0.5238, 0.4715, and 0.5029, respectively, which exceed the IoU 0.5205, 0.4704, and 0.4979 of experienced doctors. In addition, we show that BPSegSys can help doctors identify brachial plexus trunks more accurately, with IoU improvement up to 27%, which has significant clinical application value.

eess.IV↗

arXiv2022

Robotic Computing on FPGAs: Current Progress, Research Challenges, and Opportunities

Robotic computing has reached a tipping point, with a myriad of robots (e.g., drones, self-driving cars, logistic robots) being widely applied in diverse scenarios. The continuous proliferation of robotics, however, critically depends on efficient computing substrates, driven by real-time requirements, robotic size-weight-and-power constraints, cybersecurity considerations, and dynamically changing scenarios. Within all platforms, FPGA is able to deliver both software and hardware solutions with low power, high performance, reconfigurability, reliability, and adaptivity characteristics, serving as the promising computing substrate for robotic applications. This paper highlights the current progress, design techniques, challenges, and open research challenges in the domain of robotic computing on FPGAs.

cs.RO↗

arXiv2022

Topological Interface-State Lasing in a Polymer-Cholesteric Liquid Crystal Superlattice

The advance of topological photonics has heralded a revolution for manipulating light as well as for the development of novel photonic devices such as topological insulator lasers. Here, we demonstrate topological lasing of circular polarization in a polymer-cholesteric liquid crystal (P-CLC) superlattice, tunable in the visible wavelength regime. By use of the femtosecond-laser direct-writing and self-assembling techniques, we establish the P-CLC superlattice with a controlled mini-band structure and a topological interface defect, thereby achieving a low threshold for robust topological lasing at about 0.4 uJ. Thanks to the chiral liquid crystal, not only the emission wavelength is thermally tuned, but the circularly polarized lasing is readily achieved. Our results bring about the possibility to realize compact and integrated topological photonic devices at low cost, as well as to engineer an ideal platform for exploring topological physics that involves light-matter interaction in soft-matter environments.

physics.optics↗

arXiv2022

Self-Supervised Masking for Unsupervised Anomaly Detection and Localization

Recently, anomaly detection and localization in multimedia data have received significant attention among the machine learning community. In real-world applications such as medical diagnosis and industrial defect detection, anomalies only present in a fraction of the images. To extend the reconstruction-based anomaly detection architecture to the localized anomalies, we propose a self-supervised learning approach through random masking and then restoring, named Self-Supervised Masking (SSM) for unsupervised anomaly detection and localization. SSM not only enhances the training of the inpainting network but also leads to great improvement in the efficiency of mask prediction at inference. Through random masking, each image is augmented into a diverse set of training triplets, thus enabling the autoencoder to learn to reconstruct with masks of various sizes and shapes during training. To improve the efficiency and effectiveness of anomaly detection and localization at inference, we propose a novel progressive mask refinement approach that progressively uncovers the normal regions and finally locates the anomalous regions. The proposed SSM method outperforms several state-of-the-arts for both anomaly detection and anomaly localization, achieving 98.3% AUC on Retinal-OCT and 93.9% AUC on MVTec AD, respectively.

cs.CV↗

arXiv2022

Low hardware consumption, resolution-configurable Gray code oscillator time-to-digital converters implemented in 16nm, 20nm and 28nm FPGAs

This paper presents a low hardware consumption, resolution-configurable, automatically calibrating Gray code oscillator time-to-digital converter (TDC) in Xilinx 16nm UltraScale+, 20nm UltraScale and 28nm Virtex-7 field-programmable gate arrays (FPGAs). The proposed TDC has several innovations: 1) a sampling matrix to improve the resolution. 2) a virtual bin calibration method (VBCM) to realize resolution configuration and automatic calibration. 3) a hardware implementation of the VBCM in standard FPGA devices. We implemented and evaluated a 16-channel TDC system in all three FPGAs. The UltraScale+ version achieved the best resolution (least significant bit, LSB) of 20.97 ps with 0.09 LSB averaged peak-peak differential linearity (DNLpk-pk). The UltraScale and Virtex-7 versions achieved the best resolutions of 36.01 ps with 0.10 LSB averaged DNLpk-pk and 34.84 ps with 0.08 LSB averaged DNLpk-pk, respectively.

cs.AR↗

arXiv2022

Quantum Information Scrambling in Quantum Many-body Scarred Systems

Quantum many-body scarred systems host special non-thermal eigenstates that support periodic revival dynamics and weakly break the ergodicity. Here, we study the quantum information scrambling dynamics in quantum many-body scarred systems, with a focus on the "PXP" model. We use the out-of-time-ordered correlator (OTOC) and Holevo information as measures of the information scrambling, and apply an efficient numerical method based on matrix product operators to compute them up to 41 spins. We find that both the OTOC and Holevo information exhibit a linear light cone and periodic oscillations inside the light cone for initial states within the scarred subspace, which is in sharp contrast to thermal or many-body localized systems. The periodic revivals of OTOCs and Holevo information signify unusual breakdown of quantum chaos and are not equivalent to the revival dynamics of state fidelity or local observables studied in the previous literature. To explain the formation of the linear light cone structure, we provide a perturbation-type calculation based on a phenomenological model. In addition, we demonstrate that the OTOC and Holevo information dynamics of the "PXP" model can be measured using the Rydberg-atom quantum simulators with current experimental technologies, and numerically identify the measurable signatures using experimental parameters.

quant-ph↗

arXiv2022

Production of polarized particle beams via ultraintense laser pulses

High-energy spin-polarized electron, positron, and $γ$-photon beams have many significant applications in the study of material properties, nuclear structure, particle physics, and high-energy astrophysics. Thus,efficient production of such polarized beams attracts a broad spectrum of research interests. This is driven mainly by the rapid advancements in ultrashort and ultraintense laser technology. Currently available laser pulses can achieve peak intensities in the range of $10^{22}-10^{23}$ Wcm$^{-2}$, with pulse durations of tens of femtoseconds. The dynamics of particles in laser fields of the available intensities is dominated by quantum electrodynamics (QED) and the interaction mechanisms have reached regimes spanned by nonlinear multiphoton absorbtion (strong-field QED processes). In strong-field QED processes, the scattering cross sections obviously depend on the spin and polarization of the particles, and the spin-dependent photon emission and the radiation-reaction effects can be utilized to produce the polarized particles. An ultraintense laser-driven polarized particle source possesses the advantages of high-brilliance and compactness, which could open the way for the unexplored aspects in a range of researches. In this work, we briefly review the seminal conclusions from the study of the polarization effects in strong-field QED processes, as well as the progress made by recent proposals for production of the polarized particles by laser-beam or laser-plasma interactions.

physics.plasm-ph↗

arXiv2022

Few-Shot Musical Source Separation

Deep learning-based approaches to musical source separation are often limited to the instrument classes that the models are trained on and do not generalize to separate unseen instruments. To address this, we propose a few-shot musical source separation paradigm. We condition a generic U-Net source separation model using few audio examples of the target instrument. We train a few-shot conditioning encoder jointly with the U-Net to encode the audio examples into a conditioning vector to configure the U-Net via feature-wise linear modulation (FiLM). We evaluate the trained models on real musical recordings in the MUSDB18 and MedleyDB datasets. We show that our proposed few-shot conditioning paradigm outperforms the baseline one-hot instrument-class conditioned model for both seen and unseen instruments. To extend the scope of our approach to a wider variety of real-world scenarios, we also experiment with different conditioning example characteristics, including examples from different recordings, with multiple sources, or negative conditioning examples.

cs.SD↗

arXiv2022

Variational Inference with NoFAS: Normalizing Flow with Adaptive Surrogate for Computationally Expensive Models

Fast inference of numerical model parameters from data is an important prerequisite to generate predictive models for a wide range of applications. Use of sampling-based approaches such as Markov chain Monte Carlo may become intractable when each likelihood evaluation is computationally expensive. New approaches combining variational inference with normalizing flow are characterized by a computational cost that grows only linearly with the dimensionality of the latent variable space, and rely on gradient-based optimization instead of sampling, providing a more efficient approach for Bayesian inference about the model parameters. Moreover, the cost of frequently evaluating an expensive likelihood can be mitigated by replacing the true model with an offline trained surrogate model, such as neural networks. However, this approach might generate significant bias when the surrogate is insufficiently accurate around the posterior modes. To reduce the computational cost without sacrificing inferential accuracy, we propose Normalizing Flow with Adaptive Surrogate (NoFAS), an optimization strategy that alternatively updates the normalizing flow parameters and surrogate model parameters. We also propose an efficient sample weighting scheme for surrogate model training that preserves global accuracy while effectively capturing high posterior density regions. We demonstrate the inferential and computational superiority of NoFAS against various benchmarks, including cases where the underlying model lacks identifiability. The source code and numerical experiments used for this study are available at https://github.com/cedricwangyu/NoFAS.

cs.LG↗

arXiv2022

GypSum: Learning Hybrid Representations for Code Summarization

Code summarization with deep learning has been widely studied in recent years. Current deep learning models for code summarization generally follow the principle in neural machine translation and adopt the encoder-decoder framework, where the encoder learns the semantic representations from source code and the decoder transforms the learnt representations into human-readable text that describes the functionality of code snippets. Despite they achieve the new state-of-the-art performance, we notice that current models often either generate less fluent summaries, or fail to capture the core functionality, since they usually focus on a single type of code representations. As such we propose GypSum, a new deep learning model that learns hybrid representations using graph attention neural networks and a pre-trained programming and natural language model. We introduce particular edges related to the control flow of a code snippet into the abstract syntax tree for graph construction, and design two encoders to learn from the graph and the token sequence of source code, respectively. We modify the encoder-decoder sublayer in the Transformer's decoder to fuse the representations and propose a dual-copy mechanism to facilitate summary generation. Experimental results demonstrate the superior performance of GypSum over existing code summarization models.

cs.SE↗