arXiv subjects

Yu Wang

Publications and source records attributed to Yu Wang.

At least 1,099 records · Page 61Linked to original sources

arXiv2021

Learning Dynamic Preference Structure Embedding From Temporal Networks

The dynamics of temporal networks lie in the continuous interactions between nodes, which exhibit the dynamic node preferences with time elapsing. The challenges of mining temporal networks are thus two-fold: the dynamic structure of networks and the dynamic node preferences. In this paper, we investigate the dynamic graph sampling problem, aiming to capture the preference structure of nodes dynamically in cooperation with GNNs. Our proposed Dynamic Preference Structure (DPS) framework consists of two stages: structure sampling and graph fusion. In the first stage, two parameterized samplers are designed to learn the preference structure adaptively with network reconstruction tasks. In the second stage, an additional attention layer is designed to fuse two sampled temporal subgraphs of a node, generating temporal node embeddings for downstream tasks. Experimental results on many real-life temporal networks show that our DPS outperforms several state-of-the-art methods substantially owing to learning an adaptive preference structure. The code will be released soon at https://github.com/doujiang-zheng/Dynamic-Preference-Structure.

cs.IR↗

arXiv2021

Brilliant circularly polarized $γ$-ray sources via single-shot laser plasma interaction

Circularly polarized (CP) $γ$-ray sources are versatile for broad applications in nuclear physics, high-energy physics and astrophysics. The laser-plasma based particle accelerators provide accessibility for much higher flux $γ$-ray sources than conventional approaches, in which, however, the circular polarization properties of emitted $γ$-photons are used to be neglected. In this letter, we show that brilliant CP $γ$-ray beams can be generated via the combination of laser plasma wakefield acceleration and plasma mirror techniques. In weakly nonlinear Compton scattering scheme with moderate laser intensities, the helicity of the driving laser can be transferred to the emitted $γ$-photons, and their average polarization degree can reach about $\sim 37\%$ ($21\%$) with a peak brilliance of $\gtrsim 10^{21}~$photons/(s $\cdot$ mm$^2 \cdot$ mrad$^2 \cdot$ 0.1% BW) around 1~MeV (100~MeV). Moreover, our proposed method is easily feasible and robust with respect to the laser and plasma parameters.

physics.plasm-ph↗

arXiv2021

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

Multi-agent formation as well as obstacle avoidance is one of the most actively studied topics in the field of multi-agent systems. Although some classic controllers like model predictive control (MPC) and fuzzy control achieve a certain measure of success, most of them require precise global information which is not accessible in harsh environments. On the other hand, some reinforcement learning (RL) based approaches adopt the leader-follower structure to organize different agents' behaviors, which sacrifices the collaboration between agents thus suffering from bottlenecks in maneuverability and robustness. In this paper, we propose a distributed formation and obstacle avoidance method based on multi-agent reinforcement learning (MARL). Agents in our system only utilize local and relative information to make decisions and control themselves distributively. Agent in the multi-agent system will reorganize themselves into a new topology quickly in case that any of them is disconnected. Our method achieves better performance regarding formation error, formation convergence rate and on-par success rate of obstacle avoidance compared with baselines (both classic control methods and another RL-based method). The feasibility of our method is verified by both simulation and hardware implementation with Ackermann-steering vehicles.

eess.SY↗

arXiv2021

SpineOne: A One-Stage Detection Framework for Degenerative Discs and Vertebrae

Spinal degeneration plagues many elders, office workers, and even the younger generations. Effective pharmic or surgical interventions can help relieve degenerative spine conditions. However, the traditional diagnosis procedure is often too laborious. Clinical experts need to detect discs and vertebrae from spinal magnetic resonance imaging (MRI) or computed tomography (CT) images as a preliminary step to perform pathological diagnosis or preoperative evaluation. Machine learning systems have been developed to aid this procedure generally following a two-stage methodology: first perform anatomical localization, then pathological classification. Towards more efficient and accurate diagnosis, we propose a one-stage detection framework termed SpineOne to simultaneously localize and classify degenerative discs and vertebrae from MRI slices. SpineOne is built upon the following three key techniques: 1) a new design of the keypoint heatmap to facilitate simultaneous keypoint localization and classification; 2) the use of attention modules to better differentiate the representations between discs and vertebrae; and 3) a novel gradient-guided objective association mechanism to associate multiple learning objectives at the later training stage. Empirical results on the Spinal Disease Intelligent Diagnosis Tianchi Competition (SDID-TC) dataset of 550 exams demonstrate that our approach surpasses existing methods by a large margin.

cs.CV↗

arXiv2021

Dynamics of charged particles moving around Kerr black hole with inductive charge and external magnetic field

We mainly focus on the effects of small changes of parameters on the dynamics of charged particles around the Kerr black hole surrounded by an external magnetic field, which can be considered as a tidal environment. The radial motions of charged particles on the equatorial plane are studied via an effective potential. It is found that the particle energies at the local maxima values of the effective potentials increase with an increase of the black hole spin and the particle angular momenta, but decrease with an increase of one of the inductive charge parameter and magnetic field parameter. The radii of stable circular orbits on the equatorial plane also increase, whereas those of the innermost stable circular orbits decrease. On the other hand, the effects of small variations of the parameters on the orbital regular and chaotic dynamics of charged particles on the non-equatorial plane are traced by means of a time-transformed explicit symplectic integrator, Poincaré sections and fast Lyapunov indicators. It is shown that the dynamics sensitively depends on small variations of the inductive charge parameter, magnetic field parameter, energy and angular momentum. Chaos occurs easily as each of the dynamical parameters increases. When the dragging effects of the spacetime increase, the chaotic properties are not always weakened under some circumstances.

gr-qc↗

arXiv2021

Evaluating Efficient Performance Estimators of Neural Architectures

Conducting efficient performance estimations of neural architectures is a major challenge in neural architecture search (NAS). To reduce the architecture training costs in NAS, one-shot estimators (OSEs) amortize the architecture training costs by sharing the parameters of one "supernet" between all architectures. Recently, zero-shot estimators (ZSEs) that involve no training are proposed to further reduce the architecture evaluation cost. Despite the high efficiency of these estimators, the quality of such estimations has not been thoroughly studied. In this paper, we conduct an extensive and organized assessment of OSEs and ZSEs on five NAS benchmarks: NAS-Bench-101/201/301, and NDS ResNet/ResNeXt-A. Specifically, we employ a set of NAS-oriented criteria to study the behavior of OSEs and ZSEs and reveal that they have certain biases and variances. After analyzing how and why the OSE estimations are unsatisfying, we explore how to mitigate the correlation gap of OSEs from several perspectives. Through our analysis, we give out suggestions for future application and development of efficient architecture performance estimators. Furthermore, the analysis framework proposed in our work could be utilized in future research to give a more comprehensive understanding of newly designed architecture performance estimators. All codes are available at https://github.com/walkerning/aw_nas.

cs.CV↗

arXiv2021

Efficient Context-Aware Network for Abdominal Multi-organ Segmentation

The contextual information, presented in abdominal CT scan, is relative consistent. In order to make full use of the overall 3D context, we develop a whole-volume-based coarse-to-fine framework for efficient and effective abdominal multi-organ segmentation. We propose a new efficientSegNet network, which is composed of basic encoder, slim decoder and efficient context block. For the decoder module, anisotropic convolution with a k*k*1 intra-slice convolution and a 1*1*k inter-slice convolution, is designed to reduce the computation burden. For the context block, we propose strip pooling module to capture anisotropic and long-range contextual information, which exists in abdominal scene. Quantitative evaluation on the FLARE2021 validation cases, this method achieves the average dice similarity coefficient (DSC) of 0.895 and average normalized surface distance (NSD) of 0.775. This method won the 1st place on the 2021-MICCAI-FLARE challenge. Codes and models are available at https://github.com/Shanghai-Aitrox-Technology/EfficientSegmentation.

eess.IV↗

arXiv2021

Rapid Quantum Squeezing by Jumping the Harmonic Oscillator Frequency

Quantum sensing and quantum information processing use quantum advantages such as squeezed states that encode a quantity of interest with higher precision and generate quantum correlations to outperform classical methods. In harmonic oscillators, the rate of generating squeezing is set by a quantum speed limit. Therefore, the degree to which a quantum advantage can be used in practice is limited by the time needed to create the state relative to the rate of unavoidable decoherence. Alternatively, a sudden change of harmonic oscillator's frequency projects a ground state into a squeezed state which can circumvent the time constraint. Here, we create squeezed states of atomic motion by sudden changes of the harmonic oscillation frequency of atoms in an optical lattice. Building on this protocol, we demonstrate rapid quantum amplification of a displacement operator that could be used for detecting motion. Our results can speed up quantum gates and enable quantum sensing and quantum information processing in noisy environments.

quant-ph↗

arXiv2021

Meta-learning with an Adaptive Task Scheduler

To benefit the learning of a new task, meta-learning has been proposed to transfer a well-generalized meta-model learned from various meta-training tasks. Existing meta-learning algorithms randomly sample meta-training tasks with a uniform probability, under the assumption that tasks are of equal importance. However, it is likely that tasks are detrimental with noise or imbalanced given a limited number of meta-training tasks. To prevent the meta-model from being corrupted by such detrimental tasks or dominated by tasks in the majority, in this paper, we propose an adaptive task scheduler (ATS) for the meta-training process. In ATS, for the first time, we design a neural scheduler to decide which meta-training tasks to use next by predicting the probability being sampled for each candidate task, and train the scheduler to optimize the generalization capacity of the meta-model to unseen tasks. We identify two meta-model-related factors as the input of the neural scheduler, which characterize the difficulty of a candidate task to the meta-model. Theoretically, we show that a scheduler taking the two factors into account improves the meta-training loss and also the optimization landscape. Under the setting of meta-learning with noise and limited budgets, ATS improves the performance on both miniImageNet and a real-world drug discovery benchmark by up to 13% and 18%, respectively, compared to state-of-the-art task schedulers.

cs.LG↗

arXiv2021

Distance-wise Prototypical Graph Neural Network in Node Imbalance Classification

Recent years have witnessed the significant success of applying graph neural networks (GNNs) in learning effective node representations for classification. However, current GNNs are mostly built under the balanced data-splitting, which is inconsistent with many real-world networks where the number of training nodes can be extremely imbalanced among the classes. Thus, directly utilizing current GNNs on imbalanced data would generate coarse representations of nodes in minority classes and ultimately compromise the classification performance. This therefore portends the importance of developing effective GNNs for handling imbalanced graph data. In this work, we propose a novel Distance-wise Prototypical Graph Neural Network (DPGNN), which proposes a class prototype-driven training to balance the training loss between majority and minority classes and then leverages distance metric learning to differentiate the contributions of different dimensions of representations and fully encode the relative position of each node to each class prototype. Moreover, we design a new imbalanced label propagation mechanism to derive extra supervision from unlabeled nodes and employ self-supervised learning to smooth representations of adjacent nodes while separating inter-class prototypes. Comprehensive node classification experiments and parameter analysis on multiple networks are conducted and the proposed DPGNN almost always significantly outperforms all other baselines, which demonstrates its effectiveness in imbalanced node classification. The implementation of DPGNN is available at \url{https://github.com/YuWVandy/DPGNN}.

cs.LG↗

arXiv2021

Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective

Graph Neural Networks (GNNs) have been widely used in various domains, and GNNs with sophisticated computational graph lead to higher latency and larger memory consumption. Optimizing the GNN computational graph suffers from: (1) Redundant neural operator computation. The same data are propagated through the graph structure to perform the same neural operation multiple times in GNNs, leading to redundant computation which accounts for 92.4% of total operators. (2) Inconsistent thread mapping. Efficient thread mapping schemes for vertex-centric and edge-centric operators are different. This inconsistency prohibits operator fusion to reduce memory IO. (3) Excessive intermediate data. For GNN training which is usually performed concurrently with inference, intermediate data must be stored for the backward pass, consuming 91.9% of the total memory requirement. To tackle these challenges, we propose following designs to optimize the GNN computational graph from a novel coordinated computation, IO, and memory perspective: (1) Propagation-postponed operator reorganization. We reorganize operators to perform neural operations before the propagation, thus the redundant computation is eliminated. (2) Unified thread mapping for fusion. We propose a unified thread mapping scheme for both vertex- and edge-centric operators to enable fusion and reduce IO. (3) Intermediate data recomputation. Intermediate data are recomputed during the backward pass to reduce the total memory consumption. Extensive experimental results on three typical GNN models show that, we achieve up to 2.75x end-to-end speedup, 6.89x less memory IO, and 7.73x less memory consumption over state-of-the-art frameworks.

cs.LG↗

arXiv2021

Who calls the shots? Rethinking Few-Shot Learning for Audio

Few-shot learning aims to train models that can recognize novel classes given just a handful of labeled examples, known as the support set. While the field has seen notable advances in recent years, they have often focused on multi-class image classification. Audio, in contrast, is often multi-label due to overlapping sounds, resulting in unique properties such as polyphony and signal-to-noise ratios (SNR). This leads to unanswered questions concerning the impact such audio properties may have on few-shot learning system design, performance, and human-computer interaction, as it is typically up to the user to collect and provide inference-time support set examples. We address these questions through a series of experiments designed to elucidate the answers to these questions. We introduce two novel datasets, FSD-MIX-CLIPS and FSD-MIX-SED, whose programmatic generation allows us to explore these questions systematically. Our experiments lead to audio-specific insights on few-shot learning, some of which are at odds with recent findings in the image domain: there is no best one-size-fits-all model, method, and support set selection criterion. Rather, it depends on the expected application scenario. Our code and data are available at https://github.com/wangyu/rethink-audio-fsl.

cs.SD↗

arXiv2021

New Era of Deeplearning-Based Malware Intrusion Detection: The Malware Detection and Prediction Based On Deep Learning

With the development of artificial intelligence algorithms like deep learning models and the successful applications in many different fields, further similar trails of deep learning technology have been made in cyber security area. It shows the preferable performance not only in academic security research but also in industry practices when dealing with part of cyber security issues by deep learning methods compared to those conventional rules. Especially for the malware detection and classification tasks, it saves generous time cost and promotes the accuracy for a total pipeline of malware detection system. In this paper, we construct special deep neural network, ie, MalDeepNet (TB-Malnet and IB-Malnet) for malware dynamic behavior classification tasks. Then we build the family clustering algorithm based on deep learning and fulfil related testing. Except that, we also design a novel malware prediction model which could detect the malware coming in future through the Mal Generative Adversarial Network (Mal-GAN) implementation. All those algorithms present fairly considerable value in related datasets afterwards.

cs.CR↗

arXiv2021

Efficient Sparse Matrix Kernels based on Adaptive Workload-Balancing and Parallel-Reduction

Sparse matrix-vector and matrix-matrix multiplication (SpMV and SpMM) are fundamental in both conventional (graph analytics, scientific computing) and emerging (sparse DNN, GNN) domains. Workload-balancing and parallel-reduction are widely-used design principles for efficient SpMV. However, prior work fails to resolve how to implement and adaptively use the two principles for SpMV/MM. To overcome this obstacle, we first complete the implementation space with optimizations by filling three missing pieces in prior work, including: (1) We show that workload-balancing and parallel-reduction can be combined through a segment-reduction algorithm implemented with SIMD-shuffle primitives. (2) We show that parallel-reduction can be implemented in SpMM through loading the dense-matrix rows with vector memory operations. (3) We show that vectorized loading of sparse rows, being a part of the benefit of parallel-reduction, can co-exist with sequential-reduction in SpMM through temporally caching sparse-matrix elements in the shared memory. In terms of adaptive use, we analyze how the benefit of two principles change with two characteristics from the input data space: the diverse sparsity pattern and dense-matrix width. We find the benefit of the two principles fades along with the increased total workload, i.e. the increased dense-matrix width. We also identify, for SpMV and SpMM, different sparse-matrix features that impact workload-balancing effectiveness. Our design consistently exceeds cuSPARSE by 1.07-1.57x on different GPUs and dense matrix width, and the kernel selection rules involve 5-12% performance loss compared with optimal choices. Our kernel is being integrated into popular graph learning frameworks to accelerate GNN training.

cs.DC↗

arXiv2021

The Grasps Under Varied Object Orientation Dataset: Relation Between Grasps and Object Orientation

After a grasp has been planned, if the object orientation changes, the initial grasp may not have to be modified to accommodate the orientation change. For example, rotation of a cylinder by any amount around its centerline does not change its geometric shape relative to the grasper. Objects that can be approximated to solids of revolution or contain other geometric symmetries are prevalent in everyday life, and this information can be employed to improve the efficiency of existing grasp planning models. This paper experimentally investigates change in human-planned grasps under varied object orientations. With 13,440 recorded human grasps, our results indicate that during pick-and-place task of ordinary objects, stable grasps can be achieved with a small subset of grasp types, and the wrist-related parameters follow normal distribution.

cs.RO↗

arXiv2021

Synthesis and structure of carbon doped H3S compounds at high pressure

Understanding of recently reported putative close-to-room-temperature superconductivity in C-S-H compounds at 267 GPa demands reproducible synthesis protocol as well as knowledge of its structure and composition. We synthesized C-S-H compounds with various carbon composition at high pressures from elemental C and methane CH4, sulfur S, and molecular hydrogen H2. Here we focus on compounds synthesized using methane as these allow a straightforward determination of their structure and composition by combining single-crystal X-ray diffraction (XRD) and Raman spectroscopy. We applied a two-stage synthesis of ((CH4)x(H2S)(1-x))2H2 compounds by first reacting sulfur and mixed methane-hydrogen fluids and forming CH4 doped H2S crystals at 0.5-3 GPa, and then by growing single crystals of the desired hydrogen rich compound. Raman spectroscopy applied to this material shows the presence of the CH4 molecules incorporated into the lattice and allows to determine the CH4 content, while single-crystal X-ray diffraction results suggest that the methane molecules substitute H2S ones. The structural behavior of these compounds is very similar to the previously investigated methane free compounds demonstrating a transition from Al2Cu type I4/mcm structure to a modulated structure at 20-30 GPa and back to the same basic structure in an extended modification with greatly modified Raman spectra. This latter phase demonstrates a distortion into Pnma structure at 132-159 GPa and then transforms into a common Im-3m H3S phase at higher pressures, however, no structural anomaly is detected near 220 GPa, where a sharp upturn in Tc has been reported.

cond-mat.mtrl-sci↗

arXiv2021

Model-Free Learning of Safe yet Effective Controllers

We study the problem of learning safe control policies that are also effective; i.e., maximizing the probability of satisfying a linear temporal logic (LTL) specification of a task, and the discounted reward capturing the (classic) control performance. We consider unknown environments modeled as Markov decision processes. We propose a model-free reinforcement learning algorithm that learns a policy that first maximizes the probability of ensuring safety, then the probability of satisfying the given LTL specification and lastly, the sum of discounted Quality of Control rewards. Finally, we illustrate applicability of our RL-based approach.

cs.RO↗

arXiv2021

Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature

Information overload is a prevalent challenge in many high-value domains. A prominent case in point is the explosion of the biomedical literature on COVID-19, which swelled to hundreds of thousands of papers in a matter of months. In general, biomedical literature expands by two papers every minute, totalling over a million new papers every year. Search in the biomedical realm, and many other vertical domains is challenging due to the scarcity of direct supervision from click logs. Self-supervised learning has emerged as a promising direction to overcome the annotation bottleneck. We propose a general approach for vertical search based on domain-specific pretraining and present a case study for the biomedical domain. Despite being substantially simpler and not using any relevance labels for training or development, our method performs comparably or better than the best systems in the official TREC-COVID evaluation, a COVID-related biomedical search competition. Using distributed computing in modern cloud infrastructure, our system can scale to tens of millions of articles on PubMed and has been deployed as Microsoft Biomedical Search, a new search experience for biomedical literature: https://aka.ms/biomedsearch.

cs.IR↗