arXiv subjects

Xin Wang

Publications and source records attributed to Xin Wang.

At least 1,315 records · Page 73Linked to original sources

arXiv2022

Self-directed Machine Learning

Conventional machine learning (ML) relies heavily on manual design from machine learning experts to decide learning tasks, data, models, optimization algorithms, and evaluation metrics, which is labor-intensive, time-consuming, and cannot learn autonomously like humans. In education science, self-directed learning, where human learners select learning tasks and materials on their own without requiring hands-on guidance, has been shown to be more effective than passive teacher-guided learning. Inspired by the concept of self-directed human learning, we introduce the principal concept of Self-directed Machine Learning (SDML) and propose a framework for SDML. Specifically, we design SDML as a self-directed learning process guided by self-awareness, including internal awareness and external awareness. Our proposed SDML process benefits from self task selection, self data selection, self model selection, self optimization strategy selection and self evaluation metric selection through self-awareness without human guidance. Meanwhile, the learning performance of the SDML process serves as feedback to further improve self-awareness. We propose a mathematical formulation for SDML based on multi-level optimization. Furthermore, we present case studies together with potential applications of SDML, followed by discussing future research directions. We expect that SDML could enable machines to conduct human-like self-directed learning and provide a new perspective towards artificial general intelligence.

cs.LG↗

arXiv2022

Compression-Resistant Backdoor Attack against Deep Neural Networks

In recent years, many backdoor attacks based on training data poisoning have been proposed. However, in practice, those backdoor attacks are vulnerable to image compressions. When backdoor instances are compressed, the feature of specific backdoor trigger will be destroyed, which could result in the backdoor attack performance deteriorating. In this paper, we propose a compression-resistant backdoor attack based on feature consistency training. To the best of our knowledge, this is the first backdoor attack that is robust to image compressions. First, both backdoor images and their compressed versions are input into the deep neural network (DNN) for training. Then, the feature of each image is extracted by internal layers of the DNN. Next, the feature difference between backdoor images and their compressed versions are minimized. As a result, the DNN treats the feature of compressed images as the feature of backdoor images in feature space. After training, the backdoor attack against DNN is robust to image compression. Furthermore, we consider three different image compressions (i.e., JPEG, JPEG2000, WEBP) in feature consistency training, so that the backdoor attack is robust to multiple image compression algorithms. Experimental results demonstrate the effectiveness and robustness of the proposed backdoor attack. When the backdoor instances are compressed, the attack success rate of common backdoor attack is lower than 10%, while the attack success rate of our compression-resistant backdoor is greater than 97%. The compression-resistant attack is still robust even when the backdoor images are compressed with low compression quality. In addition, extensive experiments have demonstrated that, our compression-resistant backdoor attack has the generalization ability to resist image compression which is not used in the training process.

cs.CV↗

arXiv2022

Medical Matting: A New Perspective on Medical Segmentation with Uncertainty

It is difficult to accurately label ambiguous and complex shaped targets manually by binary masks. The weakness of binary mask under-expression is highlighted in medical image segmentation, where blurring is prevalent. In the case of multiple annotations, reaching a consensus for clinicians by binary masks is more challenging. Moreover, these uncertain areas are related to the lesions' structure and may contain anatomical information beneficial to diagnosis. However, current studies on uncertainty mainly focus on the uncertainty in model training and data labels. None of them investigate the influence of the ambiguous nature of the lesion itself.Inspired by image matting, this paper introduces alpha matte as a soft mask to represent uncertain areas in medical scenes and accordingly puts forward a new uncertainty quantification method to fill the gap of uncertainty research for lesion structure. In this work, we introduce a new architecture to generate binary masks and alpha mattes in a multitasking framework, which outperforms all state-of-the-art matting algorithms compared. The proposed uncertainty map is able to highlight the ambiguous regions and a novel multitasking loss weighting strategy we presented can improve performance further and demonstrate their concrete benefits. To fully-evaluate the effectiveness of our proposed method, we first labelled three medical datasets with alpha matte to address the shortage of available matting datasets in medical scenes and prove the alpha matte to be a more efficient labeling method than a binary mask from both qualitative and quantitative aspects.

cs.CV↗

arXiv2021

Self-healing and transformation characteristics of obstructed Hermite-Gaussian modes composed structured beams

The self-healing property of laser beams is of great interest. And a laser beam with spatial structures is also widely concerned due to its important applications in lots of areas. We theoretically and experimentally investigate the self-healing and transformation characteristics of obstructed structured beams composed by incoherent or coherent superposition of multiple Hermite-Gaussian (HG) modes. We reveal that partially obstructed single HG mode can recover itself or transfer to a lower order in the far-field. When the obstacle retains one pair of edged bright spots of HG mode in each direction of its two symmetry axes, the beam structure information (number of knot lines) along each axis can be restored. Otherwise, it will be transferred to the corresponding low-order HG mode or multi interference fringes in the far-field, according to the interval of the two most edged remaining spots on each axis. It's proved that the above effect is induced by the diffraction and interference results of the partially retained light field. This principle is also applicable to multi-eigenmodes composed beams with special customized optical structures. Then the self-healing and transformation characteristics of the partially obstructed HG modes composed structured beams are investigated. It's found that the HG modes incoherently composed structured beams have a strong ability to recover themselves in the far-field after occlusion. These investigations can expand the applications of optical lattice structures of laser communication, atom optical capture, and optical imaging.

physics.optics↗

arXiv2021

Provable Hierarchical Lifelong Learning with a Sketch-based Modular Architecture

We propose a modular architecture for the lifelong learning of hierarchically structured tasks. Specifically, we prove that our architecture is theoretically able to learn tasks that can be solved by functions that are learnable given access to functions for other, previously learned tasks as subroutines. We empirically show that some tasks that we can learn in this way are not learned by standard training methods in practice; indeed, prior work suggests that some such tasks cannot be learned by any efficient method without the aid of the simpler tasks. We also consider methods for identifying the tasks automatically, without relying on explicitly given indicators.

cs.LG↗

arXiv2021

A Scalable Deep Reinforcement Learning Model for Online Scheduling Coflows of Multi-Stage Jobs for High Performance Computing

Coflow is a recently proposed networking abstraction to help improve the communication performance of data-parallel computing jobs. In multi-stage jobs, each job consists of multiple coflows and is represented by a Directed Acyclic Graph (DAG). Efficiently scheduling coflows is critical to improve the data-parallel computing performance in data centers. Compared with hand-tuned scheduling heuristics, existing work DeepWeave [1] utilizes Reinforcement Learning (RL) framework to generate highly-efficient coflow scheduling policies automatically. It employs a graph neural network (GNN) to encode the job information in a set of embedding vectors, and feeds a flat embedding vector containing the whole job information to the policy network. However, this method has poor scalability as it is unable to cope with jobs represented by DAGs of arbitrary sizes and shapes, which requires a large policy network for processing a high-dimensional embedding vector that is difficult to train. In this paper, we first utilize a directed acyclic graph neural network (DAGNN) to process the input and propose a novel Pipelined-DAGNN, which can effectively speed up the feature extraction process of the DAGNN. Next, we feed the embedding sequence composed of schedulable coflows instead of a flat embedding of all coflows to the policy network, and output a priority sequence, which makes the size of the policy network depend on only the dimension of features instead of the product of dimension and number of nodes in the job's DAG.Furthermore, to improve the accuracy of the priority scheduling policy, we incorporate the Self-Attention Mechanism into a deep RL model to capture the interaction between different parts of the embedding sequence to make the output priority scores relevant. Based on this model, we then develop a coflow scheduling algorithm for online multi-stage jobs.

cs.DC↗

arXiv2021

Automated Machine Learning on Graphs: A Survey

Machine learning on graphs has been extensively studied in both academic and industry. However, as the literature on graph learning booms with a vast number of emerging methods and techniques, it becomes increasingly difficult to manually design the optimal machine learning algorithm for different graph-related tasks. To solve this critical challenge, automated machine learning (AutoML) on graphs which combines the strength of graph machine learning and AutoML together, is gaining attention from the research community. Therefore, we comprehensively survey AutoML on graphs in this paper, primarily focusing on hyper-parameter optimization (HPO) and neural architecture search (NAS) for graph machine learning. We further overview libraries related to automated graph machine learning and in-depth discuss AutoGL, the first dedicated open-source library for AutoML on graphs. In the end, we share our insights on future research directions for automated graph machine learning. This paper is the first systematic and comprehensive review of automated machine learning on graphs to the best of our knowledge.

cs.LG↗

arXiv2021

OOD-GNN: Out-of-Distribution Generalized Graph Neural Network

Graph neural networks (GNNs) have achieved impressive performance when testing and training graph data come from identical distribution. However, existing GNNs lack out-of-distribution generalization abilities so that their performance substantially degrades when there exist distribution shifts between testing and training graph data. To solve this problem, in this work, we propose an out-of-distribution generalized graph neural network (OOD-GNN) for achieving satisfactory performance on unseen testing graphs that have different distributions with training graphs. Our proposed OOD-GNN employs a novel nonlinear graph representation decorrelation method utilizing random Fourier features, which encourages the model to eliminate the statistical dependence between relevant and irrelevant graph representations through iteratively optimizing the sample graph weights and graph encoder. We further present a global weight estimator to learn weights for training graphs such that variables in graph representations are forced to be independent. The learned weights help the graph encoder to get rid of spurious correlations and, in turn, concentrate more on the true connection between learned discriminative graph representations and their ground-truth labels. We conduct extensive experiments to validate the out-of-distribution generalization abilities on two synthetic and 12 real-world datasets with distribution shifts. The results demonstrate that our proposed OOD-GNN significantly outperforms state-of-the-art baselines.

cs.LG↗

arXiv2021

Stochastic Actor-Executor-Critic for Image-to-Image Translation

Training a model-free deep reinforcement learning model to solve image-to-image translation is difficult since it involves high-dimensional continuous state and action spaces. In this paper, we draw inspiration from the recent success of the maximum entropy reinforcement learning framework designed for challenging continuous control problems to develop stochastic policies over high dimensional continuous spaces including image representation, generation, and control simultaneously. Central to this method is the Stochastic Actor-Executor-Critic (SAEC) which is an off-policy actor-critic model with an additional executor to generate realistic images. Specifically, the actor focuses on the high-level representation and control policy by a stochastic latent action, as well as explicitly directs the executor to generate low-level actions to manipulate the state. Experiments on several image-to-image translation tasks have demonstrated the effectiveness and robustness of the proposed SAEC when facing high-dimensional continuous space problems.

cs.CV↗

arXiv2021

Floquet Spin Amplification

Detection of weak electromagnetic waves and hypothetical particles aided by quantum amplification is important for fundamental physics and applications. However, demonstrations of quantum amplification are still limited; in particular, the physics of quantum amplification is not fully explored in periodically driven (Floquet) systems, which are generally defined by time-periodic Hamiltonians and enable observation of many exotic quantum phenomena such as time crystals. Here we investigate the magnetic-field signal amplification by periodically driven $^{129}$Xe spins and observe signal amplification at frequencies of transitions between Floquet spin states. This "Floquet amplification" allows to simultaneously enhance and measure multiple magnetic fields with at least one order of magnitude improvement, offering the capability of femtotesla-level measurements. Our findings extend the physics of quantum amplification to Floquet systems and can be generalized to a wide variety of existing amplifiers, enabling a previously unexplored class of "Floquet amplifiers".

quant-ph↗

arXiv2021

Analysis of the nonlinear dynamics of a chirping-frequency Alfvén mode in a Tokamak equilibrium

Chirping Alfvén modes are considered as potentially harmful in burning Tokamak plasmas. In this paper, the nonlinear evolution of a single-toroidal-number chirping mode is analysed by numerical particle simulation. This analysis can be simplified if the different resonant phase-space structures can be investigated as isolated ones. This can be done adopting a coordinate system that includes two constants of motion. In our simulations, we adopt as constants of motion, the magnetic momentum and the initial particle coordinates. For each resonant structure, a density-flattening region is formed around the respective resonance radius, with radial width that increases as the mode amplitude grows. It is delimited by two large negative density gradients, drifting inward and outward. With constant mode frequency, this density flattening would be responsible for the exhausting of the drive when large negative density gradients leave the resonance region. The frequency chirping, however, causes the resonance radius and the resonance region to drift inward. This drift delays the moment in which the inner density gradient reaches the inner boundary of the resonance region. On the other side, the island reconstitutes around the new resonance radius; as a consequence, the large negative density gradient further moves inward. This process continues as long as it allows to keep the large gradient within the resonance region. When this is no longer possible, the resonant structure ceases to be effective in driving the mode. To further grow, the mode has to tap a different resonant structure, possibly making use of additional frequency variations.

physics.plasm-ph↗

arXiv2021

A Survey of Deep Active Learning

Active learning (AL) attempts to maximize the performance gain of the model by marking the fewest samples. Deep learning (DL) is greedy for data and requires a large amount of data supply to optimize massive parameters, so that the model learns how to extract high-quality features. In recent years, due to the rapid development of internet technology, we are in an era of information torrents and we have massive amounts of data. In this way, DL has aroused strong interest of researchers and has been rapidly developed. Compared with DL, researchers have relatively low interest in AL. This is mainly because before the rise of DL, traditional machine learning requires relatively few labeled samples. Therefore, early AL is difficult to reflect the value it deserves. Although DL has made breakthroughs in various fields, most of this success is due to the publicity of the large number of existing annotation datasets. However, the acquisition of a large number of high-quality annotated datasets consumes a lot of manpower, which is not allowed in some fields that require high expertise, especially in the fields of speech recognition, information extraction, medical images, etc. Therefore, AL has gradually received due attention. A natural idea is whether AL can be used to reduce the cost of sample annotations, while retaining the powerful learning capabilities of DL. Therefore, deep active learning (DAL) has emerged. Although the related research has been quite abundant, it lacks a comprehensive survey of DAL. This article is to fill this gap, we provide a formal classification method for the existing work, and a comprehensive and systematic overview. In addition, we also analyzed and summarized the development of DAL from the perspective of application. Finally, we discussed the confusion and problems in DAL, and gave some possible development directions for DAL.

cs.LG↗

arXiv2021

Physical Implementability of Linear Maps and Its Application in Error Mitigation

Completely positive and trace-preserving maps characterize physically implementable quantum operations. On the other hand, general linear maps, such as positive but not completely positive maps, which can not be physically implemented, are fundamental ingredients in quantum information, both in theoretical and practical perspectives. This raises the question of how well one can simulate or approximate the action of a general linear map by physically implementable operations. In this work, we introduce a systematic framework to resolve this task using the quasiprobability decomposition technique. We decompose a target linear map into a linear combination of physically implementable operations and introduce the physical implementability measure as the least amount of negative portion that the quasiprobability must pertain, which directly quantifies the cost of simulating a given map using physically implementable quantum operations. We show this measure is efficiently computable by semidefinite programs and prove several properties of this measure, such as faithfulness, additivity, and unitary invariance. We derive lower and upper bounds in terms of the Choi operator's trace norm and obtain analytic expressions for several linear maps of practical interests. Furthermore, we endow this measure with an operational meaning within the quantum error mitigation scenario: it establishes the lower bound of the sampling cost achievable via the quasiprobability decomposition technique. In particular, for parallel quantum noises, we show that global error mitigation has no advantage over local error mitigation.

quant-ph↗

arXiv2021

Orthogonal Graph Neural Networks

Graph neural networks (GNNs) have received tremendous attention due to their superiority in learning node representations. These models rely on message passing and feature transformation functions to encode the structural and feature information from neighbors. However, stacking more convolutional layers significantly decreases the performance of GNNs. Most recent studies attribute this limitation to the over-smoothing issue, where node embeddings converge to indistinguishable vectors. Through a number of experimental observations, we argue that the main factor degrading the performance is the unstable forward normalization and backward gradient resulted from the improper design of the feature transformation, especially for shallow GNNs where the over-smoothing has not happened. Therefore, we propose a novel orthogonal feature transformation, named Ortho-GConv, which could generally augment the existing GNN backbones to stabilize the model training and improve the model's generalization performance. Specifically, we maintain the orthogonality of the feature transformation comprehensively from three perspectives, namely hybrid weight initialization, orthogonal transformation, and orthogonal regularization. By equipping the existing GNNs (e.g. GCN, JKNet, GCNII) with Ortho-GConv, we demonstrate the generality of the orthogonal feature transformation to enable stable training, and show its effectiveness for node and graph classification tasks.

cs.LG↗

arXiv2021

Quasi one-dimensional diffuse laser cooling of atoms

We demonstrate experimentally the generation of one-dimensional cold gases of $^{87}$Rb atoms by diffuse laser cooling (DLC). A horizontal slender vacuum glass tube with length of 105~cm and diameter of 2~cm is used in our experiment. The diffuse laser light inside the tube, which is generated by multi-reflection of injected lasers, cools the background vapor atoms. With 250~mW of cooling light and 50~mW of repumping light, an evenly distributed meter-long profile of atom cloud is obtained. We observe a factor 4 improvement on the atomic OD for a typical cooling duration of 170~ms and a sub-Doppler atomic temperature of 25~$μ$k. The maximum number of detected cold atoms remain constant for a free-fall duration of 30~ms. Such samples are ideal for many quantum optical experiments involving electromagnetically induced transparency, electronically highly excited (Rydberg) atoms and quantum precision measurements.

physics.atom-ph↗

arXiv2021

Experimental demonstration of adversarial examples in learning topological phases

Classification and identification of different phases and the transitions between them is a central task in condensed matter physics. Machine learning, which has achieved dramatic success in a wide range of applications, holds the promise to bring unprecedented perspectives for this challenging task. However, despite the exciting progress made along this direction, the reliability of machine-learning approaches likewise demands further investigation. Here, with the nitrogen-vacancy center platform, we report the first proof-of-principle experimental demonstration of adversarial examples in learning topological phases. We show that, after adding a tiny amount of carefully-designed perturbations, the experimentally observed adversarial examples can successfully deceive a splendid phase classifier, whose prediction accuracy is larger than $99.2\%$ on legitimate samples, with a notably high confidence. Our results explicitly showcase the crucial vulnerability aspect of applying machine learning techniques in classifying phases of matter, which provides an indispensable guide for future studies in this interdisciplinary field.

quant-ph↗

arXiv2021

HybridGazeNet: Geometric model guided Convolutional Neural Networks for gaze estimation

As a critical cue for understanding human intention, human gaze provides a key signal for Human-Computer Interaction(HCI) applications. Appearance-based gaze estimation, which directly regresses the gaze vector from eye images, has made great progress recently based on Convolutional Neural Networks(ConvNets) architecture and open-source large-scale gaze datasets. However, encoding model-based knowledge into CNN model to further improve the gaze estimation performance remains a topic that needs to be explored. In this paper, we propose HybridGazeNet(HGN), a unified framework that encodes the geometric eyeball model into the appearance-based CNN architecture explicitly. Composed of a multi-branch network and an uncertainty module, HybridGazeNet is trained using a hyridized strategy. Experiments on multiple challenging gaze datasets shows that HybridGazeNet has better accuracy and generalization ability compared with existing SOTA methods. The code will be released later.

cs.CV↗

arXiv2021

Active Learning Meets Optimized Item Selection

Designing recommendation systems with limited or no available training data remains a challenge. To that end, a new combinatorial optimization problem is formulated to generate optimized item selection for experimentation with the goal to shorten the time for collecting randomized training data. We first present an overview of the optimized item selection problem and a multi-level optimization framework to solve it. The approach integrates techniques from discrete optimization, unsupervised clustering, and latent text embeddings. We then discuss how to incorporate optimized item selection with active learning as part of randomized exploration in an ongoing fashion.

cs.IR↗