arXiv subjects

Xiao Wang

Publications and source records attributed to Xiao Wang.

At least 667 records · Page 37Linked to original sources

arXiv2020

Block Switching: A Stochastic Approach for Deep Learning Security

Recent study of adversarial attacks has revealed the vulnerability of modern deep learning models. That is, subtly crafted perturbations of the input can make a trained network with high accuracy produce arbitrary incorrect predictions, while maintain imperceptible to human vision system. In this paper, we introduce Block Switching (BS), a defense strategy against adversarial attacks based on stochasticity. BS replaces a block of model layers with multiple parallel channels, and the active channel is randomly assigned in the run time hence unpredictable to the adversary. We show empirically that BS leads to a more dispersed input gradient distribution and superior defense effectiveness compared with other stochastic defenses such as stochastic activation pruning (SAP). Compared to other defenses, BS is also characterized by the following features: (i) BS causes less test accuracy drop; (ii) BS is attack-independent and (iii) BS is compatible with other defenses and can be used jointly with others.

cs.LG↗

arXiv2020

Stochastic Approximate Gradient Descent via the Langevin Algorithm

We introduce a novel and efficient algorithm called the stochastic approximate gradient descent (SAGD), as an alternative to the stochastic gradient descent for cases where unbiased stochastic gradients cannot be trivially obtained. Traditional methods for such problems rely on general-purpose sampling techniques such as Markov chain Monte Carlo, which typically requires manual intervention for tuning parameters and does not work efficiently in practice. Instead, SAGD makes use of the Langevin algorithm to construct stochastic gradients that are biased in finite steps but accurate asymptotically, enabling us to theoretically establish the convergence guarantee for SAGD. Inspired by our theoretical analysis, we also provide useful guidelines for its practical implementation. Finally, we show that SAGD performs well experimentally in popular statistical and machine learning problems such as the expectation-maximization algorithm and the variational autoencoders.

cs.LG↗

arXiv2020

Structural Deep Clustering Network

Clustering is a fundamental task in data analysis. Recently, deep clustering, which derives inspiration primarily from deep learning approaches, achieves state-of-the-art performance and has attracted considerable attention. Current deep clustering methods usually boost the clustering results by means of the powerful representation ability of deep learning, e.g., autoencoder, suggesting that learning an effective representation for clustering is a crucial requirement. The strength of deep clustering methods is to extract the useful representations from the data itself, rather than the structure of data, which receives scarce attention in representation learning. Motivated by the great success of Graph Convolutional Network (GCN) in encoding the graph structure, we propose a Structural Deep Clustering Network (SDCN) to integrate the structural information into deep clustering. Specifically, we design a delivery operator to transfer the representations learned by autoencoder to the corresponding GCN layer, and a dual self-supervised mechanism to unify these two different deep neural architectures and guide the update of the whole model. In this way, the multiple structures of data, from low-order to high-order, are naturally combined with the multiple representations learned by autoencoder. Furthermore, we theoretically analyze the delivery operator, i.e., with the delivery operator, GCN improves the autoencoder-specific representation as a high-order graph regularization constraint and autoencoder helps alleviate the over-smoothing problem in GCN. Through comprehensive experiments, we demonstrate that our propose model can consistently perform better over the state-of-the-art techniques.

cs.LG↗

arXiv2020

Excitons in solids from periodic equation-of-motion coupled-cluster theory

We present an ab initio study of electronically excited states of three-dimensional solids using Gaussian-based periodic equation-of-motion coupled-cluster theory with single and double excitations (EOM-CCSD). The explicit use of translational symmetry, as implemented via Brillouin zone sampling and momentum conservation, is responsible for a large reduction in cost. Our largest system studied, which samples the Brillouin zone using 64 k-points (a 4x4x4 mesh) corresponds to a canonical EOM-CCSD calculation of 768 electrons in 640 orbitals. We study eight simple semiconductors and insulators, with direct singlet excitation energies in the range of 3 to 15 eV. Our predicted excitation energies exhibit a mean absolute error of 0.27 eV when compared to experiment. We furthermore calculate the energy of excitons with nonzero momentum and compare the exciton dispersion of LiF with experimental data from inelastic X-ray scattering. By calculating excitation energies under strain, we extract hydrostatic deformation potentials in order to quantify the strength of interactions between excitons and acoustic phonons. Our results indicate that coupled-cluster theory is a promising method for the accurate study of a variety of exciton phenomena in solids.

cond-mat.mtrl-sci↗

arXiv2020

Gravitational wave and collider signals in complex two-Higgs doublet model with dynamical CP-violation at finite temperature

Extra CP-violating source for electroweak baryogenesis can dynamically appear at finite temperature in the complex two-Higgs doublet model, which might help to alleviate the strong constraints from the electric dipole moment experiments. In this scenario, we study the detailed phase transition dynamics and the corresponding gravitational wave signals in synergy with the collider signals at future lepton colliders. For some parameter spaces, various phase transition patterns can occur, such as the multi-step phase transition and supercooling. Gravitational waves complementary to collider signals can help to pin down the underlying phase transition dynamics or different phase transition patterns.

hep-ph↗

arXiv2020

Multiplicative Weights Update as a Distributed Constrained Optimization Algorithm: Convergence to Second-order Stationary Points Almost Always

Non-concave maximization has been the subject of much recent study in the optimization and machine learning communities, specifically in deep learning. Recent papers Ge et al, Lee et al (and references therein) indicate that first order methods work well and avoid saddle points. Results as in Lee et al, however, are limited to the \textit{unconstrained} case or for cases where the critical points are in the interior of the feasibility set, which fail to capture some of the most interesting applications. In this paper we focus on \textit{constrained} non-concave maximization. We analyze a variant of a well-established algorithm in machine learning called Multiplicative Weights Update (MWU) for the maximization problem $\max_{\mathbf{x} \in D} P(\mathbf{x})$, where $P$ is non-concave, twice continuously differentiable and $D$ is a product of simplices. We show that MWU converges almost always for small enough stepsizes to critical points that satisfy the second order KKT conditions. We combine techniques from dynamical systems as well as taking advantage of a recent connection between Baum Eagon inequality and MWU (Palaiopanos et al).

math.OC↗

arXiv2020

Exciton interaction induced spin splitting in MoS$_2$ monolayer

By pumping nonresonantly a MoS$_2$ monolayer at $13$ K under a circularly polarized cw laser, we observe exciton energy redshifts that break the degeneracy between B excitons with opposite spin. The energy splitting increases monotonically with the laser power reaching as much as $18$ meV, while it diminishes with the temperature. The phenomenon can be explained theoretically by considering simultaneously the bandgap renormalization which gives rise to the redshift and exciton-exciton Coulomb exchange interaction which is responsible for the spin-dependent splitting. Our results offer a simple scheme to control the valley degree of freedom in MoS$_2$ monolayer and provide an accessible method in investigating many-body exciton exciton interaction in such materials.

cond-mat.mes-hall↗

arXiv2019

Depth-Width Trade-offs for ReLU Networks via Sharkovsky's Theorem

Understanding the representational power of Deep Neural Networks (DNNs) and how their structural properties (e.g., depth, width, type of activation unit) affect the functions they can compute, has been an important yet challenging question in deep learning and approximation theory. In a seminal paper, Telgarsky highlighted the benefits of depth by presenting a family of functions (based on simple triangular waves) for which DNNs achieve zero classification error, whereas shallow networks with fewer than exponentially many nodes incur constant error. Even though Telgarsky's work reveals the limitations of shallow neural networks, it does not inform us on why these functions are difficult to represent and in fact he states it as a tantalizing open question to characterize those functions that cannot be well-approximated by smaller depths. In this work, we point to a new connection between DNNs expressivity and Sharkovsky's Theorem from dynamical systems, that enables us to characterize the depth-width trade-offs of ReLU networks for representing functions based on the presence of generalized notion of fixed points, called periodic points (a fixed point is a point of period 1). Motivated by our observation that the triangle waves used in Telgarsky's work contain points of period 3 - a period that is special in that it implies chaotic behavior based on the celebrated result by Li-Yorke - we proceed to give general lower bounds for the width needed to represent periodic functions as a function of the depth. Technically, the crux of our approach is based on an eigenvalue analysis of the dynamical system associated with such functions.

cs.LG↗

arXiv2019

Hyperbolic Graph Attention Network

Graph neural network (GNN) has shown superior performance in dealing with graphs, which has attracted considerable research attention recently. However, most of the existing GNN models are primarily designed for graphs in Euclidean spaces. Recent research has proven that the graph data exhibits non-Euclidean latent anatomy. Unfortunately, there was rarely study of GNN in non-Euclidean settings so far. To bridge this gap, in this paper, we study the GNN with attention mechanism in hyperbolic spaces at the first attempt. The research of hyperbolic GNN has some unique challenges: since the hyperbolic spaces are not vector spaces, the vector operations (e.g., vector addition, subtraction, and scalar multiplication) cannot be carried. To tackle this problem, we employ the gyrovector spaces, which provide an elegant algebraic formalism for hyperbolic geometry, to transform the features in a graph; and then we propose the hyperbolic proximity based attention mechanism to aggregate the features. Moreover, as mathematical operations in hyperbolic spaces could be more complicated than those in Euclidean spaces, we further devise a novel acceleration strategy using logarithmic and exponential mappings to improve the efficiency of our proposed model. The comprehensive experimental results on four real-world datasets demonstrate the performance of our proposed hyperbolic graph attention network model, by comparisons with other state-of-the-art baseline methods.

cs.LG↗

arXiv2019

Nanoconfined, dynamic electrolyte gating and memory effects in multilayered graphene-based membranes

Multilayered graphene-based nanoporous membranes with electrolyte incorporated between individual sheets is a unique nano-heterostructure system in which nanoconfined electrons in graphene and ions confined in between sheets are intimately coupled throughout the entire membrane. In contrast to the general notion that the electrolyte gating is unlikely to appear in multilayered graphene stacks, it is demonstrated in this work that the electrolyte gating effect in monolayer graphene can be transferred to its corresponding multilayered porous membranes. This gating effect presented on each individual graphene sheets through electrolyte confined in nanopores provides a real-time, electrical approach for probing the complex dynamics of nanoconfined electrical double layer. This has enabled the observation of the ionic memory effect in supercapacitors and produces new insights into the charging dynamics of supercapacitors. Such discoveries may stimulate the design of novel nanoionic devices.

cond-mat.mes-hall↗

arXiv2019

Independence Promoted Graph Disentangled Networks

We address the problem of disentangled representation learning with independent latent factors in graph convolutional networks (GCNs). The current methods usually learn node representation by describing its neighborhood as a perceptual whole in a holistic manner while ignoring the entanglement of the latent factors. However, a real-world graph is formed by the complex interaction of many latent factors (e.g., the same hobby, education or work in social network). While little effort has been made toward exploring the disentangled representation in GCNs. In this paper, we propose a novel Independence Promoted Graph Disentangled Networks (IPGDN) to learn disentangled node representation while enhancing the independence among node representations. In particular, we firstly present disentangled representation learning by neighborhood routing mechanism, and then employ the Hilbert-Schmidt Independence Criterion (HSIC) to enforce independence between the latent representations, which is effectively integrated into a graph convolutional framework as a regularizer at the output layer. Experimental studies on real-world graphs validate our model and demonstrate that our algorithms outperform the state-of-the-arts by a wide margin in different network applications, including semi-supervised graph classification, graph clustering and graph visualization.

cs.LG↗

arXiv2019

Multi-Component Graph Convolutional Collaborative Filtering

The interactions of users and items in recommender system could be naturally modeled as a user-item bipartite graph. In recent years, we have witnessed an emerging research effort in exploring user-item graph for collaborative filtering methods. Nevertheless, the formation of user-item interactions typically arises from highly complex latent purchasing motivations, such as high cost performance or eye-catching appearance, which are indistinguishably represented by the edges. The existing approaches still remain the differences between various purchasing motivations unexplored, rendering the inability to capture fine-grained user preference. Therefore, in this paper we propose a novel Multi-Component graph convolutional Collaborative Filtering (MCCF) approach to distinguish the latent purchasing motivations underneath the observed explicit user-item interactions. Specifically, there are two elaborately designed modules, decomposer and combiner, inside MCCF. The former first decomposes the edges in user-item graph to identify the latent components that may cause the purchasing relationship; the latter then recombines these latent components automatically to obtain unified embeddings for prediction. Furthermore, the sparse regularizer and weighted random sample strategy are utilized to alleviate the overfitting problem and accelerate the optimization. Empirical results on three real datasets and a synthetic dataset not only show the significant performance gains of MCCF, but also well demonstrate the necessity of considering multiple components.

cs.LG↗

arXiv2019

Distributed Iterative CT Reconstruction using Multi-Agent Consensus Equilibrium

Model-Based Image Reconstruction (MBIR) methods significantly enhance the quality of computed tomographic (CT) reconstructions relative to analytical techniques, but are limited by high computational cost. In this paper, we propose a multi-agent consensus equilibrium (MACE) algorithm for distributing both the computation and memory of MBIR reconstruction across a large number of parallel nodes. In MACE, each node stores only a sparse subset of views and a small portion of the system matrix, and each parallel node performs a local sparse-view reconstruction, which based on repeated feedback from other nodes, converges to the global optimum. Our distributed approach can also incorporate advanced denoisers as priors to enhance reconstruction quality. In this case, we obtain a parallel solution to the serial framework of Plug-n-play (PnP) priors, which we call MACE-PnP. In order to make MACE practical, we introduce a partial update method that eliminates nested iterations and prove that it converges to the same global solution. Finally, we validate our approach on a distributed memory system with real CT data. We also demonstrate an implementation of our approach on a massive supercomputer that can perform large-scale reconstruction in real-time.

eess.IV↗

arXiv2019

Learning Generalized Transformation Equivariant Representations via Autoencoding Transformations

Transformation Equivariant Representations (TERs) aim to capture the intrinsic visual structures that equivary to various transformations by expanding the notion of {\em translation} equivariance underlying the success of Convolutional Neural Networks (CNNs). For this purpose, we present both deterministic AutoEncoding Transformations (AET) and probabilistic AutoEncoding Variational Transformations (AVT) models to learn visual representations from generic groups of transformations. While the AET is trained by directly decoding the transformations from the learned representations, the AVT is trained by maximizing the joint mutual information between the learned representation and transformations. This results in Generalized TERs (GTERs) equivariant against transformations in a more general fashion by capturing complex patterns of visual structures beyond the conventional linear equivariance under a transformation group. The presented approach can be extended to (semi-)supervised models by jointly maximizing the mutual information of the learned representation with both labels and transformations. Experiments demonstrate the proposed models outperform the state-of-the-art models in both unsupervised and (semi-)supervised tasks.

cs.CV↗

arXiv2019

Eavesdrop the Composition Proportion of Training Labels in Federated Learning

Federated learning (FL) has recently emerged as a new form of collaborative machine learning, where a common model can be learned while keeping all the training data on local devices. Although it is designed for enhancing the data privacy, we demonstrated in this paper a new direction in inference attacks in the context of FL, where valuable information about training data can be obtained by adversaries with very limited power. In particular, we proposed three new types of attacks to exploit this vulnerability. The first type of attack, Class Sniffing, can detect whether a certain label appears in training. The other two types of attacks can determine the quantity of each label, i.e., Quantity Inference attack determines the composition proportion of the training label owned by the selected clients in a single round, while Whole Determination attack determines that of the whole training process. We evaluated our attacks on a variety of tasks and datasets with different settings, and the corresponding results showed that our attacks work well generally. Finally, we analyzed the impact of major hyper-parameters to our attacks and discussed possible defenses.

cs.LG↗

arXiv2019

Mean exit time for stochastic dynamical systems driven by tempered stable Lévy fluctuations

We use the mean exit time to quantify macroscopic dynamical behaviors of stochastic dynamical systems driven by tempered Lévy fluctuations, which are solutions of nonlocal elliptic equations. Firstly, we construct a new numerical scheme to compute and solve the mean exit time associated with the one dimensional stochastic system. Secondly, we extend the analytical and numerical results to two dimensional case: horizontal-vertical and isotropic case. Finally, we verify the effectiveness of the presented schemes with numerical experiments in several examples.

math.DS↗

arXiv2019

Intracavity-squeezed optomechanical cooling

Quantum ground-state cooling of macroscopic mechanical resonators is of essential importance to both fundamental physics and applied science. Conventional method of laser cooling is limited by the quantum backaction, which requires mechanical sideband resolved in order to cool to ground state. This work presents an idea to break the quantum backaction limit by engineering intracavity optical squeezing. It gives rise to quantum interference for all the dissipation channels, and under certain circumstances can totally remove the influence of the cavity dissipation and the resultant quantum backaction, with much lower cooling limit irrespective of the sideband resolution. We show that our scheme enables ground-state cooling in the highly unresolved sideband limit and it also works beyond the weak coupling regime, which provides the opportunity for quantum manipulation of macroscopic mechanical systems.

quant-ph↗

arXiv2019

ISTHMUS: Secure, Scalable, Real-time and Robust Machine Learning Platform for Healthcare

In recent times, machine learning (ML) and artificial intelligence (AI) based systems have evolved and scaled across different industries such as finance, retail, insurance, energy utilities, etc. Among other things, they have been used to predict patterns of customer behavior, to generate pricing models, and to predict the return on investments. But the successes in deploying machine learning models at scale in those industries have not translated into the healthcare setting. There are multiple reasons why integrating ML models into healthcare has not been widely successful, but from a technical perspective, general-purpose commercial machine learning platforms are not a good fit for healthcare due to complexities in handling data quality issues, mandates to demonstrate clinical relevance, and a lack of ability to monitor performance in a highly regulated environment with stringent security and privacy needs. In this paper, we describe Isthmus, a turnkey, cloud-based platform which addresses the challenges above and reduces time to market for operationalizing ML/AI in healthcare. Towards the end, we describe three case studies which shed light on Isthmus capabilities. These include (1) supporting an end-to-end lifecycle of a model which predicts trauma survivability at hospital trauma centers, (2) bringing in and harmonizing data from disparate sources to create a community data platform for inferring population as well as patient level insights for Social Determinants of Health (SDoH), and (3) ingesting live-streaming data from various IoT sensors to build models, which can leverage real-time and longitudinal information to make advanced time-sensitive predictions.

cs.LG↗