arXiv subjects

Xiaoyu Wang

Publications and source records attributed to Xiaoyu Wang.

At least 235 records · Page 13Linked to original sources

arXiv2021

AIBench Training: Balanced Industry-Standard AI Training Benchmarking

Earlier-stage evaluations of a new AI architecture/system need affordable benchmarks. Only using a few AI component benchmarks like MLPerfalone in the other stages may lead to misleading conclusions. Moreover, the learning dynamics are not well understood, and the benchmarks' shelf-life is short. This paper proposes a balanced benchmarking methodology. We use real-world benchmarks to cover the factors space that impacts the learning dynamics to the most considerable extent. After performing an exhaustive survey on Internet service AI domains, we identify and implement nineteen representative AI tasks with state-of-the-art models. For repeatable performance ranking (RPR subset) and workload characterization (WC subset), we keep two subsets to a minimum for affordability. We contribute by far the most comprehensive AI training benchmark suite. The evaluations show: (1) AIBench Training (v1.1) outperforms MLPerfTraining (v0.7) in terms of diversity and representativeness of model complexity, computational cost, convergent rate, computation, and memory access patterns, and hotspot functions; (2) Against the AIBench full benchmarks, its RPR subset shortens the benchmarking cost by 64%, while maintaining the primary workload characteristics; (3) The performance ranking shows the single-purpose AI accelerator like TPU with the optimized TensorFlowframework performs better than that of GPUs while losing the latter's general support for various AI models. The specification, source code, and performance numbers are available from the AIBench homepage https://www.benchcouncil.org/aibench-training/index.html.

cs.AI↗

arXiv2021

Strong-coupling expansion of multi-band interacting models: mapping onto the transverse-field $J_1$-$J_2$ Ising model

We investigate a class of two-dimensional two-band microscopic models in which the inter-band repulsive interactions play the dominant role. We first demonstrate three different schemes of constraining the ratios between the three types of inter-band interactions -- density-density, spin exchange, and pair-hopping -- that render the model free of the fermionic sign-problem for any filling and, consequently, amenable to efficient Quantum Monte Carlo simulations. We then study the behavior of these sign-problem-free models in the strong-coupling regime. In the cases where spin-rotational invariance is preserved or lowered to a planar symmetry, the strong-coupling ground state is a quantum paramagnet. However, in the case where there is only a residual Ising symmetry, the strong-coupling expansion maps onto the transverse-field $J_1$-$J_2$ Ising model, whose pseudospins are associated with local inter-band magnetic order. We show that by varying the band structure parameters within a reasonable range of values, a variety of ground states and quantum critical points can be accessed in the strong-coupling regime, some of which are not realized in the weak-coupling regime. We compare these results with the case of the single-band Hubbard model, where only intra-band repulsion is present, and whose strong-coupling behavior is captured by a simple Heisenberg model.

cond-mat.str-el↗

arXiv2021

SEPAL: Towards a Large-scale Analysis of SEAndroid Policy Customization

To investigate the status quo of SEAndroid policy customization, we propose SEPAL, a universal tool to automatically retrieve and examine the customized policy rules. SEPAL applies the NLP technique and employs and trains a wide&deep model to quickly and precisely predict whether one rule is unregulated or not.Our evaluation shows SEPAL is effective, practical and scalable. We verify SEPAL outperforms the state of the art approach (i.e., EASEAndroid) by 15% accuracy rate on average. In our experiments, SEPAL successfully identifies 7,111 unregulated policy rules with a low false positive rate from 595,236 customized rules (extracted from 774 Android firmware images of 72 manufacturers). We further discover the policy customization problem is getting worse in newer Android versions (e.g., around 8% for Android 7 and nearly 20% for Android 9), even though more and more efforts are made. Then, we conduct a deep study and discuss why the unregulated rules are introduced and how they can compromise user devices. Last, we report some unregulated rules to seven vendors and so far four of them confirm our findings.

cs.CR↗

arXiv2021

On the Convergence of Step Decay Step-Size for Stochastic Optimization

The convergence of stochastic gradient descent is highly dependent on the step-size, especially on non-convex problems such as neural network training. Step decay step-size schedules (constant and then cut) are widely used in practice because of their excellent convergence and generalization qualities, but their theoretical properties are not yet well understood. We provide the convergence results for step decay in the non-convex regime, ensuring that the gradient norm vanishes at an $\mathcal{O}(\ln T/\sqrt{T})$ rate. We also provide the convergence guarantees for general (possibly non-smooth) convex problems, ensuring an $\mathcal{O}(\ln T/\sqrt{T})$ convergence rate. Finally, in the strongly convex case, we establish an $\mathcal{O}(\ln T/T)$ rate for smooth problems, which we also prove to be tight, and an $\mathcal{O}(\ln^2 T /T)$ rate without the smoothness assumption. We illustrate the practical efficiency of the step decay step-size in several large scale deep neural network training tasks.

math.OC↗

arXiv2021

Electron-Ion Collider in China

Lepton scattering is an established ideal tool for studying inner structure of small particles such as nucleons as well as nuclei. As a future high energy nuclear physics project, an Electron-ion collider in China (EicC) has been proposed. It will be constructed based on an upgraded heavy-ion accelerator, High Intensity heavy-ion Accelerator Facility (HIAF) which is currently under construction, together with a new electron ring. The proposed collider will provide highly polarized electrons (with a polarization of $\sim$80%) and protons (with a polarization of $\sim$70%) with variable center of mass energies from 15 to 20 GeV and the luminosity of (2-3) $\times$ 10$^{33}$ cm$^{-2}$ s$^{-1}$. Polarized deuterons and Helium-3, as well as unpolarized ion beams from Carbon to Uranium, will be also available at the EicC. The main foci of the EicC will be precision measurements of the structure of the nucleon in the sea quark region, including 3D tomography of nucleon; the partonic structure of nuclei and the parton interaction with the nuclear environment; the exotic states, especially those with heavy flavor quark contents. In addition, issues fundamental to understanding the origin of mass could be addressed by measurements of heavy quarkonia near-threshold production at the EicC. In order to achieve the above-mentioned physics goals, a hermetical detector system will be constructed with cutting-edge technologies. This document is the result of collective contributions and valuable inputs from experts across the globe. The EicC physics program complements the ongoing scientific programs at the Jefferson Laboratory and the future EIC project in the United States. The success of this project will also advance both nuclear and particle physics as well as accelerator and detector technology in China.

nucl-ex↗

arXiv2021

On the Convergence of Stochastic Gradient Descent with Bandwidth-based Step Size

We investigate the stochastic gradient descent (SGD) method where the step size lies within a banded region instead of being given by a fixed formula. The optimal convergence rate under mild conditions and large initial step size is proved. Our analysis provides comparable theoretical error bounds for SGD associated with a variety of step sizes. In addition, the convergence rates for some existing step size strategies, e.g., triangular policy and cosine-wave, can be revealed by our analytical framework under the boundary constraints. The bandwidth-based step size provides efficient and flexible step size selection in optimization. We also propose a $1/t$ up-down policy and give several non-monotonic step sizes. Numerical experiments demonstrate the efficiency and significant potential of the bandwidth-based step-size in many applications.

math.OC↗

arXiv2020

A Metastable CaSH$_3$ Phase Composed of HS Honeycomb Sheets that is Superconducting Under Pressure

Evolutionary searches predicted a number of ternary phases that could be synthesized at pressures of 100-300~GPa. $P6_3/mmc$ CaSH$_2$, $Pnma$ CaSH$_2$, $Cmc2_1$ CaSH$_6$, and $I\bar{4}$ CaSH$_{20}$ were composed of a Ca-S lattice along with H$_2$ molecules coordinated in a ``side-on'' fashion to Ca. The H-H bond lengths in these semiconducting phases were elongated because of H$_2$ $σ\rightarrow$ Ca d donation, and Ca d$\rightarrow$ H$_2$ $σ^*$ back-donation, via a Kubas-like mechanism. $P\bar{6}m2$ CaSH$_3$, consisting of two-dimensional HS and CaH$_2$ sheets, was metastable and metallic above \edit{128~GPa}. The presence of van Hove singularities increased its density of states at the Fermi level, and concomitantly the superconducting critical temperature, which was estimated to be as high as \edit{$\sim$100~K at 128~GPa}. This work will inspire the search for superconductivity in materials based upon \edit{honeycomb HX (X=S, Se, Te), and MH$_2$ (M=Mg, Ca, Sr, Ba) layers} under pressure.

cond-mat.supr-con↗

arXiv2020

Superconductivity to 262 kelvin via catalyzed hydrogenation of yttrium at high pressures

Room temperature superconductivity has been achieved under high pressure in an organically derived carbonaceous sulfur hydride with a critical superconducting transition temperature (Tc) of 288 kelvin. This development is part of a new class of dense, hydrogen rich materials with remarkably high critical temperatures. Metal superhydrides are a subclass of these materials that provide a different and potentially more promising route to very high Tc superconductivity. The most promising binary metal superhydrides contain alkaline or rare earth elements, and recent experimental observations of LaH10 have shown them capable of Tc s up to 250 to 260 kelvin. Predictions have shown yttrium superhydrides to be the most promising with an estimated Tc in excess of 300 kelvin for YH10. Here we report the synthesis of an yttrium superhydride that exhibits superconductivity at a critical temperature of 262 kelvin at 182 gigapascal. A palladium thin film assists the synthesis by protecting the sputtered yttrium from oxidation and promoting subsequent hydrogenation. Phonon mediated superconductivity is established by the observation of zero resistance, an isotope effect and the reduction of Tc under an external magnetic field. The upper critical magnetic field is 103 tesla at zero temperature. We suggest YH9 is the synthesized product based on comparison of the measured Raman spectra and Tc to calculated Raman results.

cond-mat.supr-con↗

arXiv2020

Modeling unconventional superconductivity at the crossover between strong and weak electronic interactions

High-temperature superconductivity emerges in a host of different quantum materials, often in a region of the phase diagram where the electronic kinetic energy is comparable in magnitude with the electron-electron Coulomb repulsion. Describing such an intermediate-coupling regime has proven challenging, as standard perturbative approaches are inapplicable. Hence, it is of enormous interest to find models that are amenable to be solved using exact methods. While important advances have been made in elucidating the properties of one such minimal model -- the Hubbard model -- via numerical simulations, the infamous fermionic sign-problem significantly limits the accessible parameter space. Here, we employ Quantum Monte Carlo (QMC) methods to solve a multi-band version of the Hubbard model that does not suffer from the sign-problem and in which only repulsive interband interactions are present. In contrast to previous sign-problem-free QMC studies, this model does not have pre-existing fine-tuned magnetic order, and thus treats superconducting, magnetic, and charge degrees of freedom on an equal footing. We find that, as the electron-electron repulsion increases, a dome of antiferromagnetic order emerges in the intermediate-coupling regime, accompanied by a metal-to-insulator crossover line. Superconductivity is found only near the antiferromagnetic quantum phase transition located on the metallic side of the magnetic dome. Across the antiferromagnetic quantum phase transition we find a change in the dynamical character of the magnetic fluctuations, from slow and overdamped in the metallic side to fast and propagating in the insulating side. Our findings shed new light on the intertwining between superconductivity, magnetism, and charge correlations in quantum materials.

cond-mat.str-el↗

arXiv2020

Generalised Perceptron Learning

We present a generalisation of Rosenblatt's traditional perceptron learning algorithm to the class of proximal activation functions and demonstrate how this generalisation can be interpreted as an incremental gradient method applied to a novel energy function. This novel energy function is based on a generalised Bregman distance, for which the gradient with respect to the weights and biases does not require the differentiation of the activation function. The interpretation as an energy minimisation algorithm paves the way for many new algorithms, of which we explore a novel variant of the iterative soft-thresholding algorithm for the learning of sparse perceptrons.

cs.LG↗

arXiv2020

Low frequency Raman response near Ising-nematic quantum critical point: a memory matrix approach

Recent Raman scattering experiments have revealed a "quasi-elastic peak" in $\mathrm{FeSe_{1-x}S_x}$ near an Ising-nematic quantum critical point (QCP) \cite{zhang17}. Notably, the peak occurs at sub-temperature frequencies, and softens as $T^α$ when temperature is decreased toward the QCP, with $α>1$. In this work, we present a theoretical analysis of the low-frequency Raman response using a memory matrix approach. We show that such a quasi-elastic peak is associated with the relaxation of an Ising-nematic deformation of the Fermi surface. Specifically, we find that the peak frequency is proportional to $ τ^{-1}χ^{-1}$, where $χ$ is the Ising-nematic thermodynamic susceptibility, and $τ^{-1}$ is the decay rate of the nematic deformation due to an interplay between impurity scattering and electron-electron scattering mediated by critical Ising-nematic fluctuations. We argue that the critical fluctuations play a crucial role in determining the observed temperature dependence of the frequency of the quasi-elastic peak. At frequencies larger than the temperature, we find that the Raman response is proportional to $ω^{1/3}$, consistently with earlier predictions \cite{klein18a}.

cond-mat.str-el↗

arXiv2020

A diagnosis of explicit symmetry breaking in the tight-binding constructions for symmetry-protected topological systems

It has been well established that for symmetry protected topological systems, the non-trivial topology prevents a real space representation using exponentially localized Wannier wavefunctions (WFs) in all directions, unless the protecting symmetry is sacrificed as an on-site transformation. This makes it challenging to determine the symmetry of various physical observables represented using such WFs. In this work, we propose a practical method for overcoming such challenges using the Kane-Mele model as a concrete example. We present a systematic procedure for diagnosing the symmetry of any observables, as well as a method for constructing symmetric operators up to arbitrary truncation accuracy.

cond-mat.str-el↗

arXiv2020

The Collins asymmetry in electroproduction of Kaon at the electron ion colliders within TMD factorization

We apply the transverse momentum dependent factorization formalism to investigate the transverse single spin dependent Collins asymmetry with a $\sin(ϕ_h+ϕ_s)$ modulation in the semi-inclusive production of Kaon meson in deep inelastic scattering process. The asymmetry is contributed by the convolutions of the transversity distribution function $h_1(x)$ of the target proton and the Collins function of the Kaon in the final state. We adopt the available parametrization of $h_1(x)$ as well as the recent extracted result for the Kaon Collins function. To perform the transverse momentum dependent evolution, the parametrization of the nonperturbative Sudakov form factor of the proton and final state Kaon are utilized. We numerically predict the Collins asymmetry for charged Kaon production at the electron ion colliders within the accuracy of next-to-leading-logarithmic order. It is found that the asymmetry is sizable and could be measured. We emphasize the importance of planned electron ion colliders in the aspect of constraining sea quark distribution functions as well as accessing the information of the nucleon spin structure and the hadronization mechanism.

hep-ph↗

arXiv2020

Stochastic Optimization for Non-convex Inf-Projection Problems

In this paper, we study a family of non-convex and possibly non-smooth inf-projection minimization problems, where the target objective function is equal to minimization of a joint function over another variable. This problem include difference of convex (DC) functions and a family of bi-convex functions as special cases. We develop stochastic algorithms and establish their first-order convergence for finding a (nearly) stationary solution of the target non-convex function under different conditions of the component functions. To the best of our knowledge, this is the first work that comprehensively studies stochastic optimization of non-convex inf-projection minimization problems with provable convergence guarantee. Our algorithms enable efficient stochastic optimization of a family of non-decomposable DC functions and a family of bi-convex functions. To demonstrate the power of the proposed algorithms we consider an important application in variance-based regularization. Experiments verify the effectiveness of our inf-projection based formulation and the proposed stochastic algorithm in comparison with previous stochastic algorithms based on the min-max formulation for achieving the same effect.

cs.LG↗

arXiv2020

Weakly Supervised Temporal Action Localization with Segment-Level Labels

Temporal action localization presents a trade-off between test performance and annotation-time cost. Fully supervised methods achieve good performance with time-consuming boundary annotations. Weakly supervised methods with cheaper video-level category label annotations result in worse performance. In this paper, we introduce a new segment-level supervision setting: segments are labeled when annotators observe actions happening here. We incorporate this segment-level supervision along with a novel localization module in the training. Specifically, we devise a partial segment loss regarded as a loss sampling to learn integral action parts from labeled segments. Since the labeled segments are only parts of actions, the model tends to overfit along with the training process. To tackle this problem, we first obtain a similarity matrix from discriminative features guided by a sphere loss. Then, a propagation loss is devised based on the matrix to act as a regularization term, allowing implicit unlabeled segments propagation during training. Experiments validate that our method can outperform the video-level supervision methods with almost same the annotation time.

cs.CV↗

arXiv2020

A Simple and Effective Framework for Pairwise Deep Metric Learning

Deep metric learning (DML) has received much attention in deep learning due to its wide applications in computer vision. Previous studies have focused on designing complicated losses and hard example mining methods, which are mostly heuristic and lack of theoretical understanding. In this paper, we cast DML as a simple pairwise binary classification problem that classifies a pair of examples as similar or dissimilar. It identifies the most critical issue in this problem--imbalanced data pairs. To tackle this issue, we propose a simple and effective framework to sample pairs in a batch of data for updating the model. The key to this framework is to define a robust loss for all pairs over a mini-batch of data, which is formulated by distributionally robust optimization. The flexibility in constructing the uncertainty decision set of the dual variable allows us to recover state-of-the-art complicated losses and also to induce novel variants. Empirical studies on several benchmark data sets demonstrate that our simple and effective method outperforms the state-of-the-art results. Codes are available at: https://github.com/qiqi-helloworld/A-Simple-and-Effective-Framework-for-Pairewise-Distance-Metric-Learning

cs.LG↗

arXiv2020

Non-Convex Optimization via Non-Reversible Stochastic Gradient Langevin Dynamics

Stochastic Gradient Langevin Dynamics (SGLD) is a powerful algorithm for optimizing a non-convex objective, where a controlled and properly scaled Gaussian noise is added to the stochastic gradients to steer the iterates towards a global minimum. SGLD is based on the overdamped Langevin diffusion which is reversible in time. By adding an anti-symmetric matrix to the drift term of the overdamped Langevin diffusion, one gets a non-reversible diffusion that converges to the same stationary distribution with a faster convergence rate. In this paper, we study the non reversible Stochastic Gradient Langevin Dynamics (NSGLD) which is based on discretization of the non-reversible Langevin diffusion. We provide finite-time performance bounds for the global convergence of NSGLD for solving stochastic non-convex optimization problems. Our results lead to non-asymptotic guarantees for both population and empirical risk minimization problems. Numerical experiments for Bayesian independent component analysis and neural network models show that NSGLD can outperform SGLD with proper choices of the anti-symmetric matrix.

math.OC↗

arXiv2020

Robust Partial Matching for Person Search in the Wild

Various factors like occlusions, backgrounds, etc., would lead to misaligned detected bounding boxes , e.g., ones covering only portions of human body. This issue is common but overlooked by previous person search works. To alleviate this issue, this paper proposes an Align-to-Part Network (APNet) for person detection and re-Identification (reID). APNet refines detected bounding boxes to cover the estimated holistic body regions, from which discriminative part features can be extracted and aligned. Aligned part features naturally formulate reID as a partial feature matching procedure, where valid part features are selected for similarity computation, while part features on occluded or noisy regions are discarded. This design enhances the robustness of person search to real-world challenges with marginal computation overhead. This paper also contributes a Large-Scale dataset for Person Search in the wild (LSPS), which is by far the largest and the most challenging dataset for person search. Experiments show that APNet brings considerable performance improvement on LSPS. Meanwhile, it achieves competitive performance on existing person search benchmarks like CUHK-SYSU and PRW.

cs.CV↗