arXiv subjects

Yuxin Wang

Publications and source records attributed to Yuxin Wang.

At least 217 records · Page 12Linked to original sources

arXiv2020

Free Gap Estimates from the Exponential Mechanism, Sparse Vector, Noisy Max and Related Algorithms

Private selection algorithms, such as the Exponential Mechanism, Noisy Max and Sparse Vector, are used to select items (such as queries with large answers) from a set of candidates, while controlling privacy leakage in the underlying data. Such algorithms serve as building blocks for more complex differentially private algorithms. In this paper we show that these algorithms can release additional information related to the gaps between the selected items and the other candidates for free (i.e., at no additional privacy cost). This free gap information can improve the accuracy of certain follow-up counting queries by up to 66%. We obtain these results from a careful privacy analysis of these algorithms. Based on this analysis, we further propose novel hybrid algorithms that can dynamically save additional privacy budget.

cs.DB↗

arXiv2020

Benchmarking the Performance and Energy Efficiency of AI Accelerators for AI Training

Deep learning has become widely used in complex AI applications. Yet, training a deep neural network (DNNs) model requires a considerable amount of calculations, long running time, and much energy. Nowadays, many-core AI accelerators (e.g., GPUs and TPUs) are designed to improve the performance of AI training. However, processors from different vendors perform dissimilarly in terms of performance and energy consumption. To investigate the differences among several popular off-the-shelf processors (i.e., Intel CPU, NVIDIA GPU, AMD GPU, and Google TPU) in training DNNs, we carry out a comprehensive empirical study on the performance and energy efficiency of these processors by benchmarking a representative set of deep learning workloads, including computation-intensive operations, classical convolutional neural networks (CNNs), recurrent neural networks (LSTM), Deep Speech 2, and Transformer. Different from the existing end-to-end benchmarks which only present the training time, We try to investigate the impact of hardware, vendor's software library, and deep learning framework on the performance and energy consumption of AI training. Our evaluation methods and results not only provide an informative guide for end-users to select proper AI accelerators, but also expose some opportunities for the hardware vendors to improve their software library.

cs.DC↗

arXiv2020

CheckDP: An Automated and Integrated Approach for Proving Differential Privacy or Finding Precise Counterexamples

We propose CheckDP, the first automated and integrated approach for proving or disproving claims that a mechanism is differentially private. CheckDP can find counterexamples for mechanisms with subtle bugs for which prior counterexample generators have failed. Furthermore, it was able to \emph{automatically} generate proofs for correct mechanisms for which no formal verification was reported before. CheckDP is built on static program analysis, allowing it to be more efficient and more precise in catching infrequent events than existing counterexample generators (which run mechanisms hundreds of thousands of times to estimate their output distribution). Moreover, its sound approach also allows automatic verification of correct mechanisms. When evaluated on standard benchmarks and newer privacy mechanisms, CheckDP generates proofs (for correct mechanisms) and counterexamples (for incorrect mechanisms) within 70 seconds without any false positives or false negatives.

cs.PL↗

arXiv2020

Domain Priori Knowledge based Integrated Solution Design for Internet of Services

Various types of services, such as web APIs, IoT services, O2O services, and many others, have flooded on the Internet. Interconnections among these services have resulted in a new phenomenon called "Internet of Services" (IoS). By IoS,people don't need to request multiple services by themselves to fulfill their daily requirements, but it is an IoS platform that is responsible for constructing integrated solutions for them. Since user requirements (URs) are usually coarse-grained and transboundary, IoS platforms have to integrate services from multiple domains to fulfill the requirements. Considering there are too many available services in IoS, a big challenge is how to look for a tradeoff between the construction efficiency and the precision of final solutions. For this challenge, we introduce a framework and a platform for transboundary user requirement oriented solution design in IoS. The main idea is to make use of domain priori knowledge derived from the commonness and similarities among massive historical URs and among historical integrated service solutions(ISSs). Priori knowledge is classified into three types: requirement patterns (RPs), service patterns (SPs), and probabilistic matching matrix (PMM) between RPs and SPs. A UR is modeled in the form of an intention tree (ITree) along with a set of constraints on intention nodes, and then optimal RPs are selected to cover the I-Tree as much as possible. By taking advantage of the PMM, a set of SPs are filtered out and composed together to form the final ISS. Finally, the design of a platform supporting the above process is introduced.

cs.SE↗

arXiv2020

Precision measurements with cold atoms and trapped ions

Recent progresses on quantum control of cold atoms and trapped ions in both the scientific and technological aspects greatly advance the applications in precision measurement. Thanks to the exceptional controllability and versatility of these massive quantum systems, unprecedented sensitivity has been achieved in clocks, magnetometers and interferometers based on cold atoms and ions. Besides, these systems also feature many characteristics that can be employed to facilitate the applications in different scenarios. In this review, we briefly introduce the principles of optical clocks, cold atom magnetometers and atom interferometers used for precision measurement of time, magnetic field, and inertial forces. The main content is then devoted to summarize some recent experimental and theoretical progresses in these three applications, with special attention being paid to the new designs and possibilities towards better performance. The purpose of this review is by no means to give a complete overview of all important works in this fast developing field, but to draw a rough sketch about the frontiers and show the fascinating future lying ahead.

physics.atom-ph↗

arXiv2020

ContourNet: Taking a Further Step toward Accurate Arbitrary-shaped Scene Text Detection

Scene text detection has witnessed rapid development in recent years. However, there still exists two main challenges: 1) many methods suffer from false positives in their text representations; 2) the large scale variance of scene texts makes it hard for network to learn samples. In this paper, we propose the ContourNet, which effectively handles these two problems taking a further step toward accurate arbitrary-shaped text detection. At first, a scale-insensitive Adaptive Region Proposal Network (Adaptive-RPN) is proposed to generate text proposals by only focusing on the Intersection over Union (IoU) values between predicted and ground-truth bounding boxes. Then a novel Local Orthogonal Texture-aware Module (LOTM) models the local texture information of proposal features in two orthogonal directions and represents text region with a set of contour points. Considering that the strong unidirectional or weakly orthogonal activation is usually caused by the monotonous texture characteristic of false-positive patterns (e.g. streaks.), our method effectively suppresses these false positives by only outputting predictions with high response value in both orthogonal directions. This gives more accurate description of text regions. Extensive experiments on three challenging datasets (Total-Text, CTW1500 and ICDAR2015) verify that our method achieves the state-of-the-art performance. Code is available at https://github.com/wangyuxin87/ContourNet.

cs.CV↗

arXiv2020

High-performance frequency stabilization of ultraviolet diode lasers by using dichroic atomic vapor spectroscopy and transfer cavity

Ultraviolet (UV) diode lasers are widely used in many photonics applications. But their frequency stabilization schemes are not as mature as frequency-doubling lasers, mainly due to some limitations in the UV spectral region. Here we developed a high-performance UV frequency stabilization technique implemented directly on UV diode lasers by combining the dichroic atomic vapor laser lock and the resonant transfer cavity lock. As an example, we demonstrate a stable locking with frequency standard deviations of approximately 200 KHz and 300 KHz for 399nm and 370nm diode lasers in 20 minutes. We achieve a long-term frequency drift of no more than 1 MHz for the target 370nm laser within an hour, which was further verified with fluorescence counts rates of a single trapped $^{171}$Yb$^+$ ion. We also find strong linear correlations between lock points and environmental factors such as temperature and atmospheric pressure.

physics.optics↗

arXiv2020

A Survey of Deep Learning Techniques for Neural Machine Translation

In recent years, natural language processing (NLP) has got great development with deep learning techniques. In the sub-field of machine translation, a new approach named Neural Machine Translation (NMT) has emerged and got massive attention from both academia and industry. However, with a significant number of researches proposed in the past several years, there is little work in investigating the development process of this new technology trend. This literature survey traces back the origin and principal development timeline of NMT, investigates the important branches, categorizes different research orientations, and discusses some future research trends in this field.

cs.CL↗

arXiv2020

Torsion Discriminance for Stability of Linear Time-Invariant Systems

This paper proposes a new approach to describe the stability of linear time-invariant systems via the torsion $τ(t)$ of the state trajectory. For a system $\dot{r}(t)=Ar(t)$ where $A$ is invertible, we show that (1) if there exists a measurable set $E_1$ with positive Lebesgue measure, such that $r(0)\in E_1$ implies that $\lim\limits_{t\to+\infty}τ(t)\neq0$ or $\lim\limits_{t\to+\infty}τ(t)$ does not exist, then the zero solution of the system is stable; (2) if there exists a measurable set $E_2$ with positive Lebesgue measure, such that $r(0)\in E_2$ implies that $\lim\limits_{t\to+\infty}τ(t)=+\infty$, then the zero solution of the system is asymptotically stable. Furthermore, we establish a relationship between the $i$th curvature $(i=1,2,\cdots)$ of the trajectory and the stability of the zero solution when $A$ is similar to a real diagonal matrix.

math.OC↗

arXiv2019

Anisotropic Bi2O2Se(Te) Monolayer: Realizing Ultra-High Carrier Mobility and Giant Electric Polarization in Two-Dimension

Here, we have identified the monolayer phase of Bi2O2Se as a promising two-dimensional semiconductor with ultra-high carrier mobility and giant electric polarization. Due to the strong reconstruction originated from the interlayer electrostatic force, we have applied structure prediction algorithms to explore the crystalline geometry of Bi2O2Se monolayer with the lowest total energy. Considering Se and Te belong to the same group, Bi2O2Te monolayer is also investigated based on a similar scheme. Further calculations suggest that the high carrier mobility is maintained in the monolayer phase and the moderate band gap will lead to the strong optical absorption in the visible light region. In particular, the electron mobility in Bi2O2Te can reach as high as 3610 cm2V-1s-1 at room temperature, which is almost ten times of conventional transition metal dichalcogenides (TMD) family. Because of the strong structural anisotropy, a remarkable spontaneous in-plane and out-of-plane electric polarization is additionally revealed along with significant piezoelectric properties, endowing them as promising candidates in the area of photovoltaic solar cells, optoelectronic materials and field effect transistors.

cond-mat.mtrl-sci↗

arXiv2019

Computer-Aided Clinical Skin Disease Diagnosis Using CNN and Object Detection Models

Skin disease is one of the most common types of human diseases, which may happen to everyone regardless of age, gender or race. Due to the high visual diversity, human diagnosis highly relies on personal experience; and there is a serious shortage of experienced dermatologists in many countries. To alleviate this problem, computer-aided diagnosis with state-of-the-art (SOTA) machine learning techniques would be a promising solution. In this paper, we aim at understanding the performance of convolutional neural network (CNN) based approaches. We first build two versions of skin disease datasets from Internet images: (a) Skin-10, which contains 10 common classes of skin disease with a total of 10,218 images; (b) Skin-100, which is a larger dataset that consists of 19,807 images of 100 skin disease classes. Based on these datasets, we benchmark several SOTA CNN models and show that the accuracy of skin-100 is much lower than the accuracy of skin-10. We then implement an ensemble method based on several CNN models and achieve the best accuracy of 79.01\% for Skin-10 and 53.54\% for Skin-100. We also present an object detection based approach by introducing bounding boxes into the Skin-10 dataset. Our results show that object detection can help improve the accuracy of some skin disease classes.

eess.IV↗

arXiv2019

Free Gap Information from the Differentially Private Sparse Vector and Noisy Max Mechanisms

Noisy Max and Sparse Vector are selection algorithms for differential privacy and serve as building blocks for more complex algorithms. In this paper we show that both algorithms can release additional information for free (i.e., at no additional privacy cost). Noisy Max is used to return the approximate maximizer among a set of queries. We show that it can also release for free the noisy gap between the approximate maximizer and runner-up. This free information can improve the accuracy of certain subsequent counting queries by up to 50%. Sparse Vector is used to return a set of queries that are approximately larger than a fixed threshold. We show that it can adaptively control its privacy budget (use less budget for queries that are likely to be much larger than the threshold) in order to increase the amount of queries it can process. These results follow from a careful privacy analysis.

cs.LG↗

arXiv2019

Detecting Violations of Differential Privacy

The widespread acceptance of differential privacy has led to the publication of many sophisticated algorithms for protecting privacy. However, due to the subtle nature of this privacy definition, many such algorithms have bugs that make them violate their claimed privacy. In this paper, we consider the problem of producing counterexamples for such incorrect algorithms. The counterexamples are designed to be short and human-understandable so that the counterexample generator can be used in the development process -- a developer could quickly explore variations of an algorithm and investigate where they break down. Our approach is statistical in nature. It runs a candidate algorithm many times and uses statistical tests to try to detect violations of differential privacy. An evaluation on a variety of incorrect published algorithms validates the usefulness of our approach: it correctly rejects incorrect algorithms and provides counterexamples for them within a few seconds.

cs.CR↗

arXiv2019

Proving Differential Privacy with Shadow Execution

Recent work on formal verification of differential privacy shows a trend toward usability and expressiveness -- generating a correctness proof of sophisticated algorithm while minimizing the annotation burden on programmers. Sometimes, combining those two requires substantial changes to program logics: one recent paper is able to verify Report Noisy Max automatically, but it involves a complex verification system using customized program logics and verifiers. In this paper, we propose a new proof technique, called shadow execution, and embed it into a language called ShadowDP. ShadowDP uses shadow execution to generate proofs of differential privacy with very few programmer annotations and without relying on customized logics and verifiers. In addition to verifying Report Noisy Max, we show that it can verify a new variant of Sparse Vector that reports the gap between some noisy query answers and the noisy threshold. Moreover, ShadowDP reduces the complexity of verification: for all of the algorithms we have evaluated, type checking and verification in total takes at most 3 seconds, while prior work takes minutes on the same algorithms.

cs.PL↗

arXiv2019

The Impact of GPU DVFS on the Energy and Performance of Deep Learning: an Empirical Study

Over the past years, great progress has been made in improving the computing power of general-purpose graphics processing units (GPGPUs), which facilitates the prosperity of deep neural networks (DNNs) in multiple fields like computer vision and natural language processing. A typical DNN training process repeatedly updates tens of millions of parameters, which not only requires huge computing resources but also consumes significant energy. In order to train DNNs in a more energy-efficient way, we empirically investigate the impact of GPU Dynamic Voltage and Frequency Scaling (DVFS) on the energy consumption and performance of deep learning. Our experiments cover a wide range of GPU architectures, DVFS settings, and DNN configurations. We observe that, compared to the default core frequency settings of three tested GPUs, the optimal core frequency can help conserve 8.7%$\sim$23.1% energy consumption for different DNN training cases. Regarding the inference, the benefits vary from 19.6%$\sim$26.4%. Our findings suggest that GPU DVFS has great potentials to help develop energy efficient DNN training/inference schemes.

cs.PF↗

arXiv2019

A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks

Distributed synchronous stochastic gradient descent (S-SGD) has been widely used in training large-scale deep neural networks (DNNs), but it typically requires very high communication bandwidth between computational workers (e.g., GPUs) to exchange gradients iteratively. Recently, Top-$k$ sparsification techniques have been proposed to reduce the volume of data to be exchanged among workers. Top-$k$ sparsification can zero-out a significant portion of gradients without impacting the model convergence. However, the sparse gradients should be transferred with their irregular indices, which makes the sparse gradients aggregation difficult. Current methods that use AllGather to accumulate the sparse gradients have a communication complexity of $O(kP)$, where $P$ is the number of workers, which is inefficient on low bandwidth networks with a large number of workers. We observe that not all top-$k$ gradients from $P$ workers are needed for the model update, and therefore we propose a novel global Top-$k$ (gTop-$k$) sparsification mechanism to address the problem. Specifically, we choose global top-$k$ largest absolute values of gradients from $P$ workers, instead of accumulating all local top-$k$ gradients to update the model in each iteration. The gradient aggregation method based on gTop-$k$ sparsification reduces the communication complexity from $O(kP)$ to $O(k\log P)$. Through extensive experiments on different DNNs, we verify that gTop-$k$ S-SGD has nearly consistent convergence performance with S-SGD, and it has only slight degradations on generalization performance. In terms of scaling efficiency, we evaluate gTop-$k$ on a cluster with 32 GPU machines which are interconnected with 1 Gbps Ethernet. The experimental results show that our method achieves $2.7-12\times$ higher scaling efficiency than S-SGD and $1.1-1.7\times$ improvement than the existing Top-$k$ S-SGD.

cs.DC↗

arXiv2018

Description of Stability for Linear Time-Invariant Systems Based on the First Curvature

This paper focuses on using the first curvature $κ(t)$ of trajectory to describe the stability of linear time-invariant system. We extend the results for two and three-dimensional systems [Y. Wang, H. Sun, Y. Song et al., arXiv:1808.00290] to $n$-dimensional systems. We prove that for a system $\dot{r}(t)=Ar(t)$, (i) if there exists a measurable set whose Lebesgue measure is greater than zero, such that for all initial values in this set, $\lim\limits_{t\to+\infty}κ(t)\neq0$ or $\lim\limits_{t\to+\infty}κ(t)$ does not exist, then the zero solution of the system is stable; (ii) if the matrix $A$ is invertible, and there exists a measurable set whose Lebesgue measure is greater than zero, such that for all initial values in this set, $\lim\limits_{t\to+\infty}κ(t)=+\infty$, then the zero solution of the system is asymptotically stable.

math.OC↗

arXiv2018

Description of Stability for Two and Three-Dimensional Linear Time-Invariant Systems Based on Curvature and Torsion

This paper focuses on using curvature and torsion to describe the stability of linear time-invariant system. We prove that for a two-dimensional system $\dot{r}(t)= Ar(t)$, (i) if there exists an initial value, such that zero is not the limit of curvature of trajectory as $t\to+\infty$, then the zero solution of the system is stable; (ii) if there exists an initial value, such that the limit of curvature of trajectory is infinity as $t\to+\infty$, then the zero solution of the system is asymptotically stable. For a three-dimensional system, (i) if there exists a measurable set whose Lebesgue measure is greater than zero, such that for all initial values in this set, zero is not the limit of curvature of trajectory as $t\to+\infty$, then the zero solution of the system is stable; (ii) if the coefficient matrix is invertible, and there exists a measurable set whose Lebesgue measure is greater than zero, such that for all initial values in this set, the limit of curvature of trajectory is infinity as $t\to+\infty$, then the zero solution of the system is asymptotically stable; (iii) if there exists a measurable set whose Lebesgue measure is greater than zero, such that for all initial values in this set, zero is not the limit of torsion of trajectory as $t\to+\infty$, then the zero solution of the system is asymptotically stable.

math.OC↗