arXiv subjects

Wei Wu

Publications and source records attributed to Wei Wu.

At least 541 records · Page 30Linked to original sources

arXiv2018

SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks

Going deeper and wider in neural architectures improves the accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) practitioners either need change to less desired network architectures, or nontrivially dissect a network across multiGPUs. These distract DL practitioners from concentrating on their original machine learning tasks. We present SuperNeurons: a dynamic GPU memory scheduling runtime to enable the network training far beyond the GPU DRAM capacity. SuperNeurons features 3 memory optimizations, \textit{Liveness Analysis}, \textit{Unified Tensor Pool}, and \textit{Cost-Aware Recomputation}, all together they effectively reduce the network-wide peak memory usage down to the maximal memory usage among layers. We also address the performance issues in those memory saving techniques. Given the limited GPU DRAM, SuperNeurons not only provisions the necessary memory for the training, but also dynamically allocates the memory for convolution workspaces to achieve the high performance. Evaluations against Caffe, Torch, MXNet and TensorFlow have demonstrated that SuperNeurons trains at least 3.2432 deeper network than current ones with the leading performance. Particularly, SuperNeurons can train ResNet2500 that has $10^4$ basic network layers on a 12GB K40c.

cs.DC↗

arXiv2018

Statistical theory on the analytical form of cloud particle size distributions

Several analytical forms of cloud particle size distributions (PSDs) have been used in numerical modeling and remote sensing retrieval studies of clouds and precipitation, including exponential, gamma, lognormal, and Weibull distributions. However, there is no satisfying physical explanation as to why certain distribution forms preferentially occur instead of others. Theoretically, the analytical form of a PSD can be derived by directly solving the general dynamic equation, but no analytical solutions have been found yet. Instead of a process level approach, the use of the principle of maximum entropy (MaxEnt) for determining the analytical form of PSDs from the perspective of system is examined here. MaxEnt theory states that the probability density function with the largest information entropy among a group satisfying the given properties of the variable should be chosen. Here, the issue of variability under coordinate transformations that arises using the Gibbs/Shannon definition of entropy is identified, and the use of the concept of relative entropy to avoid these problems is discussed. Focusing on cloud physics, the four-parameter generalized gamma distribution is proposed as the analytical form of a PSD using the principle of maximum (relative) entropy with assumptions on power law relations between state variables, scale invariance and a further constraint on the expectation of one state variable (e.g. bulk water mass). The four parameter generalized gamma distribution is very flexible to accommodate various type of constraints that could be assumed for cloud PSDs.

physics.ao-ph↗

arXiv2018

Potential landscape and flux field theory for turbulence and nonequilibrium fluid systems

Turbulence is a paradigm for far-from-equilibrium systems without time reversal symmetry. To capture the nonequilibrium irreversible nature of turbulence and investigate its implications, we develop a potential landscape and flux field theory for turbulent flow and more general nonequilibrium fluid systems governed by stochastic Navier-Stokes equations. We find that equilibrium fluid systems with time reversibility are characterized by a detailed balance constraint that quantifies the detailed balance condition. In nonequilibrium fluid systems with nonequilibrium steady states, detailed balance breaking leads directly to a pair of interconnected consequences, namely, the non-Gaussian potential landscape and the irreversible probability flux, forming a 'nonequilibrium trinity'. The nonequilibrium trinity characterizes the nonequilibrium irreversible essence of fluid systems with intrinsic time irreversibility and is manifested in various aspects of these systems. The nonequilibrium stochastic dynamics of fluid systems including turbulence with detailed balance breaking is shown to be driven by both the non-Gaussian potential landscape gradient and the irreversible probability flux, together with the reversible convective force and the stochastic stirring force. We reveal an underlying connection of the energy flux essential for turbulence energy cascade to the irreversible probability flux and the non-Gaussian potential landscape generated by detailed balance breaking. Using the energy flux as a center of connection, we demonstrate that the four-fifths law in fully developed turbulence is a consequence and reflection of the nonequilibrium trinity. We also show how the nonequilibrium trinity can affect the scaling laws in turbulence.

physics.flu-dyn↗

arXiv2017

Coherent dynamics of a qubit-oscillator system in a noisy environment

We investigate the non-Markovian dynamics of a qubit-oscillator system embedded in a noisy environment by employing the hierarchical equations of motion approach. It is found that the decoherence rate of the whole qubit-oscillator-bath system can be significantly suppressed by enhancing the coupling strength between the qubit and the harmonic oscillator. Moreover, we find that the non-Markovian memory character of the bath is able to facilitate a robust quantum coherent dynamics in this qubit-oscillator-bath system. Our findings may be used to engineer some tunable coherent manipulations in mesoscopic quantum circuits.

quant-ph↗

arXiv2017

A Provably Secure Ring Signature Scheme in Certificateless Cryptography

Ring signature is a kind of group-oriented signature. It allows a member of a group to sign messages on behalf of the group without revealing his/her identity. Certificateless public key cryptography was first introduced by Al-Riyami and Paterson in Asiacrypt 2003. In certificateless cryptography, it does not require the use of certificates to guarantee the authenticity of users' public keys. Meanwhile, certificateless cryptography does not have the key escrow problem, which seems to be inherent in the Identity-based cryptography. In this paper, we propose a concrete certificateless ring signature scheme. The security models of certificateless ring signature are also formalized. Our new scheme is provably secure in the random oracle model, with the assumption that the Computational Diffie-Hellman problem is hard. In addition, we also show that a generic construction of certificateless ring signature is insecure against the key replacement attack defined in our security models.

cs.CR↗

arXiv2017

Double-negative electromagnetic metamaterials due to chirality

The aim of this paper is to provide a mathematical theory for understanding the mechanism behind the double-negative refractive index phenomenon in chiral materials. The design of double-negative metamaterials generally requires the use of two different kinds of subwavelength resonators, which may limit the applicability of double-negative metamaterials. Herein, we rely on media that consist of only a single type of dielectric resonant element, and show how the chirality of the background medium induces double-negative refractive index metamaterial, which refracts waves negatively, hence acting as a superlens. Using plasmonic dielectric particles, it is proved that both the effective electric permittivity and the magnetic permeability can be negative near some resonant frequencies. A justification of the approximation of a plasmonic particle in a chiral medium by the sum of a resonant electric dipole and a resonant magnetic dipole, is provided. Moreover, the set of resonant frequencies is characterized. For an appropriate volume fraction of plasmonic particles with certain conditions on their configuration, a double-negative effective medium can be obtained when the frequency is near one of the resonant frequencies.

math.AP↗

arXiv2017

Neural Response Generation with Dynamic Vocabularies

We study response generation for open domain conversation in chatbots. Existing methods assume that words in responses are generated from an identical vocabulary regardless of their inputs, which not only makes them vulnerable to generic patterns and irrelevant noise, but also causes a high cost in decoding. We propose a dynamic vocabulary sequence-to-sequence (DVS2S) model which allows each input to possess their own vocabulary in decoding. In training, vocabulary construction and response generation are jointly learned by maximizing a lower bound of the true objective with a Monte Carlo sampling method. In inference, the model dynamically allocates a small vocabulary for an input with the word prediction model, and conducts decoding only with the small vocabulary. Because of the dynamic vocabulary mechanism, DVS2S eludes many generic patterns and irrelevant words in generation, and enjoys efficient decoding at the same time. Experimental results on both automatic metrics and human annotations show that DVS2S can significantly outperform state-of-the-art methods in terms of response quality, but only requires 60% decoding time compared to the most efficient baseline.

cs.CL↗

arXiv2017

A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots

We study the problem of response selection for multi-turn conversation in retrieval-based chatbots. The task requires matching a response candidate with a conversation context, whose challenges include how to recognize important parts of the context, and how to model the relationships among utterances in the context. Existing matching methods may lose important information in contexts as we can interpret them with a unified framework in which contexts are transformed to fixed-length vectors without any interaction with responses before matching. The analysis motivates us to propose a new matching framework that can sufficiently carry the important information in contexts to matching and model the relationships among utterances at the same time. The new framework, which we call a sequential matching framework (SMF), lets each utterance in a context interacts with a response candidate at the first step and transforms the pair to a matching vector. The matching vectors are then accumulated following the order of the utterances in the context with a recurrent neural network (RNN) which models the relationships among the utterances. The context-response matching is finally calculated with the hidden states of the RNN. Under SMF, we propose a sequential convolutional network and sequential attention network and conduct experiments on two public data sets to test their performance. Experimental results show that both models can significantly outperform the state-of-the-art matching methods. We also show that the models are interpretable with visualizations that provide us insights on how they capture and leverage the important information in contexts for matching.

cs.CL↗

arXiv2017

Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55

We introduce a large-scale 3D shape understanding benchmark using data and annotation from ShapeNet 3D object database. The benchmark consists of two tasks: part-level segmentation of 3D shapes and 3D reconstruction from single view images. Ten teams have participated in the challenge and the best performing teams have outperformed state-of-the-art approaches on both tasks. A few novel deep learning architectures have been proposed on various 3D representations on both tasks. We report the techniques used by each team and the corresponding performances. In addition, we summarize the major discoveries from the reported results and possible trends for the future work in the field.

cs.CV↗

arXiv2017

Effects of counter-rotating-wave terms on the non-Markovianity in quantum open systems

We investigate the effect of counter-rotating-wave terms on the non-Markovianity in quantum open systems by employing the hierarchical equations of motion in the framework of the non-Markovian quantum state diffusion approach. As illustrative examples, the non-Markovian memory effect of a qubit embedded in a bosonic and a fermionic environment at zero temperature are analyzed. It is found that the counter-rotating-wave terms are able to enhance the observed non-Markovianity no matter the environment is composed of bosons or fermions. This result suggests that the rotating-wave approximation may inevitably reduce the non-Markovianity in quantum open systems. Moreover, we find that the modification of the non-Markovianity due to the different statistical properties of environmental modes becomes larger with the increase of the system-environment coupling strength.

quant-ph↗

arXiv2017

Atomically-thin Femtojoule Filamentary Memristor

The morphology and dimension of the conductive filament formed in a memristive device are strongly influenced by the thickness of its switching medium layer. Aggressive scaling of this active layer thickness is critical towards reducing the operating current, voltage and energy consumption in filamentary type memristors. Previously, the thickness of this filament layer has been limited to above a few nanometers due to processing constraints, making it challenging to further suppress the on-state current and the switching voltage. Here, we study the formation of conductive filaments in a material medium with sub-nanometer thickness, formed through the oxidation of atomically-thin two-dimensional boron nitride. The resulting memristive device exhibits sub-nanometer filamentary switching with sub-pA operation current and femtojoule per bit energy consumption. Furthermore, by confining the filament to the atomic scale, we observe current switching characteristics that are distinct from that in thicker medium due to the profoundly different atomic kinetics. The filament morphology in such an aggressively scaled memristive device is also theoretically explored. These ultra-low energy devices are promising for realizing femtojoule and sub-femtojoule electronic computation, which can be attractive for applications in a wide range of electronics systems that desire ultra-low power operation.

physics.app-ph↗

arXiv2017

A Central Limit Theorem for First Passage Percolation in the Slab

We consider first-passage percolation on the edges of $\mathbb{Z}^2 \times k,$ namely the slab of width $k$. Each edge is assigned independently a passage time of either 0 (with probability $1-p_c(\mathbb{S}_k)$) or 1 ((with probability $p_c(\mathbb{S}_k)$) where $p_c$ is the critical probability. We prove central limit theorems for point-to-point and point-to-line passage times. These generalize the results of [Kesten and Zhang] to non-planar graphs.

math.PR↗

arXiv2017

Improved Consistent Weighted Sampling Revisited

Min-Hash is a popular technique for efficiently estimating the Jaccard similarity of binary sets. Consistent Weighted Sampling (CWS) generalizes the Min-Hash scheme to sketch weighted sets and has drawn increasing interest from the community. Due to its constant-time complexity independent of the values of the weights, Improved CWS (ICWS) is considered as the state-of-the-art CWS algorithm. In this paper, we revisit ICWS and analyze its underlying mechanism to show that there actually exists dependence between the two components of the hash-code produced by ICWS, which violates the condition of independence. To remedy the problem, we propose an Improved ICWS (I$^2$CWS) algorithm which not only shares the same theoretical computational complexity as ICWS but also abides by the required conditions of the CWS scheme. The experimental results on a number of synthetic data sets and real-world text data sets demonstrate that our I$^2$CWS algorithm can estimate the Jaccard similarity more accurately, and also compete with or outperform the compared methods, including ICWS, in classification and top-$K$ retrieval, after relieving the underlying dependence.

cs.DS↗

arXiv2017

Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots

We study response selection for multi-turn conversation in retrieval-based chatbots. Existing work either concatenates utterances in context or matches a response with a highly abstract context vector finally, which may lose relationships among utterances or important contextual information. We propose a sequential matching network (SMN) to address both problems. SMN first matches a response with each utterance in the context on multiple levels of granularity, and distills important matching information from each pair as a vector with convolution and pooling operations. The vectors are then accumulated in a chronological order through a recurrent neural network (RNN) which models relationships among utterances. The final matching score is calculated with the hidden states of the RNN. An empirical study on two public data sets shows that SMN can significantly outperform state-of-the-art methods for response selection in multi-turn conversation.

cs.CL↗

arXiv2017

Quantum Zeno and anti-Zeno effects in quantum dissipative systems

We investigate the quantum Zeno and anti-Zeno effects in quantum dissipative systems by employing a hierarchical equations of motion approach which is beyond the usual Markovian approximation, the rotating wave approximation, and the perturbative approximation. The quantum Zeno and anti-Zeno dynamics of a biased qubit-boson model and a biased qutrit-boson model are provided as illustrative examples. It is found that (i) there exists multiple Zeno-anti-Zeno crossover phenomena, (ii) the non-Markovian characteristic of the bath may be favorable for the accessibility of the Zeno dynamics, and (iii) high bath temperature may add the difficulty in observing the quantum Zeno effect in quantum dissipative systems.

quant-ph↗

arXiv2017

Efficient Communications in Training Large Scale Neural Networks

We consider the problem of how to reduce the cost of communication that is required for the parallel training of a neural network. The state-of-the-art method, Bulk Synchronous Parallel Stochastic Gradient Descent (BSP-SGD), requires many collective communication operations, like broadcasts of parameters or reductions for sub-gradient aggregations, which for large messages quickly dominates overall execution time and limits parallel scalability. To address this problem, we develop a new technique for collective operations, referred to as Linear Pipelining (LP). It is tuned to the message sizes that arise in BSP-SGD, and works effectively on multi-GPU systems. Theoretically, the cost of LP is invariant to $P$, where $P$ is the number of GPUs, while the cost of more conventional Minimum Spanning Tree (MST) scales like $O(\log P)$. LP also demonstrate up to 2x faster bandwidth than Bidirectional Exchange (BE) techniques that are widely adopted by current MPI implementations. We apply these collectives to BSP-SGD, showing that the proposed implementations reduce communication bottlenecks in practice while preserving the attractive convergence properties of BSP-SGD.

cs.DC↗

arXiv2017

A class of generalized positive linear maps on matrix algebras

We construct a class of positive linear maps on matrix algebras. We find conditions when these maps are atomic, decomposable and completely positive. We obtain a large class of atomic positive linear maps. As applications in quantum information theory, we discuss the structural physical approximation and optimality of entanglement witness associated with these maps.

math.OA↗

arXiv2017

The first result on 76Ge neutrinoless double beta decay from CDEX-1 experiment

We report the first result on Ge-76 neutrinoless double beta decay from CDEX-1 experiment at China Jinping Underground Laboratory. A mass of 994 g p-type point-contact high purity germanium detector has been installed to search the neutrinoless double beta decay events, as well as to directly detect dark matter particles. An exposure of 304 kg*day has been analyzed. The wideband spectrum from 500 keV to 3 MeV was obtained and the average event rate at the 2.039 MeV energy range is about 0.012 count per keV per kg per day. The half-life of Ge-76 neutrinoless double beta decay has been derived based on this result as: T 1/2 > 6.4*10^22 yr (90% C.L.). An upper limit on the effective Majorana-neutrino mass of 5.0 eV has been achieved. The possible methods to further decrease the background level have been discussed and will be pursued in the next stage of CDEX experiment.

hep-ex↗