arXiv subjects

Li Zhang

Publications and source records attributed to Li Zhang.

At least 865 records · Page 48Linked to original sources

arXiv2018

Learning to Compare: Relation Network for Few-Shot Learning

We present a conceptually simple, flexible, and general framework for few-shot learning, where a classifier must learn to recognise new classes given only few examples from each. Our method, called the Relation Network (RN), is trained end-to-end from scratch. During meta-learning, it learns to learn a deep distance metric to compare a small number of images within episodes, each of which is designed to simulate the few-shot setting. Once trained, a RN is able to classify images of new classes by computing relation scores between query images and the few examples of each new class without further updating the network. Besides providing improved performance on few-shot learning, our framework is easily extended to zero-shot learning. Extensive experiments on five benchmarks demonstrate that our simple approach provides a unified and effective approach for both of these two tasks.

cs.CV↗

arXiv2018

Analysis of micro-fluidic tweezers in the Stokes regime

Nanowire fluidic tweezers have been developed to gently and accurately capture, manipulate and deliver micro objects. The mechanism behind the capture and release process has not yet been well explained. Utilizing the method of regularized Stokeslet, we study a cylindrical nanowire tumbling and interacting with spherical particles in the Stokes regime. The capture phenomenon observed in experiments is reproduced and illustrated with the trajectories of micro-spheres and fluid tracers. The flow structure and the region of capture are quantitatively examined and compared for different sizes of particles, various tumbling rates, and dimensions of the tweezers. We find that pure kinematic effects can explain the mechanism of capture and transport of particles. We further reveal the relation between the capture region and stagnation points in the displacement field , i.e., the displacement for tracer particles in the moving frame within one rotation of the wire.

physics.flu-dyn↗

arXiv2018

Learning Differentially Private Recurrent Language Models

We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy. Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent. In particular, we add user-level privacy protection to the federated averaging algorithm, which makes "large step" updates from user-level data. Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work. We find that our private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset.

cs.LG↗

arXiv2018

Algorithmic Collusion in Cournot Duopoly Market: Evidence from Experimental Economics

Algorithmic collusion is an emerging concept in current artificial intelligence age. Whether algorithmic collusion is a creditable threat remains as an argument. In this paper, we propose an algorithm which can extort its human rival to collude in a Cournot duopoly competing market. In experiments, we show that, the algorithm can successfully extorted its human rival and gets higher profit in long run, meanwhile the human rival will fully collude with the algorithm. As a result, the social welfare declines rapidly and stably. Both in theory and in experiment, our work confirms that, algorithmic collusion can be a creditable threat. In application, we hope, the frameworks, the algorithm design as well as the experiment environment illustrated in this work, can be an incubator or a test bed for researchers and policymakers to handle the emerging algorithmic collusion.

econ.EM↗

arXiv2018

Dynamics of convective carbon dioxide dissolution in a closed porous media system

Motivated by geological carbon dioxide (CO$_2$) storage, many recent studies have investigated the fluid dynamics of solutal convection in porous media. Here we study the convective dissolution of CO$_2$ in a closed system, where the pressure in the gas declines as convection proceeds. This introduces a negative feedback that reduces the convective dissolution rate even before the brine becomes saturated. We analyse the case of an ideal gas with a solubility given by Henry's law, in the limits of very low and very high Rayleigh numbers. The equilibrium state in this system is determined by the dimensionless dissolution capacity, $Π$, which gives the fraction of the gas that can be dissolved into the underlying brine. Analytic approximations of the pure diffusion problem with $Π>0$, show that the diffusive base state is no longer self-similar and that diffusive mass transfer declines rapidly with time. Direct numerical simulations at high Rayleigh numbers show that no constant flux regime exists for $Π> 0$; nevertheless, the quantity $F/C_s^2$ remains constant, where $F$ is the dissolution flux and $C_s$ is the dissolved concentration at the top of the domain. Simple mathematical models are developed to predict the evolution of $C_s$ and $F$ for high-Rayleigh-number convection in a closed system. The negative feedback that limits convection in closed systems may explain the persistence of natural CO$_2$ accumulations over millennial timescales.

physics.flu-dyn↗

arXiv2017

Sparse Wavelet Estimation in Quantile Regression with Multiple Functional Predictors

In this manuscript, we study quantile regression in partial functional linear model where response is scalar and predictors include both scalars and multiple functions. Wavelet basis are adopted to better approximate functional slopes while effectively detect local features. The sparse group lasso penalty is imposed to select important functional predictors while capture shared information among them. The estimation problem can be reformulated into a standard second-order cone program and then solved by an interior point method. We also give a novel algorithm by using alternating direction method of multipliers (ADMM) which was recently employed by many researchers in solving penalized quantile regression problems. The asymptotic properties such as the convergence rate and prediction error bound have been established. Simulations and a real data from ADHD-200 fMRI data are investigated to show the superiority of our proposed method.

math.ST↗

arXiv2017

Actor-Critic Sequence Training for Image Captioning

Generating natural language descriptions of images is an important capability for a robot or other visual-intelligence driven AI agent that may need to communicate with human users about what it is seeing. Such image captioning methods are typically trained by maximising the likelihood of ground-truth annotated caption given the image. While simple and easy to implement, this approach does not directly maximise the language quality metrics we care about such as CIDEr. In this paper we investigate training image captioning methods based on actor-critic reinforcement learning in order to directly optimise non-differentiable quality metrics of interest. By formulating a per-token advantage and value computation strategy in this novel reinforcement learning based captioning model, we show that it is possible to achieve the state of the art performance on the widely used MSCOCO benchmark.

cs.CV↗

arXiv2017

An explanation of the formation of the peculiar periphery of Tycho's supernova remnant

Tycho's supernova remnant (SNR) has a periphery which clearly deviates from spherical shape bash on the X-ray and radio observations. The forward shock from the southeast to the north of the remnant has a relatively deformed outline with a depression in the east, although in the west it is generally round and smooth. Moreover, at some locations of the shell, the supernova ejecta distributes closely to the forward shock to produce protrusions. Using 3D hydrodynamical simulation, the dynamical evolution of the supernova ejecta in an inhomogeneous medium and the formation process of the profile of the forward shock can be studied. To reproduce the peculiar periphery of the remnant, we propose a model in which the supernova ejecta has evolved in a cavity blown by a latitude-dependent outflow. The results indicate that, with the assumptions of the wind bubble driven by the anisotropic wind with a mass loss rate of $\sim10^{-7} {\rm M_{\odot}}$~yr$^{-1}$, a pole velocity of $\sim 100$~km s$^{-1}$, a duration of $\sim10^5$~yr just before the supernova explosion, and a spatial velocity of $\sim 30$~km s$^{-1}$ of the progenitor with respect to the circumstellar medium, the depression to the east and the protrusion to the southeast on the observed periphery of the remnant can be generally reproduced. In conclusion, an explanation to the peculiar shape of the periphery of Tycho's SNR is the supernova ejecta evolved in the cavity driven by the latitude-dependent wind.

astro-ph.HE↗

arXiv2017

GaDei: On Scale-up Training As A Service For Deep Learning

Deep learning (DL) training-as-a-service (TaaS) is an important emerging industrial workload. The unique challenge of TaaS is that it must satisfy a wide range of customers who have no experience and resources to tune DL hyper-parameters, and meticulous tuning for each user's dataset is prohibitively expensive. Therefore, TaaS hyper-parameters must be fixed with values that are applicable to all users. IBM Watson Natural Language Classifier (NLC) service, the most popular IBM cognitive service used by thousands of enterprise-level clients around the globe, is a typical TaaS service. By evaluating the NLC workloads, we show that only the conservative hyper-parameter setup (e.g., small mini-batch size and small learning rate) can guarantee acceptable model accuracy for a wide range of customers. We further justify theoretically why such a setup guarantees better model convergence in general. Unfortunately, the small mini-batch size causes a high volume of communication traffic in a parameter-server based system. We characterize the high communication bandwidth requirement of TaaS using representative industrial deep learning workloads and demonstrate that none of the state-of-the-art scale-up or scale-out solutions can satisfy such a requirement. We then present GaDei, an optimized shared-memory based scale-up parameter server design. We prove that the designed protocol is deadlock-free and it processes each gradient exactly once. Our implementation is evaluated on both commercial benchmarks and public benchmarks to demonstrate that it significantly outperforms the state-of-the-art parameter-server based implementation while maintaining the required accuracy and our implementation reaches near the best possible runtime performance, constrained only by the hardware limitation. Furthermore, to the best of our knowledge, GaDei is the only scale-up DL system that provides fault-tolerance.

stat.ML↗

arXiv2017

IBM Deep Learning Service

Deep learning driven by large neural network models is overtaking traditional machine learning methods for understanding unstructured and perceptual data domains such as speech, text, and vision. At the same time, the "as-a-Service"-based business model on the cloud is fundamentally transforming the information technology industry. These two trends: deep learning, and "as-a-service" are colliding to give rise to a new business model for cognitive application delivery: deep learning as a service in the cloud. In this paper, we will discuss the details of the software architecture behind IBM's deep learning as a service (DLaaS). DLaaS provides developers the flexibility to use popular deep learning libraries such as Caffe, Torch and TensorFlow, in the cloud in a scalable and resilient manner with minimal effort. The platform uses a distribution and orchestration layer that facilitates learning from a large amount of data in a reasonable amount of time across compute nodes. A resource provisioning layer enables flexible job management on heterogeneous resources, such as graphics processing units (GPUs) and central processing units (CPUs), in an infrastructure as a service (IaaS) cloud.

cs.DC↗

arXiv2017

On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches

The recent, remarkable growth of machine learning has led to intense interest in the privacy of the data on which machine learning relies, and to new techniques for preserving privacy. However, older ideas about privacy may well remain valid and useful. This note reviews two recent works on privacy in the light of the wisdom of some of the early literature, in particular the principles distilled by Saltzer and Schroeder in the 1970s.

stat.ML↗

arXiv2017

Motion Feature Augmented Recurrent Neural Network for Skeleton-based Dynamic Hand Gesture Recognition

Dynamic hand gesture recognition has attracted increasing interests because of its importance for human computer interaction. In this paper, we propose a new motion feature augmented recurrent neural network for skeleton-based dynamic hand gesture recognition. Finger motion features are extracted to describe finger movements and global motion features are utilized to represent the global movement of hand skeleton. These motion features are then fed into a bidirectional recurrent neural network (RNN) along with the skeleton sequence, which can augment the motion features for RNN and improve the classification performance. Experiments demonstrate that our proposed method is effective and outperforms start-of-the-art methods.

cs.CV↗

arXiv2017

A possible explanation of the knee of cosmic light component spectrum from 100 TeV to 3 PeV

The mixed Hydrogen and Helium (H + He) spectrum with a clear steepening at $\sim 700$ TeV has been detected by ARGO-YBJ experiments. In this paper, we demonstrate that the observed H + He spectrum can be well reproduced with the model of cosmic rays escaping from the supernova remnants (SNRs) in our Galaxy. In this model, particles are accelerated in a SNR through a non-linear diffusive shock acceleration mechanism and three components of high energy light nuclei escaped from the SNR are considered. It should be noted that the proton spectrum observed by KASCADE can be also explained by this model given a higher acceleration efficiency.

astro-ph.HE↗

arXiv2017

Topological magnon bound-states in periodically modulated Heisenberg XXZ chains

Strongly interacting topological states in multi-particle quantum systems pose great challenges to both theory and experiment. Recently, bound states of elementary spin waves (magnons) in quantum magnets have been experimentally observed in quantum Heisenberg chains comprising ultracold Bose atoms in optical lattices. Here, we explore a strongly interacting topological state called topological magnon bound-state in the quantum Heisenberg chain under cotranslational symmetry. We find that the cotranslational symmetry is the key to the definition of a topological invariant for multi-particle quantum states, which enables us to characterize the topological features of multi-magnon excitations. We calculate energy spectra, density distributions, correlations and Chern numbers of the two-magnon bound-states and show the existence of topological protected edge bound-states. Our study not only opens a new prospect to pursue topological magnon bound-states, but also gives insights into the characterization and understanding of strongly interacting topological states.

quant-ph↗

arXiv2017

TAPAS: Two-pass Approximate Adaptive Sampling for Softmax

TAPAS is a novel adaptive sampling method for the softmax model. It uses a two pass sampling strategy where the examples used to approximate the gradient of the partition function are first sampled according to a squashed population distribution and then resampled adaptively using the context and current model. We describe an efficient distributed implementation of TAPAS. We show, on both synthetic data and a large real dataset, that TAPAS has low computational overhead and works well for minimizing the rank loss for multi-class classification problems with a very large label space.

cs.LG↗

arXiv2017

A $γ$-ray Quasi-Periodic modulation in the Blazar PKS 0301$-$243?

We report a nominally high-confidence $γ$-ray quasi-periodic modulation in the blazar PKS 0301$-$243. For this target, we analyze its \emph{Fermi}-LAT Pass 8 data covering from 2008 August to 2017 May. Two techniques, i.e., the maximum likelihood optimization and the exposure-weighted aperture photometry, are used to build the $γ$-ray light curves. Then both the Lomb-Scargle Periodogram and the Weighted Wavelet Z-transform are applied to the light curves to search for period signals. A quasi-periodicity with a period of $2.1\pm0.3$ yr appears at the significance level of $\sim5σ$, although it should be noted that this putative quasi-period variability is seen in a data set barely four times longer. We speculate that this $γ$-ray quasi-periodic modulation might be evidence of a binary supermassive black hole.

astro-ph.HE↗

arXiv2017

Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation

In this paper, we propose an alternative method to estimate room layouts of cluttered indoor scenes. This method enjoys the benefits of two novel techniques. The first one is semantic transfer (ST), which is: (1) a formulation to integrate the relationship between scene clutter and room layout into convolutional neural networks; (2) an architecture that can be end-to-end trained; (3) a practical strategy to initialize weights for very deep networks under unbalanced training data distribution. ST allows us to extract highly robust features under various circumstances, and in order to address the computation redundance hidden in these features we develop a principled and efficient inference scheme named physics inspired optimization (PIO). PIO's basic idea is to formulate some phenomena observed in ST features into mechanics concepts. Evaluations on public datasets LSUN and Hedau show that the proposed method is more accurate than state-of-the-art methods.

cs.CV↗

arXiv2017

Ultra-thin, High-efficiency Mid-Infrared Transmissive Huygens Meta-Optics

The mid-infrared (mid-IR) is a strategically important band for numerous applications ranging from night vision to biochemical sensing. Unlike visible or near-infrared optical parts which are commonplace and economically available off-the-shelf, mid-IR optics often requires exotic materials or complicated processing, which accounts for their high cost and inferior quality compared to their visible or near-infrared counterparts. Here we theoretically analyzed and experimentally realized a Huygens metasurface platform capable of fulfilling a diverse cross-section of optical functions in the mid-IR. The meta-optical elements were constructed using high-index chalcogenide films deposited on fluoride substrates:the choices of wide-band transparent materials allow the design to be scaled across a broad infrared spectrum. Capitalizing on a novel two-component Huygens' meta-atom design, the meta-optical devices feature an ultra-thin profile ($λ_0/8$ in thickness, where $λ_0$ is the free-space wavelength) and measured optical efficiencies up to 75% in transmissive mode, both of which represent major improvements over state-of-the-art. We have also demonstrated, for the first time, mid-IR transmissive meta-lenses with diffraction-limited focusing and imaging performance. The projected size, weight and power advantages, coupled with the manufacturing scalability leveraging standard microfabrication technologies, make the Huygens meta-optical devices promising for next-generation mid-IR system applications.

physics.optics↗