arXiv subjects

Xin Wang

Publications and source records attributed to Xin Wang.

At least 1,513 records · Page 84Linked to original sources

arXiv2020

How fine can fine-tuning be? Learning efficient language models

State-of-the-art performance on language understanding tasks is now achieved with increasingly large networks; the current record holder has billions of parameters. Given a language model pre-trained on massive unlabeled text corpora, only very light supervised fine-tuning is needed to learn a task: the number of fine-tuning steps is typically five orders of magnitude lower than the total parameter count. Does this mean that fine-tuning only introduces small differences from the pre-trained model in the parameter space? If so, can one avoid storing and computing an entire model for each task? In this work, we address these questions by using Bidirectional Encoder Representations from Transformers (BERT) as an example. As expected, we find that the fine-tuned models are close in parameter space to the pre-trained one, with the closeness varying from layer to layer. We show that it suffices to fine-tune only the most critical layers. Further, we find that there are surprisingly many good solutions in the set of sparsified versions of the pre-trained model. As a result, fine-tuning of huge language models can be achieved by simply setting a certain number of entries in certain layers of the pre-trained parameters to zero, saving both task-specific parameter storage and computational cost.

cs.CL↗

arXiv2020

Energy Management and Trajectory Optimization for UAV-Enabled Legitimate Monitoring Systems

Thanks to their quick placement and high flexibility, unmanned aerial vehicles (UAVs) can be very useful in the current and future wireless communication systems. With a growing number of smart devices and infrastructure-free communication networks, it is necessary to legitimately monitor these networks to prevent crimes. In this paper, a novel framework is proposed to exploit the flexibility of the UAV for legitimate monitoring via joint trajectory design and energy management. The system includes a suspicious transmission link with a terrestrial transmitter and a terrestrial receiver, and a UAV to monitor the suspicious link. The UAV can adjust its positions and send jamming signal to the suspicious receiver to ensure successful eavesdropping. Based on this model, we first develop an approach to minimize the overall jamming energy consumption of the UAV. Building on a judicious (re-)formulation, an alternating optimization approach is developed to compute a locally optimal solution in polynomial time. Furthermore, we model and include the propulsion power to minimize the overall energy consumption of the UAV. Leveraging the successive convex approximation method, an effective iterative approach is developed to find a feasible solution fulfilling the Karush-Kuhn-Tucker (KKT) conditions. Extensive numerical results are provided to verify the merits of the proposed schemes.

eess.SP↗

arXiv2020

Resonant Beam Communications with Photovoltaic Receiver for Optical Data and Power Transfer

The vision and requirements of the sixth generation (6G) mobile communication systems are expected to adopt freespace optical communication (FSO) and wireless power transfer (WPT). The laser-based WPT or wireless information transfer (WIT) usually faces the challenges of mobility and safety. We present a mobile and safe resonant beam communication (RBCom) system, which can realize high-rate simultaneous wireless information and power transfer (SWIPT). We propose an analytical model to depict its carrier beam and information transfer procedures. The numerical results show that RBCom can achieve more than 40 mW charging power and 1:6 Gbit/s channel capacity with orthogonal frequency division multiplexing (OFDM) scheme, which can be applied in future scenario where power and high-rate data are simultaneously desired.

eess.SP↗

arXiv2020

Time-dependent Hamiltonian simulation with $L^1$-norm scaling

The difficulty of simulating quantum dynamics depends on the norm of the Hamiltonian. When the Hamiltonian varies with time, the simulation complexity should only depend on this quantity instantaneously. We develop quantum simulation algorithms that exploit this intuition. For sparse Hamiltonian simulation, the gate complexity scales with the $L^1$ norm $\int_{0}^{t}\mathrm{d}τ\left\lVert H(τ)\right\lVert_{\max}$, whereas the best previous results scale with $t\max_{τ\in[0,t]}\left\lVert H(τ)\right\lVert_{\max}$. We also show analogous results for Hamiltonians that are linear combinations of unitaries. Our approaches thus provide an improvement over previous simulation algorithms that can be substantial when the Hamiltonian varies significantly. We introduce two new techniques: a classical sampler of time-dependent Hamiltonians and a rescaling principle for the Schrödinger equation. The rescaled Dyson-series algorithm is nearly optimal with respect to all parameters of interest, whereas the sampling-based approach is easier to realize for near-term simulation. These algorithms could potentially be applied to semi-classical simulations of scattering processes in quantum chemistry.

quant-ph↗

arXiv2020

Channel-Dependent Scheduling in Wireless Energy Transfer for Mobile Devices

Resonant Beam Charging (RBC) is the Wireless Power Transfer (WPT) technology, which can provide high-power, long-distance, mobile, and safe wireless charging for Internet of Things (IoT) devices. Supporting multiple IoT devices charging simultaneously is a significant feature of the RBC system. To optimize the multi-user charging performance, the transmitting power should be scheduled for charging all IoT devices simultaneously. In order to keep all IoT devices working as long as possible for fairness, we propose the First Access First Charge (FAFC) scheduling algorithm. Then, we formulate the scheduling parameters quantitatively for algorithm implementation. Finally, we analyze the performance of FAFC scheduling algorithm considering the impacts of the receiver number, the transmitting power and the charging time. Based on the analysis, we summarize the methods of improving the WPT performance for multiple IoT devices, which include limiting the receiver number, increasing the transmitting power, prolonging the charging time and improving the single-user's charging efficiency. The FAFC scheduling algorithm design and analysis provide a fair WPT solution for the multi-user RBC system.

cs.NI↗

arXiv2020

Public discourse and social network echo chambers driven by socio-cognitive biases

In recent years, social media has increasingly become an important platform for political campaigns, especially elections. It remains elusive how exactly public discourse is driven by the intricate interplay between individual socio-cognitive biases, dueling campaign efforts, and social media platforms. We examine this complex socio-political process by integrating observed retweet networks from the 2016 political networks with an agent-based model of political opinion formation and network structure. Here we show that the range of political viewpoints individuals are willing to consider is a key determinant in the formation of polarized networks and the emergence of echo chambers. We also find that winning majority support in public discourse is determined by both the effort exerted by campaigns and the relative ideological positioning of opposing campaigns. Our results demonstrate how public discourse and political polarization can be modeled as an interactive process of shifting individual opinions, evolving social networks, and political campaigns.

physics.soc-ph↗

arXiv2020

Using Cyclic Noise as the Source Signal for Neural Source-Filter-based Speech Waveform Model

Neural source-filter (NSF) waveform models generate speech waveforms by morphing sine-based source signals through dilated convolution in the time domain. Although the sine-based source signals help the NSF models to produce voiced sounds with specified pitch, the sine shape may constrain the generated waveform when the target voiced sounds are less periodic. In this paper, we propose a more flexible source signal called cyclic noise, a quasi-periodic noise sequence given by the convolution of a pulse train and a static random noise with a trainable decaying rate that controls the signal shape. We further propose a masked spectral loss to guide the NSF models to produce periodic voiced sounds from the cyclic noise-based source signal. Results from a large-scale listening test demonstrated the effectiveness of the cyclic noise and the masked spectral loss on speaker-independent NSF models in copy-synthesis experiments on the CMU ARCTIC database.

eess.AS↗

arXiv2020

Duplication of Windows Services

OS-level virtualization techniques virtualize system resources at the system call interface, has the distinct advantage of smaller run-time resource requirements as compared to HAL-level virtualization techniques, and thus forms an important building block for virtualizing parallel and distributed applications such as a HPC clusters. Because the Windows operating system puts certain critical functionalities in privileged user-level system service processes, a complete OS-level virtualization solution for the Windows platform requires duplication of such Windows service as Remote Procedure Call Server Service (RPCSS). As many implementation details of the Windows system services are proprietary, duplicating Windows system services becomes the key technical challenge for virtualizing the Windows platform at the OS level. Moreover, as a core component of cloud computing, IIS web server-related services need to be duplicated in containers (i.e., OS-level virtual machines), but so far there is no such scheme. In this paper, we thoroughly identify all issues that affect service duplication, and then propose the first known methodology to systematically duplicate both system and ordinary Windows services. Our experiments show that the methodology can duplicate a set of system and ordinary services on different versions of Windows OS.

cs.OS↗

arXiv2020

BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue.

cs.CV↗

arXiv2020

Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation

Visual navigation is a task of training an embodied agent by intelligently navigating to a target object (e.g., television) using only visual observations. A key challenge for current deep reinforcement learning models lies in the requirements for a large amount of training data. It is exceedingly expensive to construct sufficient 3D synthetic environments annotated with the target object information. In this paper, we focus on visual navigation in the low-resource setting, where we have only a few training environments annotated with object information. We propose a novel unsupervised reinforcement learning approach to learn transferable meta-skills (e.g., bypass obstacles, go straight) from unannotated environments without any supervisory signals. The agent can then fast adapt to visual navigation through learning a high-level master policy to combine these meta-skills, when the visual-navigation-specified reward is provided. Evaluation in the AI2-THOR environments shows that our method significantly outperforms the baseline by 53.34% relatively on SPL, and further qualitative analysis demonstrates that our method learns transferable motor primitives for visual navigation.

cs.CV↗

arXiv2020

Generating Synthetic Magnetism via Floquet Engineering Auxiliary Qubits in Phonon-Cavity-Based Lattice

Gauge magnetic fields have a close relation to breaking time-reversal symmetry in condensed matter. In the present of the gauge fields, we might observe nonreciprocal and topological transport. Inspired by these, there is a growing effort to realize exotic transport phenomena in optical and acoustic systems. However, due to charge neutrality, realizing analog magnetic flux for phonons in nanoscale systems is still challenging in both theoretical and experimental studies. Here we propose a novel mechanism to generate synthetic magnetic field for phonon lattice by Floquet engineering auxiliary qubits. We find that, a longitudinal Floquet drive on the qubit will produce a resonant coupling between two detuned acoustic cavities. Specially, the phase encoded into the longitudinal drive can exactly be transformed into the phonon-phonon hopping. Our proposal is general and can be realized in various types of artificial hybrid quantum systems. Moreover, by taking surface-acoustic-wave (SAW) cavities for example, we propose how to generate synthetic magnetic flux for phonon transport. In the present of synthetic magnetic flux, the time-reversal symmetry will be broken, which allows to realize the circulator transport and analog Aharonov-Bohm effects for acoustic waves. Last, we demonstrate that our proposal can be scaled to simulate topological states of matter in quantum acoustodynamics system.

quant-ph↗

arXiv2020

Forecast for FAST: from Galaxies Survey to Intensity Mapping

The Five-Hundred-Meter Aperture Spherical Radio Telescope(FAST) is the largest single-dish radio telescope in the world. In this paper, we make forecast on the FAST HI large scale structure survey by mock observations. We consider a drift scan survey with the L-band 19 beam receiver, which may be commensal with the pulsar search and Galactic HI survey. We also consider surveys at lower frequency, either using the current single feed wide band receiver, or a future multi-beam phased array feed (PAF) in the UHF band. We estimate the number density of detected HI galaxies and the measurement error in positions, the precision of the surveys are evaluated using both Fisher matrix and simulated observations. The measurement error in the HI galaxy power spectrum is estimated, and we find that the error is relatively large even at moderate redshifts, as the number of positively detected galaxies drops drastically with increasing redshift. However, good cosmological measurement could be obtained with the intensity mapping technique where the large scale HI distribution is measured without resolving individual galaxies. The figure of merit (FoM) for the dark energy equation of state with different observation times are estimated, we find that with the existing L-band multi-beam receiver, a good measurement of low redshift large scale structure can be obtained, which complements the existing optical surveys. With a PAF in the UHF band, the constraint can be much stronger, reaching the level of a dark energy task force (DETF) stage IV experiment.

astro-ph.CO↗

arXiv2020

Bridge the Domain Gap Between Ultra-wide-field and Traditional Fundus Images via Adversarial Domain Adaptation

For decades, advances in retinal imaging technology have enabled effective diagnosis and management of retinal disease using fundus cameras. Recently, ultra-wide-field (UWF) fundus imaging by Optos camera is gradually put into use because of its broader insights on fundus for some lesions that are not typically seen in traditional fundus images. Research on traditional fundus images is an active topic but studies on UWF fundus images are few. One of the most important reasons is that UWF fundus images are hard to obtain. In this paper, for the first time, we explore domain adaptation from the traditional fundus to UWF fundus images. We propose a flexible framework to bridge the domain gap between two domains and co-train a UWF fundus diagnosis model by pseudo-labelling and adversarial learning. We design a regularisation technique to regulate the domain adaptation. Also, we apply MixUp to overcome the over-fitting issue from incorrect generated pseudo-labels. Our experimental results on either single or both domains demonstrate that the proposed method can well adapt and transfer the knowledge from traditional fundus images to UWF fundus images and improve the performance of retinal disease recognition.

eess.IV↗

arXiv2020

Task-Aware Feature Generation for Zero-Shot Compositional Learning

Visual concepts (e.g., red apple, big elephant) are often semantically compositional and each element of the compositions can be reused to construct novel concepts (e.g., red elephant). Compositional feature synthesis, which generates image feature distributions exploiting the semantic compositionality, is a promising approach to sample-efficient model generalization. In this work, we propose a task-aware feature generation (TFG) framework for compositional learning, which generates features of novel visual concepts by transferring knowledge from previously seen concepts. These synthetic features are then used to train a classifier to recognize novel concepts in a zero-shot manner. Our novel TFG design injects task-conditioned noise layer-by-layer, producing task-relevant variation at each level. We find the proposed generator design improves classification accuracy and sample efficiency. Our model establishes a new state of the art on three zero-shot compositional learning (ZSCL) benchmarks, outperforming the previous discriminative models by a large margin. Our model improves the performance of the prior arts by over 2x in the generalized ZSCL setting.

cs.CV↗

arXiv2020

Influence of Laser Intensity Fluctuation on Single-Cesium Atom Trapping Lifetime in a 1064-nm Microscopic Optical Tweezer

An optical tweezer composed of a strongly focused single-spatial-mode Gaussian beam of a red-detuned 1064-nm laser can confine a single-cesium (Cs) atom at the strongest point of the light intensity. We can use this for coherent manipulation of single-quantum bits and single-photon sources. The trapping lifetime of the atoms in the optical tweezers is very short due to the impact of the background atoms, the laser intensity fluctuation of optical tweezer and the residual thermal motion of the atoms. In this paper, we analyzed the influence of the background pressure, the trap frequency of optical tweezers and the parametric heating of the optical tweezer on the atomic trapping lifetime. Combined with the external feedback loop based on an acousto-optical modulator (AOM), the intensity fluctuation of the 1064-nm laser in the time domain was suppressed from $\pm$ 3.360$\%$ to $\pm$ 0.064$\%$, and the suppression bandwidth reached approximately 33 kHz. The trapping lifetime of a single Cs atom in the microscopic optical tweezer was extended from 4.04 s to 6.34 s.

physics.atom-ph↗

arXiv2020

Frustratingly Simple Few-Shot Object Detection

Detecting rare objects from a few examples is an emerging problem. Prior works show meta-learning is a promising approach. But, fine-tuning techniques have drawn scant attention. We find that fine-tuning only the last layer of existing detectors on rare classes is crucial to the few-shot object detection task. Such a simple approach outperforms the meta-learning methods by roughly 2~20 points on current benchmarks and sometimes even doubles the accuracy of the prior methods. However, the high variance in the few samples often leads to the unreliability of existing benchmarks. We revise the evaluation protocols by sampling multiple groups of training examples to obtain stable comparisons and build new benchmarks based on three datasets: PASCAL VOC, COCO and LVIS. Again, our fine-tuning approach establishes a new state of the art on the revised benchmarks. The code as well as the pretrained models are available at https://github.com/ucbdrive/few-shot-object-detection.

cs.CV↗

arXiv2020

Interpretable CNNs for Object Classification

This paper proposes a generic method to learn interpretable convolutional filters in a deep convolutional neural network (CNN) for object classification, where each interpretable filter encodes features of a specific object part. Our method does not require additional annotations of object parts or textures for supervision. Instead, we use the same training data as traditional CNNs. Our method automatically assigns each interpretable filter in a high conv-layer with an object part of a certain category during the learning process. Such explicit knowledge representations in conv-layers of CNN help people clarify the logic encoded in the CNN, i.e., answering what patterns the CNN extracts from an input image and uses for prediction. We have tested our method using different benchmark CNNs with various structures to demonstrate the broad applicability of our method. Experiments have shown that our interpretable filters are much more semantically meaningful than traditional filters.

cs.LG↗

arXiv2020

Efficiently computable bounds for magic state distillation

Magic-state distillation (or non-stabilizer state manipulation) is a crucial component in the leading approaches to realizing scalable, fault-tolerant, and universal quantum computation. Related to non-stabilizer state manipulation is the resource theory of non-stabilizer states, for which one of the goals is to characterize and quantify non-stabilizerness of a quantum state. In this paper, we introduce the family of thauma measures to quantify the amount of non-stabilizerness in a quantum state, and we exploit this family of measures to address several open questions in the resource theory of non-stabilizer states. As a first application, we establish the hypothesis testing thauma as an efficiently computable benchmark for the one-shot distillable non-stabilizerness, which in turn leads to a variety of bounds on the rate at which non-stabilizerness can be distilled, as well as on the overhead of magic-state distillation. We then prove that the max-thauma can be used as an efficiently computable tool in benchmarking the efficiency of magic-state distillation and that it can outperform pervious approaches based on mana. Finally, we use the min-thauma to bound a quantity known in the literature as the "regularized relative entropy of magic." As a consequence of this bound, we find that two classes of states with maximal mana, a previously established non-stabilizerness measure, cannot be interconverted in the asymptotic regime at a rate equal to one. This result resolves a basic question in the resource theory of non-stabilizer states and reveals a difference between the resource theory of non-stabilizer states and other resource theories such as entanglement and coherence.

quant-ph↗