arXiv subjects

Yu Wang

Publications and source records attributed to Yu Wang.

At least 1,135 records · Page 63Linked to original sources

arXiv2021

An Adversarial Learning based Multi-Step Spoken Language Understanding System through Human-Computer Interaction

Most of the existing spoken language understanding systems can perform only semantic frame parsing based on a single-round user query. They cannot take users' feedback to update/add/remove slot values through multiround interactions with users. In this paper, we introduce a novel multi-step spoken language understanding system based on adversarial learning that can leverage the multiround user's feedback to update slot values. We perform two experiments on the benchmark ATIS dataset and demonstrate that the new system can improve parsing performance by at least $2.5\%$ in terms of F1, with only one round of feedback. The improvement becomes even larger when the number of feedback rounds increases. Furthermore, we also compare the new system with state-of-the-art dialogue state tracking systems and demonstrate that the new interactive system can perform better on multiround spoken language understanding tasks in terms of slot- and sentence-level accuracy.

cs.CL↗

arXiv2021

A Coarse to Fine Question Answering System based on Reinforcement Learning

In this paper, we present a coarse to fine question answering (CFQA) system based on reinforcement learning which can efficiently processes documents with different lengths by choosing appropriate actions. The system is designed using an actor-critic based deep reinforcement learning model to achieve multi-step question answering. Compared to previous QA models targeting on datasets mainly containing either short or long documents, our multi-step coarse to fine model takes the merits from multiple system modules, which can handle both short and long documents. The system hence obtains a much better accuracy and faster trainings speed compared to the current state-of-the-art models. We test our model on four QA datasets, WIKEREADING, WIKIREADING LONG, CNN and SQuAD, and demonstrate 1.3$\%$-1.7$\%$ accuracy improvements with 1.5x-3.4x training speed-ups in comparison to the baselines using state-of-the-art models.

cs.CL↗

arXiv2021

Multi-messenger Detection Rates and distributions of Binary Neutron Star Mergers and Their Cosmological Implications

The gravitational-wave (GW) events, produced by the coalescence of binary neutron-stars (BNS), can be treated as the standard sirens to probe the expansion history of the Universe, if their redshifts could be determined from the electromagnetic observations. For the high-redshift ($z\gtrsim 0.1$) events, the short $γ$-ray bursts (sGRBs) and the afterglows are always considered as the primary electromagnetic counterparts. In this paper, by investigating various models of sGRBs and afterglows, we discuss the rates and distributions of BNS mergers' multi-messenger observations with GW detectors in second-generation (2G), 2.5G, 3G era with the detectable sGRBs and the afterglows. For instance, for Cosmic Explorer GW detector, the rate is about (300-3500) per year with GECAM-like detector for $γ$-ray emissions and LSST/WFST detector for optical afterglows. In addition, we find these events have the redshifts $z\lesssim 2$ and the inclination angles $ι\lesssim 20^{\circ}$. These results justify the rough estimation in previous works. Considering these events as standard sirens to constrain the equation-of-state parameters of dark energy $w_{0}$ and $w_{a}$, we obtain the potential constraints of $Δw_{0}\simeq 0.02-0.05$ and $Δw_{a}\simeq 0.1-0.4$.

astro-ph.HE↗

arXiv2021

SG-PALM: a Fast Physically Interpretable Tensor Graphical Model

We propose a new graphical model inference procedure, called SG-PALM, for learning conditional dependency structure of high-dimensional tensor-variate data. Unlike most other tensor graphical models the proposed model is interpretable and computationally scalable to high dimension. Physical interpretability follows from the Sylvester generative (SG) model on which SG-PALM is based: the model is exact for any observation process that is a solution of a partial differential equation of Poisson type. Scalability follows from the fast proximal alternating linearized minimization (PALM) procedure that SG-PALM uses during training. We establish that SG-PALM converges linearly (i.e., geometric convergence rate) to a global optimum of its objective function. We demonstrate the scalability and accuracy of SG-PALM for an important but challenging climate prediction problem: spatio-temporal forecasting of solar flares from multimodal imaging data.

stat.ML↗

arXiv2021

Unsupervised Remote Sensing Super-Resolution via Migration Image Prior

Recently, satellites with high temporal resolution have fostered wide attention in various practical applications. Due to limitations of bandwidth and hardware cost, however, the spatial resolution of such satellites is considerably low, largely limiting their potentials in scenarios that require spatially explicit information. To improve image resolution, numerous approaches based on training low-high resolution pairs have been proposed to address the super-resolution (SR) task. Despite their success, however, low/high spatial resolution pairs are usually difficult to obtain in satellites with a high temporal resolution, making such approaches in SR impractical to use. In this paper, we proposed a new unsupervised learning framework, called "MIP", which achieves SR tasks without low/high resolution image pairs. First, random noise maps are fed into a designed generative adversarial network (GAN) for reconstruction. Then, the proposed method converts the reference image to latent space as the migration image prior. Finally, we update the input noise via an implicit method, and further transfer the texture and structured information from the reference image. Extensive experimental results on the Draper dataset show that MIP achieves significant improvements over state-of-the-art methods both quantitatively and qualitatively. The proposed MIP is open-sourced at http://github.com/jiaming-wang/MIP.

cs.CV↗

arXiv2021

Integrated Communication and Navigation for Ultra-Dense LEO Satellite Networks: Vision, Challenges and Solutions

Next generation beyond 5G networks are expected to provide both Terabits per second data rate communication services and centimeter-level accuracy localization services in an efficient, seamless and cost-effective manner. However, most of the current communication and localization systems are separately designed, leading to an under-utilization of radio resources and network performance degradation. In this paper, we propose an integrated communication and navigation (ICAN) framework to fully unleash the potential of ultra-dense LEO satellite networks for optimal provisioning of differentiated services. The specific benefits, feasibility analysis and challenges for ICAN enabled satellite system are explicitly discussed. In particular, a novel beam hopping based ICAN satellite system solution is devised to adaptively tune the network beam layout for dual functional communication and positioning purposes. Furthermore, a thorough experimental platform is built following the Third Generation Partnership Project (3GPP) defined non-terrestrial network simulation parameters to validate the performance gain of the ICAN satellite system

eess.SP↗

arXiv2021

Domain-Specific Suppression for Adaptive Object Detection

Domain adaptation methods face performance degradation in object detection, as the complexity of tasks require more about the transferability of the model. We propose a new perspective on how CNN models gain the transferability, viewing the weights of a model as a series of motion patterns. The directions of weights, and the gradients, can be divided into domain-specific and domain-invariant parts, and the goal of domain adaptation is to concentrate on the domain-invariant direction while eliminating the disturbance from domain-specific one. Current UDA object detection methods view the two directions as a whole while optimizing, which will cause domain-invariant direction mismatch even if the output features are perfectly aligned. In this paper, we propose the domain-specific suppression, an exemplary and generalizable constraint to the original convolution gradients in backpropagation to detach the two parts of directions and suppress the domain-specific one. We further validate our theoretical analysis and methods on several domain adaptive object detection tasks, including weather, camera configuration, and synthetic to real-world adaptation. Our experiment results show significant advance over the state-of-the-art methods in the UDA object detection field, performing a promotion of $10.2\sim12.2\%$ mAP on all these domain adaptation scenarios.

cs.CV↗

arXiv2021

ATOMS:ALMA Three-millimeter Observations of Massive Star-forming regions -- III :Catalogues of candidate hot molecular cores and Hyper/Ultra compact HII regions

We have identified 453 compact dense cores in 3 mm continuum emission maps in the ATOMS (ALMA Three-millimeter Observations of Massive Star-forming regions) survey, and compiled three catalogues of high-mass star forming cores. One catalogue, referred to as H/UC-HII catalogue, includes 89 cores that enshroud hyper/ultra compact (H/UC) HII regions as characterized by associated compact H40alpha emission. A second catalogue, referred to as pure s-cHMC, includes 32 candidate Hot Molecular Cores (HMCs) showing rich spectra (N>20lines) of complex organic molecules (COMs) but not associated with H/UC-HII regions. The third catalogue, referred to as pure w-cHMC, includes 58 candidate HMCs with relatively low levels of COM richness and not associated with H/UC-HII regions. These three catalogues of dense cores provide an important foundation for future studies of the early stages of high-mass star formation across the Milky Way. We also find that nearly half of H/UC-HII cores are candidate HMCs. From the number counts of COM-containing and H/UC-HII cores, we suggest that the duration of high-mass protostellar cores showing chemically rich features is at least comparable to the lifetime of H/UC-HII regions. For cores in the H/UC-HII catalogue, the width of the H40alpha line increases as the core size decreases, suggesting that the non-thermal dynamical and/or pressure line-broadening mechanisms dominate on the smaller scales of the H/UC-HII cores.

astro-ph.GA↗

arXiv2021

ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing

Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models taken from NLP. These LMs reach for new prediction frontiers at low inference costs. Here, we trained two auto-regressive models (Transformer-XL, XLNet) and four auto-encoder models (BERT, Albert, Electra, T5) on data from UniRef and BFD containing up to 393 billion amino acids. The LMs were trained on the Summit supercomputer using 5616 GPUs and TPU Pod up-to 1024 cores. Dimensionality reduction revealed that the raw protein LM-embeddings from unlabeled data captured some biophysical features of protein sequences. We validated the advantage of using the embeddings as exclusive input for several subsequent tasks. The first was a per-residue prediction of protein secondary structure (3-state accuracy Q3=81%-87%); the second were per-protein predictions of protein sub-cellular localization (ten-state accuracy: Q10=81%) and membrane vs. water-soluble (2-state accuracy Q2=91%). For the per-residue predictions the transfer of the most informative embeddings (ProtT5) for the first time outperformed the state-of-the-art without using evolutionary information thereby bypassing expensive database searches. Taken together, the results implied that protein LMs learned some of the grammar of the language of life. To facilitate future work, we released our models at https://github.com/agemagician/ProtTrans.

cs.LG↗

arXiv2021

Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer

Sequential Recommendation characterizes the evolving patterns by modeling item sequences chronologically. The essential target of it is to capture the item transition correlations. The recent developments of transformer inspire the community to design effective sequence encoders, \textit{e.g.,} SASRec and BERT4Rec. However, we observe that these transformer-based models suffer from the cold-start issue, \textit{i.e.,} performing poorly for short sequences. Therefore, we propose to augment short sequences while still preserving original sequential correlations. We introduce a new framework for \textbf{A}ugmenting \textbf{S}equential \textbf{Re}commendation with \textbf{P}seudo-prior items~(ASReP). We firstly pre-train a transformer with sequences in a reverse direction to predict prior items. Then, we use this transformer to generate fabricated historical items at the beginning of short sequences. Finally, we fine-tune the transformer using these augmented sequences from the time order to predict the next item. Experiments on two real-world datasets verify the effectiveness of ASReP. The code is available on \url{https://github.com/DyGRec/ASReP}.

cs.IR↗

arXiv2021

Generalizable control for quantum parameter estimation through reinforcement learning

Measurement and estimation of parameters are essential for science and engineering, where one of the main quests is to find systematic schemes that can achieve high precision. While conventional schemes for quantum parameter estimation focus on the optimization of the probe states and measurements, it has been recently realized that control during the evolution can significantly improve the precision. The identification of optimal controls, however, is often computationally demanding, as typically the optimal controls depend on the value of the parameter which then needs to be re-calculated after the update of the estimation in each iteration. Here we show that reinforcement learning provides an efficient way to identify the controls that can be employed to improve the precision. We also demonstrate that reinforcement learning is highly generalizable, namely the neural network trained under one particular value of the parameter can work for different values within a broad range. These desired features make reinforcement learning an efficient alternative to conventional optimal quantum control methods.

quant-ph↗

arXiv2021

Temperature-sensitive spatial distribution of defects in PdSe2 flakes

Defect engineering plays an important role in tailoring the electronic transport properties of van der Waals materials. However, it is usually achieved through tuning the type and concentration of defects, rather than dynamically reconfiguring their spatial distribution. Here, we report temperature-sensitive spatial redistribution of defects in PdSe2 thin flakes through scanning tunneling microscopy (STM). We observe that the spatial distribution of Se vacancies in PdSe2 flakes exhibits a strong anisotropic characteristic at 80 K, and that this orientation-dependent feature is weakened when temperature is raised. Moreover, we carry out transport measurements on PdSe2 thin flakes and show that the anisotropic features of carrier mobility and phase coherent length are also sensitive to temperature. Combining with theoretical analysis, we conclude that temperature-driven defect spatial redistribution could interpret the temperature-sensitive electrical transport behaviors in PdSe2 thin flakes. Our work highlights that engineering spatial distribution of defects in the van der Waals materials, which has been overlooked before, may open up a new avenue to tailor the physical properties of materials and explore new device functionalities.

cond-mat.mes-hall↗

arXiv2021

FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture

With the fast evolvement of embedded deep-learning computing systems, applications powered by deep learning are moving from the cloud to the edge. When deploying neural networks (NNs) onto the devices under complex environments, there are various types of possible faults: soft errors caused by cosmic radiation and radioactive impurities, voltage instability, aging, temperature variations, and malicious attackers. Thus the safety risk of deploying NNs is now drawing much attention. In this paper, after the analysis of the possible faults in various types of NN accelerators, we formalize and implement various fault models from the algorithmic perspective. We propose Fault-Tolerant Neural Architecture Search (FT-NAS) to automatically discover convolutional neural network (CNN) architectures that are reliable to various faults in nowadays devices. Then we incorporate fault-tolerant training (FTT) in the search process to achieve better results, which is referred to as FTT-NAS. Experiments on CIFAR-10 show that the discovered architectures outperform other manually designed baseline architectures significantly, with comparable or fewer floating-point operations (FLOPs) and parameters. Specifically, with the same fault settings, F-FTT-Net discovered under the feature fault model achieves an accuracy of 86.2% (VS. 68.1% achieved by MobileNet-V2), and W-FTT-Net discovered under the weight fault model achieves an accuracy of 69.6% (VS. 60.8% achieved by ResNet-20). By inspecting the discovered architectures, we find that the operation primitives, the weight quantization range, the capacity of the model, and the connection pattern have influences on the fault resilience capability of NN models.

eess.SP↗

arXiv2021

Noncollinear topological textures in two-dimensional van der Waals materials: From magnetic to polar systems

In recent years, noncollinear topological textures have long gained increasing research attentions for their high values of both fundamental researches and potential applications. The recent discovery of intrinsic orders in magnetic and polar two-dimensional van der Waals materials provides a new ideal platform for the investigation of noncollinear topological textures. Here, we review the theoretical and experimental progresses on noncollinear topological textures in two-dimensional van der Waals materials in very recent years. During these years, magnetic skyrmions of both Bloch and Néel types have been observed experimentally in a few two-dimensional van der Waals materials and related heterostructures. Concurrently, more theoretic predictions basing on various mechanisms have been reported about different noncollinear topological textures in two-dimensional van der Waals materials, such as skyrmions, bimerons, anti-biskyrmions and skyrmionium, which are still waiting to be confirmed in experiments. Besides, noncollinear topological electric dipole orders have also been predicted in two-dimensional van der Waals materials. Taking advantage of the intrinsic two-dimensional nature and high integratability, the two-dimensional van der Waals materials will play an important role in the investigation on noncollinear topological textures in both magnetic and polar systems.

cond-mat.mtrl-sci↗

arXiv2021

Reinforcement Learning with Temporal Logic Constraints for Partially-Observable Markov Decision Processes

This paper proposes a reinforcement learning method for controller synthesis of autonomous systems in unknown and partially-observable environments with subjective time-dependent safety constraints. Mathematically, we model the system dynamics by a partially-observable Markov decision process (POMDP) with unknown transition/observation probabilities. The time-dependent safety constraint is captured by iLTL, a variation of linear temporal logic for state distributions. Our Reinforcement learning method first constructs the belief MDP of the POMDP, capturing the time evolution of estimated state distributions. Then, by building the product belief MDP of the belief MDP and the limiting deterministic B\uchi automaton (LDBA) of the temporal logic constraint, we transform the time-dependent safety constraint on the POMDP into a state-dependent constraint on the product belief MDP. Finally, we learn the optimal policy by value iteration under the state-dependent constraint.

cs.RO↗

arXiv2021

Speedup of nuclear-spin diffusion in hyperpolarized solids

We propose a correction to the coefficient of nuclear-spin diffusion by a factor $1/\sqrt{1-\overline{p}^{2}}$, where $\overline{p}$ is the average nuclear-spin polarization. The correction, derived by extending the Lowe-Gade theory to low-temperature cases, implies that transportation of nuclear magnetization through nuclear-spin diffusion accelerates when the system is hyperpolarized, whereas for low-polarization the correction factor approaches unity and the diffusion coefficient coincides with the conventional diffusion coefficient valid in the high-temperature limit. The proposed scaling of the nuclear-spin diffusion coefficient can lead to observable effects in the buildup of nuclear polarization by dynamic nuclear polarization.

cond-mat.other↗

arXiv2021

Cooperative Beam Hopping for Accurate Positioning in Ultra-Dense LEO Satellite Networks

In ultra-dense LEO satellite networks, conventional communication-oriented beam pattern design cannot provide multiple favorable signals from different satellites simultaneously, and thus leads to poor positioning performance. To tackle this issue, in this paper, we propose a novel cooperative beam hopping (BH) framework to adaptively tune beam layouts suitable for multi-satellite coordinated positioning. On this basis, a joint user association, BH design and power allocation optimization problem is formulated to minimize average Cramér-Rao lower bound (CRLB). An efficient flexible BH control algorithm (FBHCA) is then proposed to solve the problem. Finally, a thorough experimental platform is built following the Third Generation Partnership Project (3GPP) defined non-terrestrial network (NTN) simulation parameters to validate the performance gain of the devised algorithm. The numerical results demonstrate that FBHCA can significantly improve CRLB performance over the benchmark scheme.

cs.NI↗

arXiv2021

Entity Context Graph: Learning Entity Representations fromSemi-Structured Textual Sources on the Web

Knowledge is captured in the form of entities and their relationships and stored in knowledge graphs. Knowledge graphs enhance the capabilities of applications in many different areas including Web search, recommendation, and natural language understanding. This is mainly because, entities enable machines to understand things that go beyond simple tokens. Many modern algorithms use learned entity embeddings from these structured representations. However, building a knowledge graph takes time and effort, hence very costly and nontrivial. On the other hand, many Web sources describe entities in some structured format and therefore, finding ways to get them into useful entity knowledge is advantageous. We propose an approach that processes entity centric textual knowledge sources to learn entity embeddings and in turn avoids the need for a traditional knowledge graph. We first extract triples into the new representation format that does not use traditional complex triple extraction methods defined by pre-determined relationship labels. Then we learn entity embeddings through this new type of triples. We show that the embeddings learned from our approach are: (i) high quality and comparable to a known knowledge graph-based embeddings and can be used to improve them further, (ii) better than a contextual language model-based entity embeddings, and (iii) easy to compute and versatile in domain-specific applications where a knowledge graph is not readily available

cs.LG↗