arXiv subjects

Xu Han

Publications and source records attributed to Xu Han.

At least 307 records · Page 17Linked to original sources

arXiv2020

Learning from Context or Names? An Empirical Study on Neural Relation Extraction

Neural models have achieved remarkable success on relation extraction (RE) benchmarks. However, there is no clear understanding which type of information affects existing RE models to make decisions and how to further improve the performance of these models. To this end, we empirically study the effect of two main information sources in text: textual context and entity mentions (names). We find that (i) while context is the main source to support the predictions, RE models also heavily rely on the information from entity mentions, most of which is type information, and (ii) existing datasets may leak shallow heuristics via entity mentions and thus contribute to the high performance on RE benchmarks. Based on the analyses, we propose an entity-masked contrastive pre-training framework for RE to gain a deeper understanding on both textual context and type information while avoiding rote memorization of entities or use of superficial cues in mentions. We carry out extensive experiments to support our views, and show that our framework can improve the effectiveness and robustness of neural models in different RE scenarios. All the code and datasets are released at https://github.com/thunlp/RE-Context-or-Names.

cs.CL↗

arXiv2020

CPM: A Large-scale Generative Chinese Pre-trained Language Model

Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570GB training data, drew a lot of attention due to the capacity of few-shot (even zero-shot) learning. However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available. In this technical report, we release the Chinese Pre-trained Language Model (CPM) with generative pre-training on large-scale Chinese training data. To the best of our knowledge, CPM, with 2.6 billion parameters and 100GB Chinese training data, is the largest Chinese pre-trained language model, which could facilitate several downstream Chinese NLP tasks, such as conversation, essay generation, cloze test, and language understanding. Extensive experiments demonstrate that CPM achieves strong performance on many NLP tasks in the settings of few-shot (even zero-shot) learning. The code and parameters are available at https://github.com/TsinghuaAI/CPM-Generate.

cs.CL↗

arXiv2020

Deep Reinforcement Learning of Transition States

Combining reinforcement learning (RL) and molecular dynamics (MD) simulations, we propose a machine-learning approach (RL$^‡$) to automatically unravel chemical reaction mechanisms. In RL$^‡$, locating the transition state of a chemical reaction is formulated as a game, where a virtual player is trained to shoot simulation trajectories connecting the reactant and product. The player utilizes two functions, one for value estimation and the other for policy making, to iteratively improve the chance of winning this game. We can directly interpret the reaction mechanism according to the value function. Meanwhile, the policy function enables efficient sampling of the transition paths, which can be further used to analyze the reaction dynamics and kinetics. Through multiple experiments, we show that RL‡ can be trained tabula rasa hence allows us to reveal chemical reaction mechanisms with minimal subjective biases.

physics.chem-ph↗

arXiv2020

Denoising Relation Extraction from Document-level Distant Supervision

Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance. However, the existing success of DS cannot be directly transferred to the more challenging document-level relation extraction (DocRE), since the inherent noise in DS may be even multiplied in document level and significantly harm the performance of RE. To address this challenge, we propose a novel pre-trained model for DocRE, which denoises the document-level DS data via multiple pre-training tasks. Experimental results on the large-scale DocRE benchmark show that our model can capture useful information from noisy DS data and achieve promising results.

cs.CL↗

arXiv2020

Floquet Cavity Electromagnonics

Hybrid magnonics has recently attracted intensive attentions as a promising platform for coherent information processing. In spite of its rapid development, on-demand control over the interaction of magnons with other information carriers, in particular microwave photons in electromagnonic systems, has been long missing, significantly limiting the broad applications of hybrid magnonics. Here, we show that by introducing Floquet engineering into cavity electromagnonics, coherent control on the magnon-microwave photon coupling can be realized. Leveraging the periodic temporal modulation from a Floquet drive, our first-of-its-kind Floquet cavity electromagnonic system can manipulate the interaction between hybridized cavity electromagnonic modes on demand. Moreover, we demonstrate a new coupling regime in such systems: the Floquet ultrastrong coupling, where the Floquet splitting is comparable with or even larger than the level spacing of the two interacting modes, resulting in the breakdown of the rotating wave approximation. Our findings open up new directions for magnon-based coherent signal processing.

cond-mat.mes-hall↗

arXiv2020

Ground-state Pulsed Cavity Electro-optics for Microwave-to-optical Conversion

In the development of quantum microwave-to-optical (MO) converters, excessive noise induced by the parametric optical drive remains a major challenge at milli-Kelvin temperatures. Here we study the extraneous noise added to an electro-optic transducer in its quantum ground state under an intense pulsed optical excitation. The integrated electro-optical transducer leverages the inherent Pockels effect of aluminum nitride microrings, flip-chip bonded to a superconducting resonator. Applying a pulsed optical drive with peak power exceeding the cooling power of the dilution refrigerator at its base temperature, we observe efficient bi-directional MO conversion, with near-ground state microwave thermal excitation ($\bar{n}_\mathrm{e}=0.09\pm0.06$). Time evolution study reveals that the residual thermal excitation is dominated by the superconductor absorption of stray light scattered off the chip-fiber interface. Our results shed light on suppressing microwave noise in a cavity electro-optic system under intense optical drive, which is an essential step towards quantum state transduction between microwave and optical frequencies.

quant-ph↗

arXiv2020

Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training

Semantic segmentation (SS) is an important perception manner for self-driving cars and robotics, which classifies each pixel into a pre-determined class. The widely-used cross entropy (CE) loss-based deep networks has achieved significant progress w.r.t. the mean Intersection-over Union (mIoU). However, the cross entropy loss can not take the different importance of each class in an self-driving system into account. For example, pedestrians in the image should be much more important than the surrounding buildings when make a decisions in the driving, so their segmentation results are expected to be as accurate as possible. In this paper, we propose to incorporate the importance-aware inter-class correlation in a Wasserstein training framework by configuring its ground distance matrix. The ground distance matrix can be pre-defined following a priori in a specific task, and the previous importance-ignored methods can be the particular cases. From an optimization perspective, we also extend our ground metric to a linear, convex or concave increasing function $w.r.t.$ pre-defined ground distance. We evaluate our method on CamVid and Cityscapes datasets with different backbones (SegNet, ENet, FCN and Deeplab) in a plug and play fashion. In our extenssive experiments, Wasserstein loss demonstrates superior segmentation performance on the predefined critical classes for safe-driving.

cs.CV↗

arXiv2020

MAVEN: A Massive General Domain Event Detection Dataset

Event detection (ED), which means identifying event trigger words and classifying event types, is the first and most fundamental step for extracting event knowledge from plain text. Most existing datasets exhibit the following issues that limit further development of ED: (1) Data scarcity. Existing small-scale datasets are not sufficient for training and stably benchmarking increasingly sophisticated modern neural methods. (2) Low coverage. Limited event types of existing datasets cannot well cover general-domain events, which restricts the applications of ED models. To alleviate these problems, we present a MAssive eVENt detection dataset (MAVEN), which contains 4,480 Wikipedia documents, 118,732 event mention instances, and 168 event types. MAVEN alleviates the data scarcity problem and covers much more general event types. We reproduce the recent state-of-the-art ED models and conduct a thorough evaluation on MAVEN. The experimental results show that existing ED methods cannot achieve promising results on MAVEN as on the small datasets, which suggests that ED in the real world remains a challenging task and requires further research efforts. We also discuss further directions for general domain ED with empirical analyses. The source code and dataset can be obtained from https://github.com/THU-KEG/MAVEN-dataset.

cs.CL↗

arXiv2020

Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph

Multi-hop reasoning has been widely studied in recent years to seek an effective and interpretable method for knowledge graph (KG) completion. Most previous reasoning methods are designed for dense KGs with enough paths between entities, but cannot work well on those sparse KGs that only contain sparse paths for reasoning. On the one hand, sparse KGs contain less information, which makes it difficult for the model to choose correct paths. On the other hand, the lack of evidential paths to target entities also makes the reasoning process difficult. To solve these problems, we propose a multi-hop reasoning model named DacKGR over sparse KGs, by applying novel dynamic anticipation and completion strategies: (1) The anticipation strategy utilizes the latent prediction of embedding-based models to make our model perform more potential path search over sparse KGs. (2) Based on the anticipation information, the completion strategy dynamically adds edges as additional actions during the path search, which further alleviates the sparseness problem of KGs. The experimental results on five datasets sampled from Freebase, NELL and Wikidata show that our method outperforms state-of-the-art baselines. Our codes and datasets can be obtained from https://github.com/THU-KEG/DacKGR

cs.CL↗

arXiv2020

More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction

Relational facts are an important component of human knowledge, which are hidden in vast amounts of text. In order to extract these facts from text, people have been working on relation extraction (RE) for years. From early pattern matching to current neural networks, existing RE methods have achieved significant progress. Yet with explosion of Web text and emergence of new relations, human knowledge is increasing drastically, and we thus require "more" from RE: a more powerful RE system that can robustly utilize more data, efficiently learn more relations, easily handle more complicated context, and flexibly generalize to more open domains. In this paper, we look back at existing RE methods, analyze key challenges we are facing nowadays, and show promising directions towards more powerful RE. We hope our view can advance this field and inspire more efforts in the community.

cs.CL↗

arXiv2020

A Deep Network for Joint Registration and Reconstruction of Images with Pathologies

Registration of images with pathologies is challenging due to tissue appearance changes and missing correspondences caused by the pathologies. Moreover, mass effects as observed for brain tumors may displace tissue, creating larger deformations over time than what is observed in a healthy brain. Deep learning models have successfully been applied to image registration to offer dramatic speed up and to use surrogate information (e.g., segmentations) during training. However, existing approaches focus on learning registration models using images from healthy patients. They are therefore not designed for the registration of images with strong pathologies for example in the context of brain tumors, and traumatic brain injuries. In this work, we explore a deep learning approach to register images with brain tumors to an atlas. Our model learns an appearance mapping from images with tumors to the atlas, while simultaneously predicting the transformation to atlas space. Using separate decoders, the network disentangles the tumor mass effect from the reconstruction of quasi-normal images. Results on both synthetic and real brain tumor scans show that our approach outperforms cost function masking for registration to the atlas and that reconstructed quasi-normal images can be used for better longitudinal registrations.

eess.IV↗

arXiv2020

An Upper Bound for Functions of Estimators in High Dimensions

We provide an upper bound as a random variable for the functions of estimators in high dimensions. This upper bound may help establish the rate of convergence of functions in high dimensions. The upper bound random variable may converge faster, slower, or at the same rate as estimators depending on the behavior of the partial derivative of the function. We illustrate this via three examples. The first two examples use the upper bound for testing in high dimensions, and third example derives the estimated out-of-sample variance of large portfolios. All our results allow for a larger number of parameters, p, than the sample size, n.

econ.EM↗

arXiv2020

Multi-stage Power Scheduling Framework for Data Center with Chilled Water Storage in Energy and Regulation Markets

Leveraging electrochemical and thermal energy storage systems has been proposed as a strategy to reduce peak power in data centers. Thermal energy storage systems, such as chilled water tanks, have gained increasing attention in data centers for load shifting due to their relatively small capital and operational costs compared to electrochemical energy storage. However, there are few studies investigating the possibility of utilizing thermal energy storage system with resources to provide ancillary services (e.g., frequency regulation) to the grid. This paper proposes a synergistic control strategy for the data center with a chilled water storage providing frequency regulation service by adjusting the chiller capacity, storage charging rate, and IT server CPU frequency. Then, a three-stage multi-market scheduling framework based on a model predictive control scheme is developed to minimize operational costs of data centers participating in both energy and regulation markets. The framework solves a power baseline scheduling problem, a regulation reserve problem, and a real-time power signal tracking problem sequentially. Simulation results show that utilizing the thermal energy storage can increase the regulation capacity bid, reduce energy costs and demand charges, and also harvest frequency regulation revenues. The proposed multi-market scheduling framework in a span of two days can reduce the operational costs up to 8.8% ($1,606.4) compared to the baseline with 0.2% (\$38.7) energy cost reduction, 6.5% (\$1,179.4) from demand reduction, and 2.1% (\$338.3) from regulation revenues.

eess.SY↗

arXiv2020

Waveguide cavity optomagnonics for broadband multimode microwave-to-optics conversion

Cavity optomagnonics has emerged as a promising platform for studying coherent photon-spin interactions as well as tunable microwave-to-optical conversion. However, current implementation of cavity optomagnonics in ferrimagnetic crystals remains orders of magnitude larger in volume than state-of-the-art cavity optomechanical devices, resulting in very limited magneto-optical interaction strength. Here, we demonstrate a cavity optomagnonic device based on integrated waveguides and its application for microwave-to-optical conversion. By designing a ferrimagnetic rib waveguide to support multiple magnon modes with maximal mode overlap to the optical field, we realize a high magneto-optical cooperativity which is three orders of magnitude higher compared to previous records obtained on polished YIG spheres. Furthermore, we achieve tunable conversion of microwave photons at around 8.45 GHz to 1550 nm light with a broad conversion bandwidth as large as 16.1 MHz. The unique features of the system point to novel applications at the crossroad between quantum optics and magnonics.

cond-mat.mes-hall↗

arXiv2020

A Perspective on Deep Learning for Molecular Modeling and Simulations

Deep learning is transforming many areas in science, and it has great potential in modeling molecular systems. However, unlike the mature deployment of deep learning in computer vision and natural language processing, its development in molecular modeling and simulations is still at an early stage, largely because the inductive biases of molecules are completely different from those of images or texts. Footed on these differences, we first reviewed the limitations of traditional deep learning models from the perspective of molecular physics, and wrapped up some relevant technical advancement at the interface between molecular modeling and deep learning. We do not focus merely on the ever more complex neural network models, instead, we emphasize the theories and ideas behind modern deep learning. We hope that transacting these ideas into molecular modeling will create new opportunities. For this purpose, we summarized several representative applications, ranging from supervised to unsupervised and reinforcement learning, and discussed their connections with the emerging trends in deep learning. Finally, we outlook promising directions which may help address the existing issues in the current framework of deep molecular modeling.

physics.comp-ph↗

arXiv2020

Magnon-photon strong coupling for tunable microwave circulators

We present a generic theoretical framework to describe non-reciprocal microwave circulation in a multimode cavity magnonic system and assess the optimal performance of practical circulator devices. We show that high isolation (> 56 dB), extremely low insertion loss (< 0.05 dB), and flexible bandwidth control can be potentially realized in high-quality-factor superconducting cavity based magnonic platforms. These circulation characteristics are analyzed with materials of different spin densities. For high-spin-density materials such as yttrium iron garnet, strong coupling operation regime can be harnessed to obtain a broader circulation bandwidth. We also provide practical design principles for a highly integratible low-spin-density material (vanadium tetracyanoethylene) for narrow-band circulator operation, which could benefit noise-sensitive quantum microwave measurements. This theory can be extended to other coupled systems and provide design guidelines for achieving tunable microwave non-reciprocity for both classical and quantum applications.

physics.app-ph↗

arXiv2020

Cavity piezo-mechanics for superconducting-nanophotonic quantum interface

Hybrid quantum systems are essential for the realization of distributed quantum networks. In particular, piezo-mechanics operating at typical superconducting qubit frequencies features low thermal excitations, and offers an appealing platform to bridge superconducting quantum processors and optical telecommunication channels. However, integrating superconducting and optomechanical elements at cryogenic temperatures with sufficiently strong interactions remains a tremendous challenge. Here, we report an integrated superconducting cavity piezo-optomechanical platform where 10-GHz phonons are resonantly coupled with photons in a superconducting and a nanophotonic cavities at the same time. Benefited from the achieved large piezo-mechanical cooperativity ($C_\mathrm{em}\sim7$) and the enhanced optomechanical coupling boosted by a pulsed optical pump, we demonstrate coherent interactions at cryogenic temperatures via the observation of efficient microwave-optical photon conversion. This hybrid interface makes a substantial step towards quantum communication at large scale, as well as novel explorations in microwave-optical photon entanglement and quantum sensing mediated by gigahertz phonons.

physics.optics↗

arXiv2020

Microwave--Optical entanglement in a Strongly Coupled Electro-Optomechanical System

Quantum transduction between microwave and optics can be realized by quantum teleportation if given reliable microwave-optical entanglement, namely entanglement-based quantum transduction. To realize this protocol, an entangled source with high-fidelity between the two frequencies is necessary. In this paper, we study the microwave and optical entanglement generation based on a generic cavity electro-optomechanical system in the strong coupling regime. Splittings are shown in the microwave and optical output spectra and the frequency entanglement between the two modes is quantified. We show that entanglement can be straightforwardly encoded in the frequency-bin degree of freedom and propose a feasible experiment to verify entangled photon pairs. The experimental implementation is systematically analyzed, and the preferable parameter regime for entanglement verification is identified. An inequality is given as a criterion for good entanglement verification with analysis of practical imperfections.

quant-ph↗