arXiv subjects

Yang Xu

Publications and source records attributed to Yang Xu.

At least 343 records · Page 19Linked to original sources

arXiv2022

HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysis

Contextualized word embeddings have demonstrated state-of-the-art performance in various natural language processing tasks including those that concern historical semantic change. However, language models such as BERT was trained primarily on contemporary corpus data. To investigate whether training on historical corpus data improves diachronic semantic analysis, we present a pre-trained BERT-based language model, HistBERT, trained on the balanced Corpus of Historical American English. We examine the effectiveness of our approach by comparing the performance of the original BERT and that of HistBERT, and we report promising results in word similarity and semantic shift analysis. Our work suggests that the effectiveness of contextual embeddings in diachronic semantic analysis is dependent on the temporal profile of the input text and care should be taken in applying this methodology to study historical semantic change.

cs.CL↗

arXiv2022

Tunable bilayer Hubbard model physics in twisted WSe2

Moiré materials with flat electronic bands provide a highly controllable quantum system for studies of strong-correlation physics and topology. In particular, angle-aligned heterobilayers of semiconducting transition metal dichalcogenides (TMDs) with large band offset realize the single-band Hubbard models. Introduction of a new layer degree of freedom is expected to foster richer interactions, enabling Hund's physics, interlayer exciton condensation and new superconducting pairing mechanisms to name a few. Here, we report competing electronic orders in twisted AB-homobilayer WSe2, which realizes a bilayer Hubbard model in the weak interlayer hopping limit for holes. We characterize the charge order, layer polarization and magnetization of the moiré bilayer, subjected to an out-of-plane electric and magnetic field, by exciton sensing and magneto circular dichroism measurements. By layer-polarizing holes via the electric field, we observe a crossover from an excitonic insulator to a charge-transfer insulator at hole density of $ν$=1 (in unit of moiré density), a transition from a paramagnetic to an antiferromagnetic charge-transfer insulator at $ν$=2, and evidence for a layer-selective Mott insulator at 1<$ν$<2. The unique coupling of charge and spin to external fields also manifests a giant magneto-electric response. Our results establish a new solid-state simulator for problems in strong-correlation physics that are described by bilayer Hubbard models.

cond-mat.str-el↗

arXiv2022

ML4CO-KIDA: Knowledge Inheritance in Dataset Aggregation

The Machine Learning for Combinatorial Optimization (ML4CO) NeurIPS 2021 competition aims to improve state-of-the-art combinatorial optimization solvers by replacing key heuristic components with machine learning models. On the dual task, we design models to make branching decisions to promote the dual bound increase faster. We propose a knowledge inheritance method to generalize knowledge of different models from the dataset aggregation process, named KIDA. Our improvement overcomes some defects of the baseline graph-neural-networks-based methods. Further, we won the $1$\textsuperscript{st} Place on the dual task. We hope this report can provide useful experience for developers and researchers. The code is available at https://github.com/megvii-research/NeurIPS2021-ML4CO-KIDA.

cs.AI↗

arXiv2022

Critical topology and pressure-induced superconductivity in the van der Waals compound AuTe2Br

The study on quantum spin Hall effect and topological insulators formed the prologue to the surge of research activities in topological materials in the past decade. Compared to intricately engineered quantum wells, three-dimensional weak topological insulators provide a natural route to the quantum spin Hall effect, due to the adiabatic connection between them and a stack of quantum spin Hall insulators, and the convenience in exfoliation of samples associated with their van der Waals-type structure. Despite these advantages, both theoretical prediction and experimental identification of weak topological insulators remain scarce. Here, based on first-principles calculations, we show that AuTe2Br locates at the boundary between a strong and a weak topological insulating state. More interestingly, the critical topology of AuTe2Br persists up to an applied pressure of ~ 15.4 GPa before a structural phase transition accompanied by a change of electronic topology and the onset of superconductivity. Our results establish AuTe2Br as a new candidate for weak topological insulators with the potential to realize various other topological phases of matter.

cond-mat.mtrl-sci↗

arXiv2022

LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding

Pre-training of text and layout has proved effective in a variety of visually-rich document understanding tasks due to its effective model architecture and the advantage of large-scale unlabeled scanned/digital-born documents. We propose LayoutLMv2 architecture with new pre-training tasks to model the interaction among text, layout, and image in a single multi-modal framework. Specifically, with a two-stream multi-modal Transformer encoder, LayoutLMv2 uses not only the existing masked visual-language modeling task but also the new text-image alignment and text-image matching tasks, which make it better capture the cross-modality interaction in the pre-training stage. Meanwhile, it also integrates a spatial-aware self-attention mechanism into the Transformer architecture so that the model can fully understand the relative positional relationship among different text blocks. Experiment results show that LayoutLMv2 outperforms LayoutLM by a large margin and achieves new state-of-the-art results on a wide variety of downstream visually-rich document understanding tasks, including FUNSD (0.7895 $\to$ 0.8420), CORD (0.9493 $\to$ 0.9601), SROIE (0.9524 $\to$ 0.9781), Kleister-NDA (0.8340 $\to$ 0.8520), RVL-CDIP (0.9443 $\to$ 0.9564), and DocVQA (0.7295 $\to$ 0.8672). We made our model and code publicly available at \url{https://aka.ms/layoutlmv2}.

cs.CL↗

arXiv2021

Nonlinear Intensity Underwater Sonar Image Matching Method Based on Phase Information and Deep Convolution Features

In the field of deep-sea exploration, sonar is presently the only efficient long-distance sensing device. The complicated underwater environment, such as noise interference, low target intensity or background dynamics, has brought many negative effects on sonar imaging. Among them, the problem of nonlinear intensity is extremely prevalent. It is also known as the anisotropy of acoustic sensor imaging, that is, when autonomous underwater vehicles (AUVs) carry sonar to detect the same target from different angles, the intensity variation between image pairs is sometimes very large, which makes the traditional matching algorithm almost ineffective. However, image matching is the basis of comprehensive tasks such as navigation, positioning, and mapping. Therefore, it is very valuable to obtain robust and accurate matching results. This paper proposes a combined matching method based on phase information and deep convolution features. It has two outstanding advantages: one is that the deep convolution features could be used to measure the similarity of the local and global positions of the sonar image; the other is that local feature matching could be performed at the key target position of the sonar image. This method does not need complex manual designs, and completes the matching task of nonlinear intensity sonar images in a close end-to-end manner. Feature matching experiments are carried out on the deep-sea sonar images captured by AUVs, and the results show that our proposal has preeminent matching accuracy and robustness.

cs.CV↗

arXiv2021

Hierarchical Multi-robot Strategies Synthesis and Optimization under Individual and Collaborative Temporal Logic Specifications

This paper presents a hierarchical framework to solve the multi-robot temporal task planning problem. We assume that each robot has its individual task specification and the robots have to jointly satisfy a global collaborative task specification, both described in linear temporal logic. Specifically, a central server firstly extracts and decomposes a collaborative task sequence from the automaton corresponding to the collaborative task specification, and allocates the subtasks in the sequence to robots. The robots can then synthesize their initial execution strategies based on locally constructed product automatons, combining the assigned collaborative tasks and their individual task specifications. Furthermore, we propose a distributed execution strategy adjusting mechanism to iteratively improve the time efficiency, by reducing wait time in collaborations caused by potential synchronization constraints. We prove the completeness of the proposed framework under assumptions, and analyze its time complexity and optimality. Extensive simulation results verify the scalability and optimization efficiency of the proposed method.

cs.RO↗

arXiv2021

Inelastic Axial and Vector Structure Functions for Lepton-Nucleon Scattering 2021 Update

We report on an update (2021) of a phenomenological model for inelastic neutrino- and electron-nucleon scattering cross sections using effective leading order parton distribution functions with a new scaling variable $ξ_w$. Non-perturbative effects are well described using the $ξ_w$ scaling variable in combination with multiplicative $K$ factors at low $Q^2$. The model describes all inelastic charged-leptron-nucleon scattering data (HERA/NMC/BCDMS/SLAC/JLab) ranging from very high $Q^2$ to very low $Q^2$ and down to the $Q^2=0$ photo-production region. The model has been developed to be used in analysis of neutrino oscillation experiments in the few GeV region. The 2021 update accounts for the difference between axial and vector structure function which brings it into much better agreement with neutrino-nucleon total cross section measurements. The model has been developed primarily for hadronic final state masses $W$ above 1.8 GeV. However with additional parameters the model also describe the $average$ neutrino cross sections in the resonance region down to $W$=1.4 GeV.

hep-ph↗

arXiv2021

Breaking the curse of dimensionality with Isolation Kernel

The curse of dimensionality has been studied in different aspects. However, breaking the curse has been elusive. We show for the first time that it is possible to break the curse using the recently introduced Isolation Kernel. We show that only Isolation Kernel performs consistently well in indexed search, spectral & density peaks clustering, SVM classification and t-SNE visualization in both low and high dimensions, compared with distance, Gaussian and linear kernels. This is also supported by our theoretical analyses that Isolation Kernel is the only kernel that has the provable ability to break the curse, compared with existing metric-based Lipschitz continuous kernels.

cs.LG↗

arXiv2021

Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chaining

Natural language relies on a finite lexicon to express an unbounded set of emerging ideas. One result of this tension is the formation of new compositions, such that existing linguistic units can be combined with emerging items into novel expressions. We develop a framework that exploits the cognitive mechanisms of chaining and multimodal knowledge to predict emergent compositional expressions through time. We present the syntactic frame extension model (SFEM) that draws on the theory of chaining and knowledge from "percept", "concept", and "language" to infer how verbs extend their frames to form new compositions with existing and novel nouns. We evaluate SFEM rigorously on the 1) modalities of knowledge and 2) categorization models of chaining, in a syntactically parsed English corpus over the past 150 years. We show that multimodal SFEM predicts newly emerged verb syntax and arguments substantially better than competing models using purely linguistic or unimodal knowledge. We find support for an exemplar view of chaining as opposed to a prototype view and reveal how the joint approach of multimodal chaining may be fundamental to the creation of literal and figurative language uses including metaphor and metonymy.

cs.CL↗

arXiv2021

An unsupervised framework for tracing textual sources of moral change

Morality plays an important role in social well-being, but people's moral perception is not stable and changes over time. Recent advances in natural language processing have shown that text is an effective medium for informing moral change, but no attempt has been made to quantify the origins of these changes. We present a novel unsupervised framework for tracing textual sources of moral change toward entities through time. We characterize moral change with probabilistic topical distributions and infer the source text that exerts prominent influence on the moral time course. We evaluate our framework on a diverse set of data ranging from social media to news articles. We show that our framework not only captures fine-grained human moral judgments, but also identifies coherent source topics of moral change triggered by historical events. We apply our methodology to analyze the news in the COVID-19 pandemic and demonstrate its utility in identifying sources of moral change in high-impact and real-time social events.

cs.CL↗

arXiv2021

Evolution of emotion semantics

Humans possess the unique ability to communicate emotions through language. Although concepts like anger or awe are abstract, there is a shared consensus about what these English emotion words mean. This consensus may give the impression that their meaning is static, but we propose this is not the case. We cannot travel back to earlier periods to study emotion concepts directly, but we can examine text corpora, which have partially preserved the meaning of emotion words. Using natural language processing of historical text, we found evidence for semantic change in emotion words over the past century and that varying rates of change were predicted in part by an emotion concept's prototypicality - how representative it is of the broader category of "emotion". Prototypicality negatively correlated with historical rates of emotion semantic change obtained from text-based word embeddings, beyond more established variables including usage frequency in English and a second comparison language, French. This effect for prototypicality did not consistently extend to the semantic category of birds, suggesting its relevance for predicting semantic change may be category-dependent. Our results suggest emotion semantics are evolving over time, with prototypical emotion words remaining semantically stable, while other emotion words evolve more freely.

cs.CL↗

arXiv2021

A Galois Connection Approach to Wei-Type Duality Theorems

In $1991$, Wei proved a duality theorem that established an interesting connection between the generalized Hamming weights of a linear code and those of its dual code. Wei's duality theorem has since been extensively studied from different perspectives and extended to other settings. In this paper, we re-examine Wei's duality theorem and its various extensions, henceforth referred to as Wei-type duality theorems, from a new Galois connection perspective. Our approach is based on the observation that the generalized Hamming weights and the dimension/length profiles of a linear code form a Galois connection. The central result in this paper is a general Wei-type duality theorem for two Galois connections between finite subsets of $\mathbb{Z}$, from which all the known Wei-type duality theorems can be recovered. As corollaries of our central result, we prove new Wei-type duality theorems for $w$-demimatroids defined over finite sets and $w$-demi-polymatroids defined over modules with a composition series, which further allows us to unify and generalize all the known Wei-type duality theorems established for codes endowed with various metrics.

cs.IT↗

arXiv2021

Fourier-Reflexive Partitions Induced by Poset Metric

Let $\mathbf{H}$ be the cartesian product of a family of finite abelian groups indexed by a finite set $Ω$. A given poset (i.e., partially ordered set) $\mathbf{P}=(Ω,\preccurlyeq_{\mathbf{P}})$ gives rise to a poset metric on $\mathbf{H}$, which further leads to a partition $\mathcal{Q}(\mathbf{H},\mathbf{P})$ of $\mathbf{H}$. We prove that if $\mathcal{Q}(\mathbf{H},\mathbf{P})$ is Fourier-reflexive, then its dual partition $Λ$ coincides with the partition of $\hat{\mathbf{H}}$ induced by $\mathbf{\overline{P}}$, the dual poset of $\mathbf{P}$, and moreover, $\mathbf{P}$ is necessarily hierarchical. This result establishes a conjecture proposed by Gluesing-Luerssen in \cite{4}. We also show that with some other assumptions, $Λ$ is finer than the partition of $\hat{\mathbf{H}}$ induced by $\mathbf{\overline{P}}$. In addition, we give some necessary and sufficient conditions for $\mathbf{P}$ to be hierarchical, and for the case that $\mathbf{P}$ is hierarchical, we give an explicit criterion for determining whether two codewords in $\hat{\mathbf{H}}$ belong to the same block of $Λ$. We prove these results by relating the involved partitions with certain family of polynomials, a generalized version of which is also proposed and studied to generalize the aforementioned results.

cs.IT↗

arXiv2021

What do writing features tell us about AI papers?

As the numbers of submissions to conferences grow quickly, the task of assessing the quality of academic papers automatically, convincingly, and with high accuracy attracts increasing attention. We argue that studying interpretable dimensions of these submissions could lead to scalable solutions. We extract a collection of writing features, and construct a suite of prediction tasks to assess the usefulness of these features in predicting citation counts and the publication of AI-related papers. Depending on the venues, the writing features can predict the conference vs. workshop appearance with F1 scores up to 60-90, sometimes even outperforming the content-based tf-idf features and RoBERTa. We show that the features describe writing style more than content. To further understand the results, we estimate the causal impact of the most indicative features. Our analysis on writing features provides a perspective to assessing and refining the writing of academic articles at scale.

cs.CL↗

arXiv2021

Measuring Financial Time Series Similarity With a View to Identifying Profitable Stock Market Opportunities

Forecasting stock returns is a challenging problem due to the highly stochastic nature of the market and the vast array of factors and events that can influence trading volume and prices. Nevertheless it has proven to be an attractive target for machine learning research because of the potential for even modest levels of prediction accuracy to deliver significant benefits. In this paper, we describe a case-based reasoning approach to predicting stock market returns using only historical pricing data. We argue that one of the impediments for case-based stock prediction has been the lack of a suitable similarity metric when it comes to identifying similar pricing histories as the basis for a future prediction -- traditional Euclidean and correlation based approaches are not effective for a variety of reasons -- and in this regard, a key contribution of this work is the development of a novel similarity metric for comparing historical pricing data. We demonstrate the benefits of this metric and the case-based approach in a real-world application in comparison to a variety of conventional benchmarks.

q-fin.ST↗

arXiv2021

SIMPL: Generating Synthetic Overhead Imagery to Address Zero-shot and Few-Shot Detection Problems

Recently deep neural networks (DNNs) have achieved tremendous success for object detection in overhead (e.g., satellite) imagery. One ongoing challenge however is the acquisition of training data, due to high costs of obtaining satellite imagery and annotating objects in it. In this work we present a simple approach - termed Synthetic object IMPLantation (SIMPL) - to easily and rapidly generate large quantities of synthetic overhead training data for custom target objects. We demonstrate the effectiveness of using SIMPL synthetic imagery for training DNNs in zero-shot scenarios where no real imagery is available; and few-shot learning scenarios, where limited real-world imagery is available. We also conduct experiments to study the sensitivity of SIMPL's effectiveness to some key design parameters, providing users for insights when designing synthetic imagery for custom objects. We release a software implementation of our SIMPL approach so that others can build upon it, or use it for their own custom problems.

cs.CV↗

arXiv2021

A diachronic evaluation of gender asymmetry in euphemism

The use of euphemisms is a known driver of language change. It has been proposed that women use euphemisms more than men. Although there have been several studies investigating gender differences in language, the claim about euphemism usage has not been tested comprehensively through time. If women do use euphemisms more, this could mean that women also lead the formation of new euphemisms and language change over time. Using four large diachronic text corpora of English, we evaluate the claim that women use euphemisms more than men through a quantitative analysis. We assembled a list of 106 euphemism-taboo pairs to analyze their relative use through time by each gender in the corpora. Contrary to the existing belief, our results show that women do not use euphemisms with a higher proportion than men. We repeated the analysis using different subsets of the euphemism-taboo pairs list and found that our result was robust. Our study indicates that in a broad range of settings involving both speech and writing, and with varying degrees of formality, women do not use or form euphemisms more than men.

cs.CL↗