arXiv subjects

Fei Huang

Publications and source records attributed to Fei Huang.

At least 487 records · Page 27Linked to original sources

arXiv2021

A Semantic-based Method for Unsupervised Commonsense Question Answering

Unsupervised commonsense question answering is appealing since it does not rely on any labeled task data. Among existing work, a popular solution is to use pre-trained language models to score candidate choices directly conditioned on the question or context. However, such scores from language models can be easily affected by irrelevant factors, such as word frequencies, sentence structures, etc. These distracting factors may not only mislead the model to choose a wrong answer but also make it oversensitive to lexical perturbations in candidate answers. In this paper, we present a novel SEmantic-based Question Answering method (SEQA) for unsupervised commonsense question answering. Instead of directly scoring each answer choice, our method first generates a set of plausible answers with generative models (e.g., GPT-2), and then uses these plausible answers to select the correct choice by considering the semantic similarity between each plausible answer and each choice. We devise a simple, yet sound formalism for this idea and verify its effectiveness and robustness with extensive experiments. We evaluate the proposed method on four benchmark datasets, and our method achieves the best results in unsupervised settings. Moreover, when attacked by TextFooler with synonym replacement, SEQA demonstrates much less performance drops than baselines, thereby indicating stronger robustness.

cs.CL↗

arXiv2021

Analysis of the differential cross section and photon beam asymmetry data for $γp \to η^\prime p$

The photoproduction reaction of $γp \to η^\prime p$ is investigated based on an effective Lagrangian approach in the tree-level approximation, with the purpose being to understand the reaction mechanisms and to extract the resonance contents and the associated resonance parameters in this reaction. Apart from the $t$-channel $ρ$ and $ω$ exchanges, $s$- and $u$-channel nucleon exchanges, and generalized contact term, the exchanges of a minimum number of nucleon resonances in the $s$ channel are taken into account in constructing the reaction amplitudes to describe the experimental data. It is found that a satisfactory description of the available data on both differential cross sections and photon beam asymmetries can be obtained by including in the $s$ channel the exchanges of the $N(1875)3/2^-$ and $N(2040)3/2^+$ resonances. The reaction mechanisms of $γp \to η^\prime p$ are discussed and a prediction for the target nucleon asymmetries is presented.

hep-ph↗

arXiv2021

Hidden charm tetraquark states in a diquark model

The purpose of the present study is to explore the mass spectrum of the hidden charm tetraquark states within a diquark model. Proposing that a tetraquark state is composed of a diquark and an antidiquark, the masses of all possible $[qc][\bar{q}\bar{c}]$, $[sc][\bar{s}\bar{c}]$, and $[qc][\bar{s}\bar{c}]$ $\left([sc][\bar{q}\bar{c}]\right)$ hidden charm tetraquark states are systematically calculated by use of an effective Hamiltonian, which contains color, spin, and flavor dependent interactions. Apart from the $X(3872)$, $Z(3900)$, $χ_{c2}(3930)$, and $X(4350)$ which are taken as input to fix the model parameters, the calculated results support that the $χ_{c0}(3860)$, $X(4020)$, $X(4050)$ are $[qc][\bar{q}\bar{c}]$ states with $I^GJ^{PC}=0^+0^{++}$, $1^+1^{+-}$, and $1^-2^{++}$, respectively, the $χ_{c1}(4274)$ is an $[sc][\bar{s}\bar{c}]$ state with $I^GJ^{PC}=0^+1^{++}$, the $X(3940)$ is a $[qc][\bar{q}\bar{c}]$ state with $I^GJ^{PC}=1^-0^{++}$ or $1^-1^{++}$, the $Z_{cs}(3985)^-$ is an $[sc][\bar{q}\bar{c}]$ state with $J^{P}=0^{+}$ or $1^+$, and the $Z_{cs}(4000)^+$ and $Z_{cs}(4220)^+$ are $[qc][\bar{s}\bar{c}]$ states with $J^{P}=1^{+}$. Predictions for other possible tetraquark states are also given.

hep-ph↗

arXiv2021

StructuralLM: Structural Pre-training for Form Understanding

Large pre-trained language models achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, they almost exclusively focus on text-only representation, while neglecting cell-level layout information that is important for form image understanding. In this paper, we propose a new pre-training approach, StructuralLM, to jointly leverage cell and layout information from scanned documents. Specifically, we pre-train StructuralLM with two new designs to make the most of the interactions of cell and layout information: 1) each cell as a semantic unit; 2) classification of cell positions. The pre-trained StructuralLM achieves new state-of-the-art results in different types of downstream tasks, including form understanding (from 78.95 to 85.14), document visual question answering (from 72.59 to 83.94) and document image classification (from 94.43 to 96.08).

cs.CL↗

arXiv2021

Improving Biomedical Pretrained Language Models with Knowledge

Pretrained language models have shown success in many natural language processing tasks. Many works explore incorporating knowledge into language models. In the biomedical domain, experts have taken decades of effort on building large-scale knowledge bases. For example, the Unified Medical Language System (UMLS) contains millions of entities with their synonyms and defines hundreds of relations among entities. Leveraging this knowledge can benefit a variety of downstream tasks such as named entity recognition and relation extraction. To this end, we propose KeBioLM, a biomedical pretrained language model that explicitly leverages knowledge from the UMLS knowledge bases. Specifically, we extract entities from PubMed abstracts and link them to UMLS. We then train a knowledge-aware language model that firstly applies a text-only encoding layer to learn entity representation and applies a text-entity fusion encoding to aggregate entity representation. Besides, we add two training objectives as entity detection and entity linking. Experiments on the named entity recognition and relation extraction from the BLURB benchmark demonstrate the effectiveness of our approach. Further analysis on a collected probing dataset shows that our model has better ability to model medical knowledge.

cs.CL↗

arXiv2021

Aspect Based Sentiment Analysis with Aspect-Specific Opinion Spans

Aspect based sentiment analysis, predicting sentiment polarity of given aspects, has drawn extensive attention. Previous attention-based models emphasize using aspect semantics to help extract opinion features for classification. However, these works are either not able to capture opinion spans as a whole, or not able to capture variable-length opinion spans. In this paper, we present a neat and effective structured attention model by aggregating multiple linear-chain CRFs. Such a design allows the model to extract aspect-specific opinion spans and then evaluate sentiment polarity by exploiting the extracted opinion features. The experimental results on four datasets demonstrate the effectiveness of the proposed model, and our analysis demonstrates that our model can capture aspect-specific opinion spans.

cs.CL↗

arXiv2021

Prototypical Representation Learning for Relation Extraction

Recognizing relations between entities is a pivotal task of relational learning. Learning relation representations from distantly-labeled datasets is difficult because of the abundant label noise and complicated expressions in human language. This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data that are effective in different settings, including supervised, distantly supervised, and few-shot learning. Instead of solely relying on the supervision from noisy labels, we propose to learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Prototypes are representations in the feature space abstracting the essential semantics of relations between entities in sentences. We learn prototypes based on objectives with clear geometric interpretation, where the prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered at the end of their corresponding prototype vectors on the surface of the ball. This approach allows us to learn meaningful, interpretable prototypes for the final classification. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art models. We further demonstrate the robustness of the encoder and the interpretability of prototypes with extensive experiments.

cs.CL↗

arXiv2021

Nuclear effects in neutrino-nucleus DIS and probe for short range correlation

We investigate charged-current neutrino-nucleus deep inelastic scattering with particular interests in relationship of short range correlation (SRC) and EMC effect. The structure functions $F^A_2(x,Q^2)$, $x F^A_3(x,Q^2)$ and ratios of differential cross sections are presented where the nuclei $A$ are chosen to be carbon, iron and lead. We propose a kind of universal modification functions which would provide a nontrivial test of SRC universality on the platform of neutrino-nucleus scattering and improve our understanding of nucleon structure substantially.

hep-ph↗

arXiv2021

Effects of $N(2000){5/2}^+$ on $γp \to K^+ Λ(1405)$

The photoproduction reaction of $γp \to K^+Λ(1405)$ is investigated based on an effective Lagrangian approach at the tree-level approximation with the purpose of understanding the reaction mechanism and extracting the resonance contents and the associated resonance parameters in this reaction. Apart from the $t$-channel $K$ and $K^\ast$ exchanges, $s$-channel nucleon ($N$) exchange, $u$-channel $Σ$, $Λ$, and $Λ(1405)$ exchanges, and generalized contact term, the exchanges of a minimum number of $N$ resonances in the $s$ channel are taken into account in constructing the reaction amplitudes to describe the experimental data. It is found that by introducing the $N(2000){5/2}^+$ resonance exchange in the $s$ channel, one can reproduce the most recent differential cross-section data from the CLAS Collaboration quite well. Further analysis shows that the cross sections of $γp \to K^+Λ(1405)$ at high energies are dominated by the $t$-channel $K$ exchange, while the contributions from the $s$-channel $N$ and $N(2000){5/2}^+$ exchanges are rather significant to the cross sections in the near-threshold energy region. Predictions for the beam and target asymmetries for $γp \to K^+Λ(1405)$ are given.

hep-ph↗

arXiv2021

LIGO as a probe of Dark Sectors

We show how current LIGO data is able to probe interesting theories beyond the Standard Model, particularly Dark Sectors where a Dark Higgs triggers symmetry breaking via a first-order phase transition. We use publicly available LIGO O2 data to illustrate how these sectors, even if disconnected from the Standard Model, can be probed by Gravitational Wave detectors. We link the LIGO measurements with the model content and mass scale of the Dark Sector, finding that current O2 data is testing a broad set of scenarios where the breaking of $SU(N)$ theories with $N_f$ fermions is triggered by a Dark Higgs at scales $Λ\simeq 10^8 - 10^9$ GeV with reasonable parameters for the scalar potential.

hep-ph↗

arXiv2021

Photoproduction $γp \to K^+Λ(1520)$ in an effective Lagrangian approach

The data on differential cross sections and photon-beam asymmetries for the $γp \to K^+Λ(1520)$ reaction have been analyzed within a tree-level effective Lagrangian approach. In addition to the $t$-channel $K$ and $K^\ast$ exchanges, the $u$-channel $Λ$ exchange, the $s$-channel nucleon exchange, and the interaction current, a minimal number of nucleon resonances in the $s$ channel are introduced in constructing the reaction amplitudes to describe the data. The results show that the experimental data can be well reproduced by including either the $N(2060)5/2^-$ or the $N(2120)3/2^-$ resonance. In both cases, the contact term and the $K$ exchange are found to make significant contributions, while the contributions from the $K^\ast$ and $Λ$ exchanges are negligible in the former case and considerable in the latter case. Measurements of the data on target asymmetries are called on to further pin down the resonance contents and to clarify the roles of the $K^\ast$ and $Λ$ exchanges in this reaction.

hep-ph↗

arXiv2021

Mortality Forecasting using Factor Models: Time-varying or Time-invariant Factor Loadings?

Many existing mortality models follow the framework of classical factor models, such as the Lee-Carter model and its variants. Latent common factors in factor models are defined as time-related mortality indices (such as $κ_t$ in the Lee-Carter model). Factor loadings, which capture the linear relationship between age variables and latent common factors (such as $β_x$ in the Lee-Carter model), are assumed to be time-invariant in the classical framework. This assumption is usually too restrictive in reality as mortality datasets typically span a long period of time. Driving forces such as medical improvement of certain diseases, environmental changes and technological progress may significantly influence the relationship of different variables. In this paper, we first develop a factor model with time-varying factor loadings (time-varying factor model) as an extension of the classical factor model for mortality modelling. Two forecasting methods to extrapolate the factor loadings, the local regression method and the naive method, are proposed for the time-varying factor model. From the empirical data analysis, we find that the new model can capture the empirical feature of time-varying factor loadings and improve mortality forecasting over different horizons and countries. Further, we propose a novel approach based on change point analysis to estimate the optimal `boundary' between short-term and long-term forecasting, which is favoured by the local linear regression and naive method, respectively. Additionally, simulation studies are provided to show the performance of the time-varying factor model under various scenarios.

stat.ME↗

arXiv2021

Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing

Semantic parsing has long been a fundamental problem in natural language processing. Recently, cross-domain context-dependent semantic parsing has become a new focus of research. Central to the problem is the challenge of leveraging contextual information of both natural language utterance and database schemas in the interaction history. In this paper, we present a dynamic graph framework that is capable of effectively modelling contextual utterances, tokens, database schemas, and their complicated interaction as the conversation proceeds. The framework employs a dynamic memory decay mechanism that incorporates inductive bias to integrate enriched contextual relation representation, which is further enhanced with a powerful reranking model. At the time of writing, we demonstrate that the proposed framework outperforms all existing models by large margins, achieving new state-of-the-art performance on two large-scale benchmarks, the SParC and CoSQL datasets. Specifically, the model attains a 55.8% question-match and 30.8% interaction-match accuracy on SParC, and a 46.8% question-match and 17.0% interaction-match accuracy on CoSQL.

cs.CL↗

arXiv2020

Nested Named Entity Recognition with Partially-Observed TreeCRFs

Named entity recognition (NER) is a well-studied task in natural language processing. However, the widely-used sequence labeling framework is difficult to detect entities with nested structures. In this work, we view nested NER as constituency parsing with partially-observed trees and model it with partially-observed TreeCRFs. Specifically, we view all labeled entity spans as observed nodes in a constituency tree, and other spans as latent nodes. With the TreeCRF we achieve a uniform way to jointly model the observed and the latent nodes. To compute the probability of partial trees with partial marginalization, we propose a variant of the Inside algorithm, the \textsc{Masked Inside} algorithm, that supports different inference operations for different nodes (evaluation for the observed, marginalization for the latent, and rejection for nodes incompatible with the observed) with efficient parallelized implementation, thus significantly speeding up training and inference. Experiments show that our approach achieves the state-of-the-art (SOTA) F1 scores on the ACE2004, ACE2005 dataset, and shows comparable performance to SOTA models on the GENIA dataset. Our approach is implemented at: \url{https://github.com/FranxYao/Partially-Observed-TreeCRFs}.

cs.CL↗

arXiv2020

Fuzzy model identification based on mixture distribution analysis for bearings remaining useful life estimation using small training data set

The research work presented in this paper proposes a data-driven modeling method for bearings remaining useful life estimation based on Takagi-Sugeno (T-S) fuzzy inference system (FIS). This method allows identifying the parameters of a classic T-S FIS, starting with a small quantity of data. In this work, we used the vibration signals data from a small number of bearings over an entire period of run-to-failure. The FIS model inputs are features extracted from the vibration signals data observed periodically on the training bearings. The number of rules and the input parameters of each rule of the FIS model are identified using the subtractive clustering method. Furthermore, we propose to use the maximum likelihood method of mixture distribution analysis to calculate the parameters of clusters on the time axis and the probability corresponding to rules on degradation stages. Based on this result, we identified the output parameters of each rule using a weighted least square estimation. We then benchmarked the proposed method with some existing methods from the literature, through numerical experiments conducted on available datasets to highlight its effectiveness.

eess.SP↗

arXiv2020

Bearings degradation monitoring indicators based on discarded projected space information and piecewise linear representation

Condition-based maintenance of rotating mechanics requests efficient bearings degradation monitoring. The accuracy of bearings degradation measure depends largely on degradation indicators. To extract efficient indicators, in this paper we propose a method based on the discarded projected space information and piecewise linear representation (PLR) to build three bearings degradation monitoring indicators which are named SDHT2, VSDHT2 and NVSDHT2. The discarded projected space information is measured by the segmented discarded Hotelling T square we propose in this paper. For illustration, the IEEE PHM 2012 benchmark dataset is used in this paper. The results show that the three new indicators are all sensitive and monotonic during the bearings whole lifecycle. They describe the whole degradation process history and carry the real-time information of bearings degradation. And NVSDHT2 is the generalised version of VSDHT2, which is promising to monitor bearings degradation.

cs.IR↗

arXiv2020

An Investigation of Potential Function Designs for Neural CRF

The neural linear-chain CRF model is one of the most widely-used approach to sequence labeling. In this paper, we investigate a series of increasingly expressive potential functions for neural CRF models, which not only integrate the emission and transition functions, but also explicitly take the representations of the contextual words as input. Our extensive experiments show that the decomposed quadrilinear potential function based on the vector representations of two neighboring labels and two neighboring words consistently achieves the best performance.

cs.CL↗

arXiv2020

Keyphrase Extraction with Dynamic Graph Convolutional Networks and Diversified Inference

Keyphrase extraction (KE) aims to summarize a set of phrases that accurately express a concept or a topic covered in a given document. Recently, Sequence-to-Sequence (Seq2Seq) based generative framework is widely used in KE task, and it has obtained competitive performance on various benchmarks. The main challenges of Seq2Seq methods lie in acquiring informative latent document representation and better modeling the compositionality of the target keyphrases set, which will directly affect the quality of generated keyphrases. In this paper, we propose to adopt the Dynamic Graph Convolutional Networks (DGCN) to solve the above two problems simultaneously. Concretely, we explore to integrate dependency trees with GCN for latent representation learning. Moreover, the graph structure in our model is dynamically modified during the learning process according to the generated keyphrases. To this end, our approach is able to explicitly learn the relations within the keyphrases collection and guarantee the information interchange between encoder and decoder in both directions. Extensive experiments on various KE benchmark datasets demonstrate the effectiveness of our approach.

cs.CL↗