arXiv subjects

Fei Huang

Publications and source records attributed to Fei Huang.

At least 469 records · Page 26Linked to original sources

arXiv2021

SU(3) symmetry and its breaking effects in semileptonic heavy baryon decays

We employ the flavor SU(3) symmetry to analyze semileptonic decays of anti-triplet charmed baryons ($Λ^{+}_{c},Ξ_{c}^{+,0}$) and find that the experimental data on ${\cal B}(Λ_c\toΛ\ell^+ν_{\ell})$ implies $\mathcal{B}(Ξ_c^0\toΞ^-e^+ν_{e})=(4.10\pm0.46)\%$ and $\mathcal{B}(Ξ_c^0\toΞ^-μ^+ν_μ)= (3.98\pm0.57)\%$. When this prediction is confronted with recent experimental results from Belle collaboration $\mathcal{B}(Ξ_c^0\toΞ^-e^+ν_e)=1.31(04\pm07\pm38)\%$ and $\mathcal{B}(Ξ_c^0\toΞ^-μ^+ν_μ)=1.27(06\pm10\pm 37)\%$, it is found that the SU(3) symmetry is severely broken. We then consider the generic SU(3) breaking effects in these decays and find that the data can be accommodated in different scenarios but the breaking effects are inevitably large. In some interesting scenarios, we also explore the testable implications in these scenarios which can be tested with more data become available. Similar analyses are carried out for semileptonic anti-triplet beauty baryon to octet baryons and anti-triplet charmed baryons. The validity of SU(3) for these decays can also be examined when data become available.

hep-ph↗

arXiv2021

Learning to Ask for Data-Efficient Event Argument Extraction

Event argument extraction (EAE) is an important task for information extraction to discover specific argument roles. In this study, we cast EAE as a question-based cloze task and empirically analyze fixed discrete token template performance. As generating human-annotated question templates is often time-consuming and labor-intensive, we further propose a novel approach called "Learning to Ask," which can learn optimized question templates for EAE without human annotations. Experiments using the ACE-2005 dataset demonstrate that our method based on optimized questions achieves state-of-the-art performance in both the few-shot and supervised settings.

cs.CL↗

arXiv2021

DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddings

Learning sentence embeddings from dialogues has drawn increasing attention due to its low annotation cost and high domain adaptability. Conventional approaches employ the siamese-network for this task, which obtains the sentence embeddings through modeling the context-response semantic relevance by applying a feed-forward network on top of the sentence encoders. However, as the semantic textual similarity is commonly measured through the element-wise distance metrics (e.g. cosine and L2 distance), such architecture yields a large gap between training and evaluating. In this paper, we propose DialogueCSE, a dialogue-based contrastive learning approach to tackle this issue. DialogueCSE first introduces a novel matching-guided embedding (MGE) mechanism, which generates a context-aware embedding for each candidate response embedding (i.e. the context-free embedding) according to the guidance of the multi-turn context-response matching matrices. Then it pairs each context-aware embedding with its corresponding context-free embedding and finally minimizes the contrastive loss across all pairs. We evaluate our model on three multi-turn dialogue datasets: the Microsoft Dialogue Corpus, the Jing Dong Dialogue Corpus, and the E-commerce Dialogue Corpus. Evaluation results show that our approach significantly outperforms the baselines across all three datasets in terms of MAP and Spearman's correlation measures, demonstrating its effectiveness. Further quantitative experiments show that our approach achieves better performance when leveraging more dialogue context and remains robust when less training data is provided.

cs.CL↗

arXiv2021

Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning

Recent pretrained language models extend from millions to billions of parameters. Thus the need to fine-tune an extremely large pretrained model with a limited training corpus arises in various downstream tasks. In this paper, we propose a straightforward yet effective fine-tuning technique, Child-Tuning, which updates a subset of parameters (called child network) of large pretrained models via strategically masking out the gradients of the non-child network during the backward process. Experiments on various downstream tasks in GLUE benchmark show that Child-Tuning consistently outperforms the vanilla fine-tuning by 1.5~8.6 average score among four different pretrained models, and surpasses the prior fine-tuning techniques by 0.6~1.3 points. Furthermore, empirical results on domain transfer and task transfer show that Child-Tuning can obtain better generalization performance by large margins.

cs.CL↗

arXiv2021

MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

Entity retrieval, which aims at disambiguating mentions to canonical entities from massive KBs, is essential for many tasks in natural language processing. Recent progress in entity retrieval shows that the dual-encoder structure is a powerful and efficient framework to nominate candidates if entities are only identified by descriptions. However, they ignore the property that meanings of entity mentions diverge in different contexts and are related to various portions of descriptions, which are treated equally in previous works. In this work, we propose Multi-View Entity Representations (MuVER), a novel approach for entity retrieval that constructs multi-view representations for entity descriptions and approximates the optimal view for mentions via a heuristic searching method. Our method achieves the state-of-the-art performance on ZESHEL and improves the quality of candidates on three standard Entity Linking datasets

cs.CL↗

arXiv2021

Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training

Translating e-commercial product descriptions, a.k.a product-oriented machine translation (PMT), is essential to serve e-shoppers all over the world. However, due to the domain specialty, the PMT task is more challenging than traditional machine translation problems. Firstly, there are many specialized jargons in the product description, which are ambiguous to translate without the product image. Secondly, product descriptions are related to the image in more complicated ways than standard image descriptions, involving various visual aspects such as objects, shapes, colors or even subjective styles. Moreover, existing PMT datasets are small in scale to support the research. In this paper, we first construct a large-scale bilingual product description dataset called Fashion-MMT, which contains over 114k noisy and 40k manually cleaned description translations with multiple product images. To effectively learn semantic alignments among product images and bilingual texts in translation, we design a unified product-oriented cross-modal cross-lingual model (\upoc~) for pre-training and fine-tuning. Experiments on the Fashion-MMT and Multi30k datasets show that our model significantly outperforms the state-of-the-art models even pre-trained on the same dataset. It is also shown to benefit more from large-scale noisy data to improve the translation quality. We will release the dataset and codes at https://github.com/syuqings/Fashion-MMT.

cs.CV↗

arXiv2021

Document-level Relation Extraction as Semantic Segmentation

Document-level relation extraction aims to extract relations among multiple entity pairs from a document. Previously proposed graph-based or transformer-based models utilize the entities independently, regardless of global information among relational triples. This paper approaches the problem by predicting an entity-level relation matrix to capture local and global information, parallel to the semantic segmentation task in computer vision. Herein, we propose a Document U-shaped Network for document-level relation extraction. Specifically, we leverage an encoder module to capture the context information of entities and a U-shaped segmentation module over the image-style feature map to capture global interdependency among triples. Experimental results show that our approach can obtain state-of-the-art performance on three benchmark datasets DocRED, CDR, and GDA.

cs.CL↗

arXiv2021

Angular Distributions for Multi-body Semileptonic Charmed Baryon Decays

We perform an analysis of angular distributions in semileptonic decays of charmed baryons $B_1^{(\prime)}\to B_2^{(\prime)}(\to B_3^{(\prime)}B_4^{(\prime)})\ell^+ν_{\ell}$, where the $B_1=(Λ_c^+,Ξ_c^{(0,+)})$ are the SU(3)-antitriplet baryons and $B_1'=Ω_c^-$ is an SU(3) sextet. We will firstly derive analytic expressions for angular distributions using helicity amplitude technique. Based on the lattice QCD results for $Λ_c^+\toΛ$ and $Ξ_c^0\toΞ^-$ form factors and model calculation of the $Ω_c^0\toΩ^-$ transition, we predict branching fractions: $\mathcal{B}(Λ_{c}^{+} \rightarrow p π^{-} e^{+} ν_{e})=2.48(15)\%$, $\mathcal{B}(Λ_{c}^+\rightarrow p π^{-}μ^{+}ν_μ)=2.50(14)\%$, $\mathcal{B}(Ξ_{c}\rightarrow Λπ^{-}e^{+}ν_{e})=2.40(30)\%$, $\mathcal{B}(Ξ_{c}\rightarrow Λπ^{-}μ^{+}ν_ν)=2.41(30)\%$, $\mathcal{B}(Ω_{c}\rightarrow ΛK^{-}e^{+}ν_{e})=0.362(14)\%$, $\mathcal{B}(Ω_{c}\rightarrow ΛK^{-}μ^{+}ν_ν)=0.350(14)\%$. Besides, we also predict the $q^2$-dependence and angular distributions of these processes, in particular the coefficients for the $\cos nθ_{\ell}$ ($\cos nθ_{h}$, $\cos nϕ$) $(n=0, 1, 2, \cdots)$ terms. This work can provide a theoretical basis for the ongoing experiments at BESIII, LHCb and BELLE-II.

hep-ph↗

arXiv2021

Deciphering Weak Decays of Triply Heavy Baryons by SU(3) Analysis

Baryons with three heavy quarks are the last missing pieces of the lowest-lying baryon multiplets in the quark model after the discovery of doubly heavy baryons. In this work, we study nonleptonic weak decays of triply heavy baryons $Ω_{ccc}^{++}$, $Ω_{bbb}^{-}$, $Ω_{ccb}^{+}$, and $Ω_{cbb}^{0}$. Decay amplitudes for various processes have been parametrized in terms of the SU(3) irreducible nonperturbative amplitudes. A number of relations for the partial decay widths can be deduced from these results that can be examined in future. Some decay channels and cascade decay modes which likely to be used to reconstruct the triply heavy baryons have been also listed.

hep-ph↗

arXiv2021

Hidden charm pentaquark states in a diquark model

The mass spectrum of hidden charm pentaquark states composed of two diquarks and an antiquark are calculated by use of an effective Hamiltonian which includes explicitly the spin, color, and flavor dependent interactions. The results show that the $P_c(4312)^+$ and $P_c(4440)^+$ states could be explained as hidden charm pentaquark states with isospin and spin-parity $IJ^P=1/2\left(3/2^-\right)$, the $P_c(4457)^+$ state could be explained as a hidden charm pentaquark state with $IJ^P=1/2\left(5/2^-\right)$, and the $P_{cs}(4459)^+$ state could be explained as a hidden charm pentaquark state with $IJ^P=0\left(1/2^-\right)$ or $0\left(3/2^-\right)$. Predications for the masses of other possible pentaquark states are also given, and the possible decay channels of these hidden charm pentaquark states are discussed.

hep-ph↗

arXiv2021

Relational Learning with Gated and Attentive Neighbor Aggregator for Few-Shot Knowledge Graph Completion

Aiming at expanding few-shot relations' coverage in knowledge graphs (KGs), few-shot knowledge graph completion (FKGC) has recently gained more research interests. Some existing models employ a few-shot relation's multi-hop neighbor information to enhance its semantic representation. However, noise neighbor information might be amplified when the neighborhood is excessively sparse and no neighbor is available to represent the few-shot relation. Moreover, modeling and inferring complex relations of one-to-many (1-N), many-to-one (N-1), and many-to-many (N-N) by previous knowledge graph completion approaches requires high model complexity and a large amount of training instances. Thus, inferring complex relations in the few-shot scenario is difficult for FKGC models due to limited training instances. In this paper, we propose a few-shot relational learning with global-local framework to address the above issues. At the global stage, a novel gated and attentive neighbor aggregator is built for accurately integrating the semantics of a few-shot relation's neighborhood, which helps filtering the noise neighbors even if a KG contains extremely sparse neighborhoods. For the local stage, a meta-learning based TransH (MTransH) method is designed to model complex relations and train our model in a few-shot learning fashion. Extensive experiments show that our model outperforms the state-of-the-art FKGC approaches on the frequently-used benchmark datasets NELL-One and Wiki-One. Compared with the strong baseline model MetaR, our model achieves 5-shot FKGC performance improvements of 8.0% on NELL-One and 2.8% on Wiki-One by the metric Hits@10.

cs.AI↗

arXiv2021

E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning

Vision-language pre-training (VLP) on large-scale image-text pairs has achieved huge success for the cross-modal downstream tasks. The most existing pre-training methods mainly adopt a two-step training procedure, which firstly employs a pre-trained object detector to extract region-based visual features, then concatenates the image representation and text embedding as the input of Transformer to train. However, these methods face problems of using task-specific visual representation of the specific object detector for generic cross-modal understanding, and the computation inefficiency of two-stage pipeline. In this paper, we propose the first end-to-end vision-language pre-trained model for both V+L understanding and generation, namely E2E-VLP, where we build a unified Transformer framework to jointly learn visual representation, and semantic alignments between image and text. We incorporate the tasks of object detection and image captioning into pre-training with a unified Transformer encoder-decoder architecture for enhancing visual learning. An extensive set of experiments have been conducted on well-established vision-language downstream tasks to demonstrate the effectiveness of this novel VLP paradigm.

cs.CV↗

arXiv2021

NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer

Autoregressive models have been widely used in unsupervised text style transfer. Despite their success, these models still suffer from the content preservation problem that they usually ignore part of the source sentence and generate some irrelevant words with strong styles. In this paper, we propose a Non-Autoregressive generator for unsupervised text Style Transfer (NAST), which alleviates the problem from two aspects. First, we observe that most words in the transferred sentence can be aligned with related words in the source sentence, so we explicitly model word alignments to suppress irrelevant words. Second, existing models trained with the cycle loss align sentences in two stylistic text spaces, which lacks fine-grained control at the word level. The proposed non-autoregressive generator focuses on the connections between aligned words, which learns the word-level transfer between styles. For experiments, we integrate the proposed generator into two base models and evaluate them on two style transfer tasks. The results show that NAST can significantly improve the overall performance and provide explainable word alignments. Moreover, the non-autoregressive generator achieves over 10x speedups at inference. Our codes are available at https://github.com/thu-coai/NAST.

cs.CL↗

arXiv2021

More Embeddings, Better Sequence Labelers?

Recent work proposes a family of contextual embeddings that significantly improves the accuracy of sequence labelers over non-contextual embeddings. However, there is no definite conclusion on whether we can build better sequence labelers by combining different kinds of embeddings in various settings. In this paper, we conduct extensive experiments on 3 tasks over 18 datasets and 8 languages to study the accuracy of sequence labeling with various embedding concatenations and make three observations: (1) concatenating more embedding variants leads to better accuracy in rich-resource and cross-domain settings and some conditions of low-resource settings; (2) concatenating additional contextual sub-word embeddings with contextual character embeddings hurts the accuracy in extremely low-resource settings; (3) based on the conclusion of (1), concatenating additional similar contextual embeddings cannot lead to further improvements. We hope these conclusions can help people build stronger sequence labelers in various settings.

cs.CL↗

arXiv2021

Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor

Knowledge distillation is a critical technique to transfer knowledge between models, typically from a large model (the teacher) to a more fine-grained one (the student). The objective function of knowledge distillation is typically the cross-entropy between the teacher and the student's output distributions. However, for structured prediction problems, the output space is exponential in size; therefore, the cross-entropy objective becomes intractable to compute and optimize directly. In this paper, we derive a factorized form of the knowledge distillation objective for structured prediction, which is tractable for many typical choices of the teacher and student models. In particular, we show the tractability and empirical effectiveness of structural knowledge distillation between sequence labeling and dependency parsing models under four different scenarios: 1) the teacher and student share the same factorization form of the output structure scoring function; 2) the student factorization produces more fine-grained substructures than the teacher factorization; 3) the teacher factorization produces more fine-grained substructures than the student factorization; 4) the factorization forms from the teacher and the student are incompatible.

cs.CL↗

arXiv2021

VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation

Existing work in multilingual pretraining has demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages. However, much of this work only relies on the shared vocabulary and bilingual contexts to encourage the correlation across languages, which is loose and implicit for aligning the contextual representations between languages. In this paper, we plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. More importantly, when fine-tuning on downstream tasks, the cross-attention module can be plugged in or out on-demand, thus naturally benefiting a wider range of cross-lingual tasks, from language understanding to generation. As a result, the proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark, covering text classification, sequence labeling, question answering, and sentence retrieval. For cross-lingual generation tasks, it also outperforms all existing cross-lingual models and state-of-the-art Transformer variants on WMT14 English-to-German and English-to-French translation datasets, with gains of up to 1~2 BLEU.

cs.CL↗

arXiv2021

Automated Concatenation of Embeddings for Structured Prediction

Pretrained contextualized embeddings are powerful word representations for structured prediction tasks. Recent work found that better word representations can be obtained by concatenating different types of embeddings. However, the selection of embeddings to form the best concatenated representation usually varies depending on the task and the collection of candidate embeddings, and the ever-increasing number of embedding types makes it a more difficult problem. In this paper, we propose Automated Concatenation of Embeddings (ACE) to automate the process of finding better concatenations of embeddings for structured prediction tasks, based on a formulation inspired by recent progress on neural architecture search. Specifically, a controller alternately samples a concatenation of embeddings, according to its current belief of the effectiveness of individual embedding types in consideration for a task, and updates the belief based on a reward. We follow strategies in reinforcement learning to optimize the parameters of the controller and compute the reward based on the accuracy of a task model, which is fed with the sampled concatenation as input and trained on a task dataset. Empirical results on 6 tasks and 21 datasets show that our approach outperforms strong baselines and achieves state-of-the-art performance with fine-tuned embeddings in all the evaluations.

cs.CL↗

arXiv2021

Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Tracking

Existing dialog state tracking (DST) models are trained with dialog data in a random order, neglecting rich structural information in a dataset. In this paper, we propose to use curriculum learning (CL) to better leverage both the curriculum structure and schema structure for task-oriented dialogs. Specifically, we propose a model-agnostic framework called Schema-aware Curriculum Learning for Dialog State Tracking (SaCLog), which consists of a preview module that pre-trains a DST model with schema information, a curriculum module that optimizes the model with CL, and a review module that augments mispredicted data to reinforce the CL training. We show that our proposed approach improves DST performance over both a transformer-based and RNN-based DST model (TripPy and TRADE) and achieves new state-of-the-art results on WOZ2.0 and MultiWOZ2.1.

cs.CL↗