arXiv subjects

Yue Zhang

Publications and source records attributed to Yue Zhang.

At least 613 records · Page 34Linked to original sources

arXiv2022

Multi-Scale Context-Guided Lumbar Spine Disease Identification with Coarse-to-fine Localization and Classification

Accurate and efficient lumbar spine disease identification is crucial for clinical diagnosis. However, existing deep learning models with millions of parameters often fail to learn with only hundreds or dozens of medical images. These models also ignore the contextual relationship between adjacent objects, such as between vertebras and intervertebral discs. This work introduces a multi-scale context-guided network with coarse-to-fine localization and classification, named CCF-Net, for lumbar spine disease identification. Specifically, in learning, we divide the localization objective into two parallel tasks, coarse and fine, which are more straightforward and effectively reduce the number of parameters and computational cost. The experimental results show that the coarse-to-fine design presents the potential to achieve high performance with fewer parameters and data requirements. Moreover, the multi-scale context-guided module can significantly improve the performance by 6.45% and 5.51% with ResNet18 and ResNet50, respectively. Our code is available at https://github.com/czifan/CCFNet.pytorch.

cs.CV↗

arXiv2022

Towards Robust Online Dialogue Response Generation

Although pre-trained sequence-to-sequence models have achieved great success in dialogue response generation, chatbots still suffer from generating inconsistent responses in real-world practice, especially in multi-turn settings. We argue that this can be caused by a discrepancy between training and real-world testing. At training time, chatbot generates the response with the golden context, while it has to generate based on the context consisting of both user utterances and the model predicted utterances during real-world testing. With the growth of the number of utterances, this discrepancy becomes more serious in the multi-turn settings. In this paper, we propose a hierarchical sampling-based method consisting of both utterance-level sampling and semi-utterance-level sampling, to alleviate the discrepancy, which implicitly increases the dialogue coherence. We further adopt reinforcement learning and re-ranking methods to explicitly optimize the dialogue coherence during training and inference, respectively. Empirical experiments show the effectiveness of the proposed methods for improving the robustness of chatbots in real practice.

cs.CL↗

arXiv2022

Neutrino Self-Interactions: A White Paper

Neutrinos are the Standard Model (SM) particles which we understand the least, often due to how weakly they interact with the other SM particles. Beyond this, very little is known about interactions among the neutrinos, i.e., their self-interactions. The SM predicts neutrino self-interactions at a level beyond any current experimental capabilities, leaving open the possibility for beyond-the-SM interactions across many energy scales. In this white paper, we review the current knowledge of neutrino self-interactions from a vast array of probes, from cosmology, to astrophysics, to the laboratory. We also discuss theoretical motivations for such self-interactions, including neutrino masses and possible connections to dark matter. Looking forward, we discuss the capabilities of searches in the next generation and beyond, highlighting the possibility of future discovery of this beyond-the-SM physics.

hep-ph↗

arXiv2022

Do Prompts Solve NLP Tasks Using Natural Language?

Thanks to the advanced improvement of large pre-trained language models, prompt-based fine-tuning is shown to be effective on a variety of downstream tasks. Though many prompting methods have been investigated, it remains unknown which type of prompts are the most effective among three types of prompts (i.e., human-designed prompts, schema prompts and null prompts). In this work, we empirically compare the three types of prompts under both few-shot and fully-supervised settings. Our experimental results show that schema prompts are the most effective in general. Besides, the performance gaps tend to diminish when the scale of training data grows large.

cs.CL↗

arXiv2022

Revisiting QMIX: Discriminative Credit Assignment by Gradient Entropy Regularization

In cooperative multi-agent systems, agents jointly take actions and receive a team reward instead of individual rewards. In the absence of individual reward signals, credit assignment mechanisms are usually introduced to discriminate the contributions of different agents so as to achieve effective cooperation. Recently, the value decomposition paradigm has been widely adopted to realize credit assignment, and QMIX has become the state-of-the-art solution. In this paper, we revisit QMIX from two aspects. First, we propose a new perspective on credit assignment measurement and empirically show that QMIX suffers limited discriminability on the assignment of credits to agents. Second, we propose a gradient entropy regularization with QMIX to realize a discriminative credit assignment, thereby improving the overall performance. The experiments demonstrate that our approach can comparatively improve learning efficiency and achieve better performance.

cs.AI↗

arXiv2022

NumHTML: Numeric-Oriented Hierarchical Transformer Model for Multi-task Financial Forecasting

Financial forecasting has been an important and active area of machine learning research because of the challenges it presents and the potential rewards that even minor improvements in prediction accuracy or forecasting may entail. Traditionally, financial forecasting has heavily relied on quantitative indicators and metrics derived from structured financial statements. Earnings conference call data, including text and audio, is an important source of unstructured data that has been used for various prediction tasks using deep earning and related approaches. However, current deep learning-based methods are limited in the way that they deal with numeric data; numbers are typically treated as plain-text tokens without taking advantage of their underlying numeric structure. This paper describes a numeric-oriented hierarchical transformer model to predict stock returns, and financial risk using multi-modal aligned earnings calls data by taking advantage of the different categories of numbers (monetary, temporal, percentages etc.) and their magnitude. We present the results of a comprehensive evaluation of NumHTML against several state-of-the-art baselines using a real-world publicly available dataset. The results indicate that NumHTML significantly outperforms the current state-of-the-art across a variety of evaluation metrics and that it has the potential to offer significant financial gains in a practical trading context.

cs.LG↗

arXiv2022

Analysis of lane-change conflict between cars and trucks at merging section using UAV video data

The freeway on-ramp merging section is often identified as a crash-prone spot due to the high frequency of traffic conflicts. Very few traffic conflict analysis studies comprehensively consider different vehicle types at freeway merging section. Thus, the main objective of this study is to analyse conflicts between different vehicle types at freeway merging section. Field data are collected by Unmanned Aerial Vehicle (UAV) at merging areas in Shanghai, China. Vehicle extraction method is utilized to obtain vehicle trajectories. Time-to-collision (TTC) is utilized as the surrogate safety measure. TTC of car-car conflicts are the smallest while TTC of truck-truck conflicts are the largest. Traffic conflicts frequently occur at on-ramp and acceleration lane. Results show the spatial distribution of lane-change conflicts is significantly different between different vehicle types, suggesting that vehicle drivers should maintain safe distance especially car drivers. Besides, in order to decrease lane-change conflict at merging area, traffic management agencies are suggested to change dotted lie to solid lane at the beginning of acceleration lane.

cs.RO↗

arXiv2021

Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter

Lexicon information and pre-trained models, such as BERT, have been combined to explore Chinese sequence labelling tasks due to their respective strengths. However, existing methods solely fuse lexicon features via a shallow and random initialized sequence layer and do not integrate them into the bottom layers of BERT. In this paper, we propose Lexicon Enhanced BERT (LEBERT) for Chinese sequence labelling, which integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer. Compared with the existing methods, our model facilitates deep lexicon knowledge fusion at the lower layers of BERT. Experiments on ten Chinese datasets of three tasks including Named Entity Recognition, Word Segmentation, and Part-of-Speech tagging, show that LEBERT achieves the state-of-the-art results.

cs.CL↗

arXiv2021

Multi-Domain Spoken Language Understanding Using Domain- and Task-Aware Parameterization

Spoken language understanding has been addressed as a supervised learning problem, where a set of training data is available for each domain. However, annotating data for each domain is both financially costly and non-scalable so we should fully utilize information across all domains. One existing approach solves the problem by conducting multi-domain learning, using shared parameters for joint training across domains. We propose to improve the parameterization of this method by using domain-specific and task-specific model parameters to improve knowledge learning and transfer. Experiments on 5 domains show that our model is more effective for multi-domain SLU and obtain the best results. In addition, we show its transferability by outperforming the prior best model by 12.4\% when adapting to a new domain with little data.

cs.CL↗

arXiv2021

SemGloVe: Semantic Co-occurrences for GloVe from BERT

GloVe learns word embeddings by leveraging statistical information from word co-occurrence matrices. However, word pairs in the matrices are extracted from a predefined local context window, which might lead to limited word pairs and potentially semantic irrelevant word pairs. In this paper, we propose SemGloVe, which distills semantic co-occurrences from BERT into static GloVe word embeddings. Particularly, we propose two models to extract co-occurrence statistics based on either the masked language model or the multi-head attention weights of BERT. Our methods can extract word pairs without limiting by the local window assumption and can define the co-occurrence weights by directly considering the semantic distance between word pairs. Experiments on several word similarity datasets and four external tasks show that SemGloVe can outperform GloVe.

cs.CL↗

arXiv2021

Speeding Up Dark Matter With Solar Neutrinos

We present a novel mechanism of using solar neutrinos to speed up dark matter, inspired by the fact that neutrinos are the most energetic particles from the Sun with a well-understood spectrum. In a neutrino portal dark sector model, we show that dark matter with sub-GeV mass could be accelerated by the $pp$ neutrinos to velocities well above $10^{-3}c$ and capable of depositing large enough energy at direct detection experiments. A crucial ingredient of this mechanism is the dissociation of stable dark matter bound states that exist in Nature. The resulting dark matter velocity distribution bears a strong resemblance in shape to the solar neutrino spectrum. As an application, we derive a leading limit on light dark matter interaction by reinterpreting a recent PICO experiment result.

hep-ph↗

arXiv2021

Fast determination of thickness through scanning moiré fringe in scanning transmission electron microscopy

Sample thickness is an important parameter in transmission electron microscopy (TEM) imaging, for interpreting image contrast and understanding the relationship between properties and microstructure. In this study, we introduce a method for determining thickness in scanning transmission electron microscopy (STEM) mode based on scanning moiré fringe (SMF). Sample thickness can be determined in situ in the medium magnification using focal-series SMF imaging, with beam damage and contamination avoided to a large extent. This method provides a fast and convenient way for determining thickness in TEM imaging, which is particularly useful for beam-sensitive materials such as Metal-Organic Frameworks.

cond-mat.mtrl-sci↗

arXiv2021

Mass-energy equivalence for terahertz magnon excitation in antiferromagnetic domain walls

The theory of special relativity is one of the most significant achievements in modern physics, with several important predictions such as time dilation, size contraction for a moving object and mass-energy equivalence. Recent studies have demonstrated size contraction for an antiferromagnetic (AFM) domain wall (DW). Here, we show the mass-energy equivalence by numerically investigating the excitation of terahertz (THz) magnons from a moving AFM DW under the magnetic anisotropy energy gradient. The energy of magnons comes from the loss of DW mass, accompanied with a DW width broadening, overcoming the Lorentz contraction effect. Our results pave the way to study relativistic physics in AFM textures and to efficiently generate THz magnons by electric means.

cond-mat.mes-hall↗

arXiv2021

Probing Neutrino-Portal Dark Matter at the Forward Physics Facility

The Forward Physics Facility (FPF), planned to operate near the ATLAS interaction point at the LHC, offers exciting new terrain to explore neutrino properties at TeV energy scales. It will reach an unprecedented regime for terrestrial neutrino experiments and provide the opportunity to reveal new physics of neutrinos at higher energy scales. We demonstrate that future detectors at the FPF have the potential to discover new mediators that couple predominantly to neutrinos, with masses between 0.3 and 20 GeV and small couplings not yet probed by existing searches. Such a neutrinophilic mediator is well motivated for addressing the origin of several neutrino-portal dark matter candidates, including thermal freeze-out and sterile-neutrino dark matter scenarios. Experimentally, the corresponding signatures include neutrino charged-current scattering events associated with large missing transverse momentum, and excessive apparent tau-neutrino events. We discuss the FPF detector capabilities needed for this search, most importantly the hadronic energy resolution.

hep-ph↗

arXiv2021

Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning

To quantitatively and intuitively explore the generalization ability of pre-trained language models (PLMs), we have designed several tasks of arithmetic and logical reasoning. We both analyse how well PLMs generalize when the test data is in the same distribution as the train data and when it is different, for the latter analysis, we have also designed a cross-distribution test set other than the in-distribution test set. We conduct experiments on one of the most advanced and publicly released generative PLM - BART. Our research finds that the PLMs can easily generalize when the distribution is the same, however, it is still difficult for them to generalize out of the distribution.

cs.CL↗

arXiv2021

Entity Relation Extraction as Dependency Parsing in Visually Rich Documents

Previous works on key information extraction from visually rich documents (VRDs) mainly focus on labeling the text within each bounding box (i.e., semantic entity), while the relations in-between are largely unexplored. In this paper, we adapt the popular dependency parsing model, the biaffine parser, to this entity relation extraction task. Being different from the original dependency parsing model which recognizes dependency relations between words, we identify relations between groups of words with layout information instead. We have compared different representations of the semantic entity, different VRD encoders, and different relation decoders. The results demonstrate that our proposed model achieves 65.96% F1 score on the FUNSD dataset. As for the real-world application, our model has been applied to the in-house customs data, achieving reliable performance in the production setting.

cs.CL↗

arXiv2021

PolyNet: Polynomial Neural Network for 3D Shape Recognition with PolyShape Representation

3D shape representation and its processing have substantial effects on 3D shape recognition. The polygon mesh as a 3D shape representation has many advantages in computer graphics and geometry processing. However, there are still some challenges for the existing deep neural network (DNN)-based methods on polygon mesh representation, such as handling the variations in the degree and permutations of the vertices and their pairwise distances. To overcome these challenges, we propose a DNN-based method (PolyNet) and a specific polygon mesh representation (PolyShape) with a multi-resolution structure. PolyNet contains two operations; (1) a polynomial convolution (PolyConv) operation with learnable coefficients, which learns continuous distributions as the convolutional filters to share the weights across different vertices, and (2) a polygonal pooling (PolyPool) procedure by utilizing the multi-resolution structure of PolyShape to aggregate the features in a much lower dimension. Our experiments demonstrate the strength and the advantages of PolyNet on both 3D shape classification and retrieval tasks compared to existing polygon mesh-based methods and its superiority in classifying graph representations of images. The code is publicly available from https://myavartanoo.github.io/polynet/.

cs.CV↗

arXiv2021

Solving Aspect Category Sentiment Analysis as a Text Generation Task

Aspect category sentiment analysis has attracted increasing research attention. The dominant methods make use of pre-trained language models by learning effective aspect category-specific representations, and adding specific output layers to its pre-trained representation. We consider a more direct way of making use of pre-trained language models, by casting the ACSA tasks into natural language generation tasks, using natural language sentences to represent the output. Our method allows more direct use of pre-trained knowledge in seq2seq language models by directly following the task setting during pre-training. Experiments on several benchmarks show that our method gives the best reported results, having large advantages in few-shot and zero-shot settings.

cs.CL↗