Search arXiv⌕ Search

arXiv subjects

Yu Zhao

Publications and source records attributed to Yu Zhao.

At least 253 records · Page 14Linked to original sources
arXiv2022

Learning Bi-typed Multi-relational Heterogeneous Graph via Dual Hierarchical Attention Networks

Bi-type multi-relational heterogeneous graph (BMHG) is one of the most common graphs in practice, for example, academic networks, e-commerce user behavior graph and enterprise knowledge graph. It is a critical and challenge problem on how to learn the numerical representation for each node to characterize subtle structures. However, most previous studies treat all node relations in BMHG as the same class of relation without distinguishing the different characteristics between the intra-class relations and inter-class relations of the bi-typed nodes, causing the loss of significant structure information. To address this issue, we propose a novel Dual Hierarchical Attention Networks (DHAN) based on the bi-typed multi-relational heterogeneous graphs to learn comprehensive node representations with the intra-class and inter-class attention-based encoder under a hierarchical mechanism. Specifically, the former encoder aggregates information from the same type of nodes, while the latter aggregates node representations from its different types of neighbors. Moreover, to sufficiently model node multi-relational information in BMHG, we adopt a newly proposed hierarchical mechanism. By doing so, the proposed dual hierarchical attention operations enable our model to fully capture the complex structures of the bi-typed multi-relational heterogeneous graphs. Experimental results on various tasks against the state-of-the-arts sufficiently confirm the capability of DHAN in learning node representations on the BMHGs.

cs.LG↗
arXiv2022

Stock Movement Prediction Based on Bi-typed Hybrid-relational Market Knowledge Graph via Dual Attention Networks

Stock Movement Prediction (SMP) aims at predicting listed companies' stock future price trend, which is a challenging task due to the volatile nature of financial markets. Recent financial studies show that the momentum spillover effect plays a significant role in stock fluctuation. However, previous studies typically only learn the simple connection information among related companies, which inevitably fail to model complex relations of listed companies in the real financial market. To address this issue, we first construct a more comprehensive Market Knowledge Graph (MKG) which contains bi-typed entities including listed companies and their associated executives, and hybrid-relations including the explicit relations and implicit relations. Afterward, we propose DanSmp, a novel Dual Attention Networks to learn the momentum spillover signals based upon the constructed MKG for stock prediction. The empirical experiments on our constructed datasets against nine SOTA baselines demonstrate that the proposed DanSmp is capable of improving stock prediction with the constructed MKG.

q-fin.ST↗
arXiv2022

Improving Human-Object Interaction Detection via Phrase Learning and Label Composition

Human-Object Interaction (HOI) detection is a fundamental task in high-level human-centric scene understanding. We propose PhraseHOI, containing a HOI branch and a novel phrase branch, to leverage language prior and improve relation expression. Specifically, the phrase branch is supervised by semantic embeddings, whose ground truths are automatically converted from the original HOI annotations without extra human efforts. Meanwhile, a novel label composition method is proposed to deal with the long-tailed problem in HOI, which composites novel phrase labels by semantic neighbors. Further, to optimize the phrase branch, a loss composed of a distilling loss and a balanced triplet loss is proposed. Extensive experiments are conducted to prove the effectiveness of the proposed PhraseHOI, which achieves significant improvement over the baseline and surpasses previous state-of-the-art methods on Full and NonRare on the challenging HICO-DET benchmark.

cs.CV↗
arXiv2022

Differentiable Channel Sparsity Search via Weight Sharing within Filters

In this paper, we propose the differentiable channel sparsity search (DCSS) for convolutional neural networks. Unlike traditional channel pruning algorithms which require users to manually set prune ratios for each convolutional layer, DCSS automatically searches the optimal combination of sparsities. Inspired by the differentiable architecture search (DARTS), we draw lessons from the continuous relaxation and leverage the gradient information to balance the computational cost and metrics. Since directly applying the scheme of DARTS causes shape mismatching and excessive memory consumption, we introduce a novel technique called weight sharing within filters. This technique elegantly eliminates the problem of shape mismatching with negligible additional resources. We conduct comprehensive experiments on not only image classification but also find-grained tasks including semantic segmentation and image super resolution to verify the effectiveness of DCSS. Compared with previous network pruning approaches, DCSS achieves state-of-the-art results for image classification. Experimental results of semantic segmentation and image super resolution indicate that task-specific search achieves better performance than transferring slim models, demonstrating the wide applicability and high efficiency of DCSS.

cs.CV↗
arXiv2022

On the equilibrium of the Poisson-Nernst-Planck-Bikermann model equipping with the steric and correlation effects

The Poisson-Nernst-Planck-Bikermann (PNPB) model, in which the ions and water molecules are treated as different species with non-uniform sizes and valences with interstitial voids, can describe the steric and correlation effects in ionic solution neglected by the Poisson-Nernst-Planck and Poisson-Boltzmann theories with point charge assumption. In the PNPB model, the electric potential is governed by the fourth-order Poisson-Bikermann (4PBik) equation instead of the Poisson equation so that it can describe the correlation effect. What's more, the steric potential is included in the ionic and water fluxes as well as the equilibrium Fermi-like distributions which characterizes the steric effect quantitatively. In this work, after doing a nondimensionalization step, we analyze the self-adjointness and the kernel of the fourth-order operator of the 4PBik equation. Also, we show the positivity of the void volume function and the convexity of the free energy. Following these properties, the well-posedness of the PNPB model in equilibrium is given. Furthermore, because the PNPB model has an energy dissipated structure, we adopt a finite volume scheme which preserves the energy dissipated property at the semi-discrete level. After that, various numerical investigations are given to show the parameter dependence of the steric effect to the steady state.

math-ph↗
arXiv2021

QEMind: Alibaba's Submission to the WMT21 Quality Estimation Shared Task

Quality Estimation, as a crucial step of quality control for machine translation, has been explored for years. The goal is to investigate automatic methods for estimating the quality of machine translation results without reference translations. In this year's WMT QE shared task, we utilize the large-scale XLM-Roberta pre-trained model and additionally propose several useful features to evaluate the uncertainty of the translations to build our QE system, named \textit{QEMind}. The system has been applied to the sentence-level scoring task of Direct Assessment and the binary score prediction task of Critical Error Detection. In this paper, we present our submissions to the WMT 2021 QE shared task and an extensive set of experimental results have shown us that our multilingual systems outperform the best system in the Direct Assessment QE task of WMT 2020.

cs.CL↗
arXiv2021

MSHT: Multi-stage Hybrid Transformer for the ROSE Image Analysis of Pancreatic Cancer

Pancreatic cancer is one of the most malignant cancers in the world, which deteriorates rapidly with very high mortality. The rapid on-site evaluation (ROSE) technique innovates the workflow by immediately analyzing the fast stained cytopathological images with on-site pathologists, which enables faster diagnosis in this time-pressured process. However, the wider expansion of ROSE diagnosis has been hindered by the lack of experienced pathologists. To overcome this problem, we propose a hybrid high-performance deep learning model to enable the automated workflow, thus freeing the occupation of the valuable time of pathologists. By firstly introducing the Transformer block into this field with our particular multi-stage hybrid design, the spatial features generated by the convolutional neural network (CNN) significantly enhance the Transformer global modeling. Turning multi-stage spatial features as global attention guidance, this design combines the robustness from the inductive bias of CNN with the sophisticated global modeling power of Transformer. A dataset of 4240 ROSE images is collected to evaluate the method in this unexplored field. The proposed multi-stage hybrid Transformer (MSHT) achieves 95.68% in classification accuracy, which is distinctively higher than the state-of-the-art models. Facing the need for interpretability, MSHT outperforms its counterparts with more accurate attention regions. The results demonstrate that the MSHT can distinguish cancer samples accurately at an unprecedented image scale, laying the foundation for deploying automatic decision systems and enabling the expansion of ROSE in clinical practice. The code and records are available at: https://github.com/sagizty/Multi-Stage-Hybrid-Transformer.

eess.IV↗
arXiv2021

Deep Keyphrase Completion

Keyphrase provides accurate information of document content that is highly compact, concise, full of meanings, and widely used for discourse comprehension, organization, and text retrieval. Though previous studies have made substantial efforts for automated keyphrase extraction and generation, surprisingly, few studies have been made for \textit{keyphrase completion} (KPC). KPC aims to generate more keyphrases for document (e.g. scientific publication) taking advantage of document content along with a very limited number of known keyphrases, which can be applied to improve text indexing system, etc. In this paper, we propose a novel KPC method with an encoder-decoder framework. We name it \textit{deep keyphrase completion} (DKPC) since it attempts to capture the deep semantic meaning of the document content together with known keyphrases via a deep learning framework. Specifically, the encoder and the decoder in DKPC play different roles to make full use of the known keyphrases. The former considers the keyphrase-guiding factors, which aggregates information of known keyphrases into context. On the contrary, the latter considers the keyphrase-inhibited factor to inhibit semantically repeated keyphrase generation. Extensive experiments on benchmark datasets demonstrate the efficacy of our proposed model.

cs.IR↗
arXiv2021

Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translation

Quality Estimation (QE) plays an essential role in applications of Machine Translation (MT). Traditionally, a QE system accepts the original source text and translation from a black-box MT system as input. Recently, a few studies indicate that as a by-product of translation, QE benefits from the model and training data's information of the MT system where the translations come from, and it is called the "glass-box QE". In this paper, we extend the definition of "glass-box QE" generally to uncertainty quantification with both "black-box" and "glass-box" approaches and design several features deduced from them to blaze a new trial in improving QE's performance. We propose a framework to fuse the feature engineering of uncertainty quantification into a pre-trained cross-lingual language model to predict the translation quality. Experiment results show that our method achieves state-of-the-art performances on the datasets of WMT 2020 QE shared task.

cs.CL↗
arXiv2021

A Spontaneously Formed Plasmonic-MoTe2 Hybrid Platform for Ultrasensitive Raman Enhancement

To develop highly sensitive, stable and repeatable surface-enhanced Raman scattering (SERS) substrates is crucial for analytical detection, which is a challenge for traditional metallic structures. Herein, by taking advantage of the high surface activity of 1T' transition metal telluride, we have fabricated high-density gold nanoparticles (AuNPs) that are spontaneously in-situ prepared on the 1T' MoTe2 atomic layers via a facile method, forming a plasmonic-2D material hybrid SERS substrate. This AuNP formation is unique to the 1T' phase, which is repressed in 2H MoTe2 with less surface activity. The hybrid structure generates coupling effects of electromagnetic and chemical enhancements, as well as excellent molecule adsorption, leading to the ultrasensitive (4*10^-17 M) and reproducible detection. Additionally, the immense fluorescence and photobleaching phenomena are mostly avoided. Flexible SERS tapes have been demonstrated in practical applications. Our approach facilitates the ultrasensitive SERS detection by a facile method, as well as the better mechanistic understanding of SERS beyond plasmonic effects.

physics.optics↗
arXiv2021

GlyphCRM: Bidirectional Encoder Representation for Chinese Character with its Glyph

Previous works indicate that the glyph of Chinese characters contains rich semantic information and has the potential to enhance the representation of Chinese characters. The typical method to utilize the glyph features is by incorporating them into the character embedding space. Inspired by previous methods, we innovatively propose a Chinese pre-trained representation model named as GlyphCRM, which abandons the ID-based character embedding method yet solely based on sequential character images. We render each character into a binary grayscale image and design two-channel position feature maps for it. Formally, we first design a two-layer residual convolutional neural network, namely HanGlyph to generate the initial glyph representation of Chinese characters, and subsequently adopt multiple bidirectional encoder Transformer blocks as the superstructure to capture the context-sensitive information. Meanwhile, we feed the glyph features extracted from each layer of the HanGlyph module into the underlying Transformer blocks by skip-connection method to fully exploit the glyph features of Chinese characters. As the HanGlyph module can obtain a sufficient glyph representation of any Chinese character, the long-standing out-of-vocabulary problem could be effectively solved. Extensive experimental results indicate that GlyphCRM substantially outperforms the previous BERT-based state-of-the-art model on 9 fine-tuning tasks, and it has strong transferability and generalization on specialized fields and low-resource tasks. We hope this work could spark further research beyond the realms of well-established representation of Chinese texts.

cs.AI↗
arXiv2021

Design Study of APSU Type Hybrid MBA Lattice for Mid Energy DLSR

In recent years, a new generation of storage ring based light sources, known as diffraction limited storage rings (DLSRs), whose emittance approaches the diffraction limit for the range of X ray wavelengths of interest to the scientific community, has garnered significant attention worldwide. Researchers have begun to design and build DLSRs. Among various DLSR proposals, the hybrid multibend achromat (HMBA) lattice enables sextupole strengths to be maintained at a reasonable level when minimizing the emittance; hence, it has been adopted in many DLSR designs. Based on the H7BA lattice, the design of the Advanced Photon Source Upgrade Project (APSU) can effectively reduce emittance by replacing six quadrupoles with antibends. Herein, we discuss the feasibility of designing an APSU type HMBA lattice for the Southern Advanced Photon Source, a mid energy DLSR light source with ultralow emittance that has been proposed to be built adjacent to the China Spallation Neutron Source. Both linear and nonlinear dynamics are optimized to obtain a detailed design of this type of lattice. The emittance is minimized, while a sufficiently large dynamic aperture (DA) and momentum acceptance (MA) are maintained. A design comprising 36 APSU type H7BAs, with an energy of 3 GeV and a circumference of 972 m, is achieved. The horizontal natural emittance is 20 pm rad, with a horizontal DA of 5.8 mm, a vertical DA of 4.5 mm, and an MA of 4%, as well as a long longitudinal damping time of 120 ms. Subsequently, a few modifications are performed based on the APSU type lattice to reduce the longitudinal damping time from 120 to 44 ms while maintaining other performance parameters at the same level.

physics.acc-ph↗
arXiv2021

Feedback Graph Attention Convolutional Network for Medical Image Enhancement

Artifacts, blur and noise are the common distortions degrading MRI images during the acquisition process, and deep neural networks have been demonstrated to help in improving image quality. To well exploit global structural information and texture details, we propose a novel biomedical image enhancement network, named Feedback Graph Attention Convolutional Network (FB-GACN). As a key innovation, we consider the global structure of an image by building a graph network from image sub-regions that we consider to be node features, linking them non-locally according to their similarity. The proposed model consists of three main parts: 1) The parallel graph similarity branch and content branch, where the graph similarity branch aims at exploiting the similarity and symmetry across different image sub-regions in low-resolution feature space and provides additional priors for the content branch to enhance texture details. 2) A feedback mechanism with a recurrent structure to refine low-level representations with high-level information and generate powerful high-level texture details by handling the feedback connections. 3) A reconstruction to remove the artifacts and recover super-resolution images by using the estimated sub-region correlation priors obtained from the graph similarity branch. We evaluate our method on two image enhancement tasks: i) cross-protocol super resolution of diffusion MRI; ii) artifact removal of FLAIR MR images. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art methods.

eess.IV↗
arXiv2021

End-to-End Human Object Interaction Detection with HOI Transformer

We propose HOI Transformer to tackle human object interaction (HOI) detection in an end-to-end manner. Current approaches either decouple HOI task into separated stages of object detection and interaction classification or introduce surrogate interaction problem. In contrast, our method, named HOI Transformer, streamlines the HOI pipeline by eliminating the need for many hand-designed components. HOI Transformer reasons about the relations of objects and humans from global image context and directly predicts HOI instances in parallel. A quintuple matching loss is introduced to force HOI predictions in a unified way. Our method is conceptually much simpler and demonstrates improved accuracy. Without bells and whistles, HOI Transformer achieves $26.61\% $ $ AP $ on HICO-DET and $52.9\%$ $AP_{role}$ on V-COCO, surpassing previous methods with the advantage of being much simpler. We hope our approach will serve as a simple and effective alternative for HOI tasks. Code is available at https://github.com/bbepoch/HoiTransformer .

cs.CV↗
arXiv2020

Field model for complex ionic fluids: analytical properties and numerical investigation

In this paper, we consider the field model for complex ionic fluids with an energy variational structure, and analyze the well-posedness to this model with regularized kernels. Furthermore, we deduce the estimate of the maximal density function to quantify the finite size effect. On the numerical side, we adopt a finite volume scheme to the field model, which satisfies the following properties: positivity-preserving, mass conservation and energy dissipation. Besides, series of numerical experiments are provided to demonstrate the properties of the steady state and the finite size effect by showing the equilibrium profiles with different values of the parameter in the kernel.

math.AP↗
arXiv2020

The Lightest 2D Nanomaterial: Freestanding Ultrathin Li Nanosheets by in-situ Electron Microscopy

Lithium (Li) is the simplest metal and the lightest solid element. Here we report the first demonstration of controlled growth of two-dimensional (2D) ultrathin Li nanosheets with large lateral dimensions up to several hundreds of nanometres and thickness limited to just a few nanometres by in-situ transmission electron microscopy (TEM). The nanoscale dynamics of nanosheets growth were unravelled by real-time TEM imaging, which, in combination with density function theory (DFT) calculations indicates that the growth of bcc structured Li into 2D nanosheets is a consequence of kinetic control as mediated by preferential oxidization of the (111) surfaces due to the trace amount of O2 (~10-6 Pa) within TEM chamber. The plasmonic optical properties of the as-grown Li nanosheets were probed by cathodoluminescence (CL) spectroscopy equipped within TEM, and a broadband visible emission was observed that contains contributions of both in-plane and out-of-plane plasmon resonance modes.

physics.chem-ph↗
arXiv2020

A Categorical Quantum Toroidal Action on Hilbert Schemes

We categorify the commutation of Nakajima's Heisenberg operators $P_{\pm 1}$ and their infinitely many counterparts in the quantum toroidal algebra $U_{q_1,q_2}(\ddot{gl_1})$ acting on the Grothendieck groups of Hilbert schemes. By combining our result with arxiv:1804.03645 , one obtains a geometric categorical $U_{q_1,q_2}(\ddot{gl_1})$ action on the derived category of Hilbert schemes. Our main technical tool is a detailed geometric study of certain nested Hilbert schemes of triples and quadruples, through the lens of the minimal model program, by showing that these nested Hilbert schemes are either canonical or semi-divisorial log terminal singularities.

math.AG↗