arXiv subjects

Jiarui Zhang

Publications and source records attributed to Jiarui Zhang.

79 records · Page 5Linked to original sources

arXiv2022

An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs

Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models, in zero-shot evaluation on various downstream language reasoning tasks. Since these improvements are reported in aggregate, however, little is known about (i) how to select the appropriate knowledge for solid performance across tasks, (ii) how to combine this knowledge with neural language models, and (iii) how these pairings affect granular task performance. In this paper, we study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models. We study the effect of different synthetic datasets on language models with various architectures and sizes. The resulting models are evaluated against four task properties: domain overlap, answer similarity, vocabulary overlap, and answer length. Our experiments show that encoder-decoder models benefit from more data to learn from, whereas sampling strategies that balance across different aspects yield best performance. Most of the improvement occurs on questions with short answers and dissimilar answer candidates, which corresponds to the characteristics of the data used for pre-training.

cs.CL↗

arXiv2022

Viscous dissipation in the fluid core of the Moon

The spin axes of the mantle, fluid core and solid inner core of the Moon precess at frequency $Ω_p=2π/18.6$ yr$^{-1}$ though with different orientations, leading to viscous friction at the core-mantle boundary (CMB) and inner core boundary (ICB). Here, we use a rotational model of the Moon with a range of inner core and outer core radii to investigate the relative importance of viscous dissipation at the CMB and ICB, and to show how this dissipation is connected to the phase lead angle ($ϕ_p$) of the mantle ahead of its Cassini state. We show that when the inner core radius is $>80$ km and the free inner core nutation frequency $Ω_{ficn}$ approaches $Ω_p$, viscous dissipation at the ICB can be comparable to that at the CMB, and in the most extreme cases exceed it by as much as a factor 10. If so, the viscous dissipation in the lunar core projected back in time depends on how $Ω_{ficn}$ has evolved relative to $Ω_p$. We further show that constraints on the CMB and ICB radii of the lunar core can in principle be extracted by matching the observed phase lead of $ϕ_p=0.27$ arcsec; this requires an improved estimate of tidal dissipation and an accurate model of the turbulent viscous torque. Lastly, when our rotational model is constrained to match $ϕ_p=0.27$ arcsec, our results suggest that the viscous dissipation at the ICB is likely insufficient to have ever been above the threshold to power a thermally driven dynamo.

astro-ph.EP↗

arXiv2021

CCPM: A Chinese Classical Poetry Matching Dataset

Poetry is one of the most important art forms of human languages. Recently many studies have focused on incorporating some linguistic features of poetry, such as style and sentiment, into its understanding or generation system. However, there is no focus on understanding or evaluating the semantics of poetry. Therefore, we propose a novel task to assess a model's semantic understanding of poetry by poem matching. Specifically, this task requires the model to select one line of Chinese classical poetry among four candidates according to the modern Chinese translation of a line of poetry. To construct this dataset, we first obtain a set of parallel data of Chinese classical poetry and modern Chinese translation. Then we retrieve similar lines of poetry with the lines in a poetry corpus as negative choices. We name the dataset Chinese Classical Poetry Matching Dataset (CCPM) and release it at https://github.com/THUNLP-AIPoet/CCPM. We hope this dataset can further enhance the study on incorporating deep semantics into the understanding and generation system of Chinese classical poetry. We also preliminarily run two variants of BERT on this dataset as the baselines for this dataset.

cs.CL↗

arXiv2020

Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation ($\text{POINT}^2$)

We propose to tackle the problem of multiview 2D/3D rigid registration for intervention via a Point-Of-Interest Network for Tracking and Triangulation ($\text{POINT}^2$). $\text{POINT}^2$ learns to establish 2D point-to-point correspondences between the pre- and intra-intervention images by tracking a set of random POIs. The 3D pose of the pre-intervention volume is then estimated through a triangulation layer. In $\text{POINT}^2$, the unified framework of the POI tracker and the triangulation layer enables learning informative 2D features and estimating 3D pose jointly. In contrast to existing approaches, $\text{POINT}^2$ only requires a single forward-pass to achieve a reliable 2D/3D registration. As the POI tracker is shift-invariant, $\text{POINT}^2$ is more robust to the initial pose of the 3D pre-intervention image. Extensive experiments on a large-scale clinical cone-beam CT (CBCT) dataset show that the proposed $\text{POINT}^2$ method outperforms the existing learning-based method in terms of accuracy, robustness and running time. Furthermore, when used as an initial pose estimator, our method also improves the robustness and speed of the state-of-the-art optimization-based approaches by ten folds.

cs.CV↗

arXiv2020

Modeling Discourse Structure for Document-level Neural Machine Translation

Recently, document-level neural machine translation (NMT) has become a hot topic in the community of machine translation. Despite its success, most of existing studies ignored the discourse structure information of the input document to be translated, which has shown effective in other tasks. In this paper, we propose to improve document-level NMT with the aid of discourse structure information. Our encoder is based on a hierarchical attention network (HAN). Specifically, we first parse the input document to obtain its discourse structure. Then, we introduce a Transformer-based path encoder to embed the discourse structure information of each word. Finally, we combine the discourse structure information with the word embedding before it is fed into the encoder. Experimental results on the English-to-German dataset show that our model can significantly outperform both Transformer and Transformer+HAN.

cs.CL↗

arXiv2018

Piecewise interlacing zeros of polynomials

We introduce the concept of piecewise interlacing zeros for studying the relation of root distribution of two polynomials. The concept is pregnant with an idea of confirming the real-rootedness of polynomials in a sequence. Roughly speaking, one constructs a collection of disjoint intervals such that one may show by induction that consecutive polynomials have interlacing zeros over each of the intervals. We confirm the real-rootedness of some polynomials satisfying a recurrence with linear polynomial coefficients. This extends Gross et al.'s work where one of the polynomial coefficients is a constant.

math.CO↗

arXiv2017

Root geometry of polynomial sequences III: Type $(1,1)$ with positive coefficients

In this paper, we study the root distribution of some univariate polynomials $W_n(z)$ satisfying a recurrence of order two with linear polynomial coefficients over positive numbers. We discover a sufficient and necessary condition for the overall real-rootedness of all the polynomials, in terms of the polynomial coefficients of the recurrence. Moreover, in the real-rooted case, we find the set of limits of zeros, which turns out to be the union of a closed interval and one or two isolated points; when non-real-rooted polynomial exists, we present a sufficient condition under which every polynomial with $n$ large has a real zero.

math.CO↗