Automated Discovery of Mathematical Definitions in Text with Deep Neural Networks

Abstract

Automatic definition extraction from texts is an important task that has numerous applications in several natural language processing fields such as summarization, analysis of scientific texts, automatic taxonomy generation, ontology generation, concept identification, and question answering. For definitions that are contained within a single sentence, this problem can be viewed as a binary classification of sentences into definitions and non-definitions. In this paper, we focus on automatic detection of one-sentence definitions in mathematical texts, which are difficult to separate from surrounding text. We experiment with several data representations, which include sentence syntactic structure and word embeddings, and apply deep learning methods such as the Convolutional Neural Network (CNN) and the Long Short-Term Memory network (LSTM), in order to identify mathematical definitions. Our experiments demonstrate the superiority of CNN and its combination with LSTM, when applied on the syntactically-enriched input representation. We also present a new dataset for definition extraction from mathematical texts. We demonstrate that this dataset is beneficial for training supervised models aimed at extraction of mathematical definitions. Our experiments with different domains demonstrate that mathematical definitions require special treatment, and that using cross-domain learning is inefficient for that task.