arXiv subjects

Language models

Find arXiv papers on language models and computational linguistics, including cs.CL metadata. Use abstracts to explore model training, evaluation and natural-language processing.

Follow this topic with RSS

At least 55 records · Page 3Linked to original sources

arXiv2013

Recurrent Convolutional Neural Networks for Discourse Compositionality

The compositionality of meaning extends beyond the single sentence. Just as words combine to form the meaning of sentences, so do sentences combine to form the meaning of paragraphs, dialogues and general discourse. We introduce both a sentence model and a discourse model corresponding to the two levels of compositionality. The sentence model adopts convolution as the central operation for composing semantic vectors and is based on a novel hierarchical convolutional neural network. The discourse model extends the sentence model and is based on a recurrent neural network that is conditioned in a novel way both on the current sentence and on the current speaker. The discourse model is able to capture both the sequentiality of sentences and the interaction between different speakers. Without feature engineering or pretraining and with simple greedy decoding, the discourse model coupled to the sentence model obtains state of the art performance on a dialogue act classification experiment.

cs.CL↗

arXiv2013

An open diachronic corpus of historical Spanish: annotation criteria and automatic modernisation of spelling

The IMPACT-es diachronic corpus of historical Spanish compiles over one hundred books --containing approximately 8 million words-- in addition to a complementary lexicon which links more than 10 thousand lemmas with attestations of the different variants found in the documents. This textual corpus and the accompanying lexicon have been released under an open license (Creative Commons by-nc-sa) in order to permit their intensive exploitation in linguistic research. Approximately 7% of the words in the corpus (a selection aimed at enhancing the coverage of the most frequent word forms) have been annotated with their lemma, part of speech, and modern equivalent. This paper describes the annotation criteria followed and the standards, based on the Text Encoding Initiative recommendations, used to the represent the texts in digital form. As an illustration of the possible synergies between diachronic textual resources and linguistic research, we describe the application of statistical machine translation techniques to infer probabilistic context-sensitive rules for the automatic modernisation of spelling. The automatic modernisation with this type of statistical methods leads to very low character error rates when the output is compared with the supervised modern version of the text.

cs.CL↗

arXiv2013

Dialogue System: A Brief Review

A Dialogue System is a system which interacts with human in natural language. At present many universities are developing the dialogue system in their regional language. This paper will discuss about dialogue system, its components, challenges and its evaluation. This paper helps the researchers for getting info regarding dialogues system.

cs.CL↗

arXiv2013

Punjabi Language Interface to Database: a brief review

Unlike most user-computer interfaces, a natural language interface allows users to communicate fluently with a computer system with very little preparation. Databases are often hard to use in cooperating with the users because of their rigid interface. A good NLIDB allows a user to enter commands and ask questions in native language and then after interpreting respond to the user in native language. For a large number of applications requiring interaction between humans and the computer systems, it would be convenient to provide the end-user friendly interface. Punjabi language interface to database would proof fruitful to native people of Punjab, as it provides ease to them to use various e-governance applications like Punjab Sewa, Suwidha, Online Public Utility Forms, Online Grievance Cell, Land Records Management System,legacy matters, e-District, agriculture, etc. Punjabi is the mother tongue of more than 110 million people all around the world. According to available information, Punjabi ranks 10th from top out of a total of 6,900 languages recognized internationally by the United Nations. This paper covers a brief overview of the Natural language interface to database, its different components, its advantages, disadvantages, approaches and techniques used. The paper ends with the work done on Punjabi language interface to database and future enhancements that can be done.

cs.CL↗

arXiv2013

Supervised Topical Key Phrase Extraction of News Stories using Crowdsourcing, Light Filtering and Co-reference Normalization

Fast and effective automated indexing is critical for search and personalized services. Key phrases that consist of one or more words and represent the main concepts of the document are often used for the purpose of indexing. In this paper, we investigate the use of additional semantic features and pre-processing steps to improve automatic key phrase extraction. These features include the use of signal words and freebase categories. Some of these features lead to significant improvements in the accuracy of the results. We also experimented with 2 forms of document pre-processing that we call light filtering and co-reference normalization. Light filtering removes sentences from the document, which are judged peripheral to its main content. Co-reference normalization unifies several written forms of the same named entity into a unique form. We also needed a "Gold Standard" - a set of labeled documents for training and evaluation. While the subjective nature of key phrase selection precludes a true "Gold Standard", we used Amazon's Mechanical Turk service to obtain a useful approximation. Our data indicates that the biggest improvements in performance were due to shallow semantic features, news categories, and rhetorical signals (nDCG 78.47% vs. 68.93%). The inclusion of deeper semantic features such as Freebase sub-categories was not beneficial by itself, but in combination with pre-processing, did cause slight improvements in the nDCG scores.

cs.CL↗

arXiv2013

Key Phrase Extraction of Lightly Filtered Broadcast News

This paper explores the impact of light filtering on automatic key phrase extraction (AKE) applied to Broadcast News (BN). Key phrases are words and expressions that best characterize the content of a document. Key phrases are often used to index the document or as features in further processing. This makes improvements in AKE accuracy particularly important. We hypothesized that filtering out marginally relevant sentences from a document would improve AKE accuracy. Our experiments confirmed this hypothesis. Elimination of as little as 10% of the document sentences lead to a 2% improvement in AKE precision and recall. AKE is built over MAUI toolkit that follows a supervised learning approach. We trained and tested our AKE method on a gold standard made of 8 BN programs containing 110 manually annotated news stories. The experiments were conducted within a Multimedia Monitoring Solution (MMS) system for TV and radio news/programs, running daily, and monitoring 12 TV and 4 radio channels.

cs.CL↗

arXiv2013

Recognition of Named-Event Passages in News Articles

We extend the concept of Named Entities to Named Events - commonly occurring events such as battles and earthquakes. We propose a method for finding specific passages in news articles that contain information about such events and report our preliminary evaluation results. Collecting "Gold Standard" data presents many problems, both practical and conceptual. We present a method for obtaining such data using the Amazon Mechanical Turk service.

cs.CL↗

arXiv2013

Clinical Relationships Extraction Techniques from Patient Narratives

The Clinical E-Science Framework (CLEF) project was used to extract important information from medical texts by building a system for the purpose of clinical research, evidence-based healthcare and genotype-meets-phenotype informatics. The system is divided into two parts, one part concerns with the identification of relationships between clinically important entities in the text. The full parses and domain-specific grammars had been used to apply many approaches to extract the relationship. In the second part of the system, statistical machine learning (ML) approaches are applied to extract relationship. A corpus of oncology narratives that hand annotated with clinical relationships can be used to train and test a system that has been designed and implemented by supervised machine learning (ML) approaches. Many features can be extracted from these texts that are used to build a model by the classifier. Multiple supervised machine learning algorithms can be applied for relationship extraction. Effects of adding the features, changing the size of the corpus, and changing the type of the algorithm on relationship extraction are examined. Keywords: Text mining; information extraction; NLP; entities; and relations.

cs.IR↗

arXiv2013

Discriminative Training: Learning to Describe Video with Sentences, from Video Described with Sentences

We present a method for learning word meanings from complex and realistic video clips by discriminatively training (DT) positive sentential labels against negative ones, and then use the trained word models to generate sentential descriptions for new video. This new work is inspired by recent work which adopts a maximum likelihood (ML) framework to address the same problem using only positive sentential labels. The new method, like the ML-based one, is able to automatically determine which words in the sentence correspond to which concepts in the video (i.e., ground words to meanings) in a weakly supervised fashion. While both DT and ML yield comparable results with sufficient training data, DT outperforms ML significantly with smaller training sets because it can exploit negative training labels to better constrain the learning problem.

cs.CV↗

arXiv2013

Competency Tracking for English as a Second or Foreign Language Learners

My system utilizes the outcomes feature found in Moodle and other learning content management systems (LCMSs) to keep track of where students are in terms of what language competencies they have mastered and the competencies they need to get where they want to go. These competencies are based on the Common European Framework for (English) Language Learning. This data can be available for everyone involved with a given student's progress (e.g. educators, parents, supervisors and the students themselves). A given student's record of past accomplishments can also be meshed with those of his classmates. Not only are a student's competencies easily seen and tracked, educators can view competencies of a group of students that were achieved prior to enrollment in the class. This should make curriculum decision making easier and more efficient for educators.

cs.CL↗

arXiv2013

The DeLiVerMATH project - Text analysis in mathematics

A high-quality content analysis is essential for retrieval functionalities but the manual extraction of key phrases and classification is expensive. Natural language processing provides a framework to automatize the process. Here, a machine-based approach for the content analysis of mathematical texts is described. A prototype for key phrase extraction and classification of mathematical texts is presented.

cs.CL↗

arXiv2013

Semantics and pragmatics in actual software applications and in web search engines: exploring innovations

While new ways to use the Semantic Web are developed every week, which allow the user to find information on web more accurately - for example in search engines - some sophisticated pragmatic tools are becoming more important - for example in web interfaces known as Social Intelligence, or in the most famous Siri by Apple. The work aims to analyze whether and where we can identify the boundary between semantics and pragmatics in the software used by analyzed systems. examining how the linguistic disciplines are fundamental in their progress. Is it possible to assume that the tools of social intelligence have a pragmatic approach to the questions of the user, or it is just a use of a very rich vocabulary, with the use of semantic tools?

cs.IR↗

arXiv2013

WebSets: Extracting Sets of Entities from the Web Using Unsupervised Information Extraction

We describe a open-domain information extraction method for extracting concept-instance pairs from an HTML corpus. Most earlier approaches to this problem rely on combining clusters of distributionally similar terms and concept-instance pairs obtained with Hearst patterns. In contrast, our method relies on a novel approach for clustering terms found in HTML tables, and then assigning concept names to these clusters using Hearst patterns. The method can be efficiently applied to a large corpus, and experimental results on several datasets show that our method can accurately extract large numbers of concept-instance pairs.

cs.LG↗

arXiv2013

Improving Pointwise Mutual Information (PMI) by Incorporating Significant Co-occurrence

We design a new co-occurrence based word association measure by incorporating the concept of significant cooccurrence in the popular word association measure Pointwise Mutual Information (PMI). By extensive experiments with a large number of publicly available datasets we show that the newly introduced measure performs better than other co-occurrence based measures and despite being resource-light, compares well with the best known resource-heavy distributional similarity and knowledge based word association measures. We investigate the source of this performance improvement and find that of the two types of significant co-occurrence - corpus-level and document-level, the concept of corpus level significance combined with the use of document counts in place of word counts is responsible for all the performance gains observed. The concept of document level significance is not helpful for PMI adaptation.

cs.CL↗

arXiv2013

Intelligent Hybrid Man-Machine Translation Quality Estimation

Inferring evaluation scores based on human judgments is invaluable compared to using current evaluation metrics which are not suitable for real-time applications e.g. post-editing. However, these judgments are much more expensive to collect especially from expert translators, compared to evaluation based on indicators contrasting source and translation texts. This work introduces a novel approach for quality estimation by combining learnt confidence scores from a probabilistic inference model based on human judgments, with selective linguistic features-based scores, where the proposed inference model infers the credibility of given human ranks to solve the scarcity and inconsistency issues of human judgments. Experimental results, using challenging language-pairs, demonstrate improvement in correlation with human judgments over traditional evaluation metrics.

cs.CL↗

arXiv2013

Genetic approach for arabic part of speech tagging

With the growing number of textual resources available, the ability to understand them becomes critical. An essential first step in understanding these sources is the ability to identify the part of speech in each sentence. Arabic is a morphologically rich language, wich presents a challenge for part of speech tagging. In this paper, our goal is to propose, improve and implement a part of speech tagger based on a genetic alorithm. The accuracy obtained with this method is comparable to that of other probabilistic approaches.

cs.CL↗

arXiv2018

Discovering User Groups for Natural Language Generation

We present a model which predicts how individual users of a dialog system understand and produce utterances based on user groups. In contrast to previous work, these user groups are not specified beforehand, but learned in training. We evaluate on two referring expression (RE) generation tasks; our experiments show that our model can identify user groups and learn how to most effectively talk to them, and can dynamically assign unseen users to the correct groups as they interact with the system.

cs.CL↗

arXiv2018

The Historical Significance of Textual Distances

Measuring similarity is a basic task in information retrieval, and now often a building-block for more complex arguments about cultural change. But do measures of textual similarity and distance really correspond to evidence about cultural proximity and differentiation? To explore that question empirically, this paper compares textual and social measures of the similarities between genres of English-language fiction. Existing measures of textual similarity (cosine similarity on tf-idf vectors or topic vectors) are also compared to new strategies that use supervised learning to anchor textual measurement in a social context.

cs.CL↗