REAP: Relation-Aware Elicitation and Parsing for Closed-Book Knowledge Base Construction from LLMs

Thanh-Dan Bui; Thanh-Trung Do; Tuan-Phong Nguyen

Home/Archive/arXiv

arXiv · 2608.10963

REAP: Relation-Aware Elicitation and Parsing for Closed-Book Knowledge Base Construction from LLMs

Thanh-Dan Bui·Thanh-Trung Do·Tuan-Phong Nguyen

View original source ↗Read source document ↗Export metadata ↓

Abstract

We present the REAP system for the AKBC Shared Task 2026 on constructing knowledge bases from language models in a closed-book setting, subject to a budget of at most 32B parameters and no model fine-tuning. Our system combines structured chain-of-thought reasoning, relation-specific query strategies, and a reasoning-based empty-set gate to elicit parametric knowledge, followed by direct extraction into valid JSON arrays. On the test set, the system, built on the Mistral-Small-24B-Instruct-2501 model, achieves a macro-F1 score of 0.62, with particularly strong results on countryLandBordersCountry (F1 = 0.95), companyTradesAtStockExchange (F1 = 0.73), and hasArea (F1 = 0.77). Our code is publicly available at https://github.com/yammdd/AKBC-Shared-Task-2026.

Explore related subjects

cs.CL

Keep this discovery

BibTeX RIS

Thanh-Dan Bui, Thanh-Trung Do, Tuan-Phong Nguyen. 2026-09-02. REAP: Relation-Aware Elicitation and Parsing for Closed-Book Knowledge Base Construction from LLMs. https://arxiv.org/abs/2608.10963

Cite the original work for its findings. Save a collection to share your selection of sources.

Discover connections

cs.CL subject

Connections use source metadata and explicit phrase matches, not verified experimental comparisons.

KEEP EXPLORING

Related discoveries

arXiv2024

Density Matrices for Metaphor Understanding

In physics, density matrices are used to represent mixed states, i.e. probabilistic mixtures of pure states. This concept has previously been used to model lexical ambiguity. In this paper, we consider metaphor as a type of lexical ambiguity, and examine whether metaphorical meaning can be effectively modelled using mixtures of word senses. We find that modelling metaphor is significantly more difficult than other kinds of lexical ambiguity, but that our best-performing density matrix method outperforms simple baselines as well as some neural language models.

cs.CL↗

arXiv2026

Informational Antilocality and the Locality Bias in LLMs

We consider the ability of transformer-based language models (LLMs) to learn what we call k-antilocal languages, i.e., languages that have no mutual information across any span of $k$ contiguous symbols. We construct such languages with increasing $k$, finding that LLMs trained on them achieve comparable cross-entropy loss regardless of antilocality, but converge more slowly on more antilocal languages. Our findings support the idea that non-local dependencies are more difficult to learn, but the evidence for this bias comes from learning speed rather than learning success.

cs.CL↗

arXiv2026

FLAME: A New Dataset on FLemish Accounts of Momentary Experiences

We introduce FLAME (FLemish Accounts of Momentary Experiences), a corpus of nearly 25,000 personal narratives in Belgian-Dutch (Flemish), collected through experience sampling to support Natural Language Processing (NLP) research on an underrepresented variety. Such everyday narratives are rich in culturally grounded themes, but their informal register and low-resource setting make thematic extraction hard. Comparing K-Means, LDA, and BERTopic, we find that human evaluation favors BERTopic, which produces the most coherent, culturally resonant topics. FLAME, thereby, offers a new resource for studying everyday language use in a low-resource variety.

cs.CL↗