Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.13607

BLiMPPI: Benchmark of Linguistic Minimal Pairs for Polarity Items

Arzu Burcu Güven·Anna Rogers·Leonie Weissweiler·Joakim Nivre·Rob van der Goot
View original source ↗Read source document ↗Export metadata ↓

Abstract

Negative polarity items (NPIs), such as "anymore" and "ever", are grammatical only in select environments, predominantly in the context of negation. Because their grammaticality depends on both syntax and semantics, they have a rich, multilingual literature in theoretical linguistics, yet work on NPIs in language models is almost entirely confined to English. We introduce BLiMPPI, a multilingual evaluation suite of over 12,000 NPI minimal pairs in 33 languages, built by a pipeline that discovers NPIs in Universal Dependencies (UD) treebanks, labels their licensing environments, and perturbs these into minimal pairs. Evaluating 12 multilingual and 2 monolingual models alongside n-gram baselines, we find that multilingual models struggle most with lower-resource languages and with pairs that are not n-gram solvable. We further examine the effects of NPI typology, language family, licensing variability, and NPI-licenser distance. Data: https://github.com/arzuburcuguven/blimppi-data

Explore related subjects

cs.CL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Arzu Burcu Güven, Anna Rogers, Leonie Weissweiler, Joakim Nivre, Rob van der Goot. 2026-10-09. BLiMPPI: Benchmark of Linguistic Minimal Pairs for Polarity Items. https://arxiv.org/abs/2610.13607

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Retelling

Counterfactual storytelling exposes LLM shortcomings in causal reasoning, where few-shot prompting and SFT fail to produce logical and rational narratives. This case study introduces Retell, Reward, Repeat (RRR), a narrative theory-informed RL pipeline that teaches causal story structure. We extend an existing counterfactual stories dataset with human-annotated stages of narrative equilibrium, synthesising Structuralist Narratology with scalar narrativity. An aligned LLM-as-judge automates this framework to output reward signals, teaching the policy LLM to generate texts with causal narrative structure. Evaluations show that RRR-trained 7/8B LLMs outperform few-shot and SFT baselines in logic, rationality, and completeness, also aligning with a blind human preference subset. RRR offers a linguistically grounded, cost-effective post-training mechanism for LLM storytelling using a small, query-only dataset, showing the value of integrating linguistic theories into contemporary NLP.

cs.CL↗
arXiv2026

Role-aware Heuristic Episodic Attention for Conversational LLMs

Large language models often lose track of persistent instructions and relevant information as multi-turn conversations grow. We study this cumulative contextual decay through three related failure modes: attention pollution, dilution, and drift. We propose REA (Role-aware Heuristic Episodic Attention), a context-management framework that assigns different persistence and representation policies to instructions and episodic interactions. Instructional Memory retains identified global constraints in a dedicated prefix. Episodic Memory preserves user inputs and compresses model replies, while heuristic retrieval selects raw text, compressed representations, or omission for each historical turn. On Long-MT-Bench+, REA improves the judge score from 6.32 to 7.36 on a 10-point scale, a 16.5% relative gain over the Vanilla baseline, and reduces average latency by 2.91$\times$. Additional evaluations show aggregate gains on three backbones spanning 1.7B-7B parameters and on Chinese and English role-playing tasks. These results support role-aware context management as a practical approach to maintaining conversational continuity and instruction adherence.

cs.CL↗
arXiv2026

The ÌròyìnSpeech Text Corpus: 24,905 Curated Yorùbá Sentences for Speech and Language Technology

ÌròyìnSpeech is a 42-hour, 80-speaker Yorùbá read-speech corpus whose audio has been distributed by ELRA since 2024. This paper describes the release of its text component: 24,905 unique, hand-verified, tone-marked Yorùbá sentences (275,897 tokens; 15,687 types), curated in 2022 as recording prompts. Roughly 11,000 sentences were adapted from openly licensed news material; the remainder were written in-house to broaden coverage beyond the religious translation that dominates existing Yorùbá corpora. Every sentence was checked by hand for tone-mark accuracy, edited for read-aloud clarity and a neutral register, and localised so that non-Yorùbá personal and place names appear in Yorùbá form. Preparing the text for release surfaced systematic Unicode normalisation failures affecting more than 60% of lines (with precomposed and decomposed forms of the same letter co-occurring within single sentences) which we document and correct. The corpus supports diacritic restoration, grapheme-to-phoneme conversion, TTS front-end development and orthographic research, and serves as a validated prompt set for new recording.

cs.CL↗