Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.13648

SPARQL-LLM in the TEXT2SPARQL 2026 Challenge

Panayiotis Smeros·Vincent Emonet·Ruijie Wang·Ana-Claudia Sima·Tarcisio Mendes de Farias
View original source ↗Read source document ↗Export metadata ↓

Abstract

Querying distributed data represented as Knowledge Graphs (KGs) via SPARQL remains a difficult task, requiring fluency in query syntax, endpoint specifications, and schema details. Knowledge Graph Question Answering (KGQA) systems address this by automatically translating natural language into SPARQL, and recent advances in Large Language Models (LLMs) have made this translation increasingly feasible. In this paper, we describe SPARQL-LLM, our open-source KGQA system, in the context of the TEXT2SPARQL 2026 challenge. In this challenge, SPARQL-LLM ranks first overall among eight competing approaches across all datasets, achieving up to 59% higher accuracy than the second-best approach and supporting multilingual question answering in English, German, and Spanish. SPARQL-LLM is publicly released at https://github.com/sib-swiss/sparql-llm and deployed in production over decentralized bioinformatics KGs at https://www.expasy.org/chat.

Explore related subjects

cs.IR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Panayiotis Smeros, Vincent Emonet, Ruijie Wang, Ana-Claudia Sima, Tarcisio Mendes de Farias. 2026-10-09. SPARQL-LLM in the TEXT2SPARQL 2026 Challenge. https://arxiv.org/abs/2610.13648

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

LLMRec: Benchmarking Large Language Models on Recommendation Task

Recently, the fast development of Large Language Models (LLMs) such as ChatGPT has significantly advanced NLP tasks by enhancing the capabilities of conversational models. However, the application of LLMs in the recommendation domain has not been thoroughly investigated. To bridge this gap, we propose LLMRec, a LLM-based recommender system designed for benchmarking LLMs on various recommendation tasks. Specifically, we benchmark several popular off-the-shelf LLMs, such as ChatGPT, LLaMA, ChatGLM, on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. Furthermore, we investigate the effectiveness of supervised finetuning to improve LLMs' instruction compliance ability. The benchmark results indicate that LLMs displayed only moderate proficiency in accuracy-based tasks such as sequential and direct recommendation. However, they demonstrated comparable performance to state-of-the-art methods in explainability-based tasks. We also conduct qualitative evaluations to further evaluate the quality of contents generated by different models, and the results show that LLMs can truly understand the provided information and generate clearer and more reasonable results. We aspire that this benchmark will serve as an inspiration for researchers to delve deeper into the potential of LLMs in enhancing recommendation performance. Our codes, processed data and benchmark results are available at https://github.com/williamliujl/LLMRec.

cs.IR↗
arXiv2026

Ontology Interoperability: A Comprehensive Framework for Industrial-Scale Applications

Different ontologies with conflicting and overlapping concepts cause havoc in the design, develop, and deploy of ontology-driven applications. In this work, we propose a comprehensive ontology interoperability framework for industrial-scale ontology-driven applications. The framework employs three state-of-the-art semantic techniques in different phases of the ontology engineering (OE) life cycle: ontology design patterns (ODPs) in the design phase, ontology matching and versioning (OM\&OV) in the develop phase, and ontology validation (OVA) in the deploy phase, to achieve better ontology interoperability.

cs.IR↗
arXiv2026

ScholarQuest: A Taxonomy-Guided Benchmark for Agentic Academic Paper Search in Open Literature Environments

Academic paper search is a core step in scientific research, and LLM-based search agents are emerging as a promising paradigm for iterative, intent-driven literature exploration. However, existing benchmarks offer limited coverage of research intents and insufficient support for scalable answer construction and reproducible multi-turn evaluation. We propose \textbf{ScholarQuest}, a taxonomy-guided benchmark constructed from over 1,000 computer science topics and organized around four research intents. To construct answer sets at scale, we combine multi-source retrieval, citation expansion, and multi-stage relevance adjudication. We further build \textit{\textbf{ScholarBase}}, a million-scale literature environment with unified retrieval and citation-expansion interfaces, together with a shared multi-turn search harness. We benchmark complete literature search systems and compare language models within this harness, evaluating retrieval coverage, search efficiency, and robustness across research intents. Results show that agentic methods outperform single-shot retrieval baselines, yet the best-performing search system achieves only 0.312 Recall@100 and 0.352 Recall@All. Comparisons under the shared harness reveal intent-dependent model strengths, with higher Planner token consumption not consistently yielding higher recall. We release the benchmark, retrieval backend, and evaluation harness for reproducible research

cs.IR↗