Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.35412

Read the Route: Attention-Steered Hypergraph Retrieval for Multi-hop RAG

Jiate Liu·Liuyi Chen·Xiaofei Yuan·Yifu Tang·Zihan Yang·Xu Zhou·Zhengyi Yang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Graph-based retrieval-augmented generation (GraphRAG) answers multi-hop questions by organizing a corpus into a graph and retrieving along its structure. Existing retrieval methods face two limitations. Diffusion- and similarity-based methods expand retrieval along fixed edge weights or by similarity to the question, without resolving its implicit entity references, allowing relevance errors to accumulate across hops. In contrast, exploration-based methods let an LLM reason about the next hop, but require a separate generation at each hop, leading to high inference costs. We observe that an LLM can resolve implicit entity references from retrieved passages and a multi-hop question before generating any tokens. Further analysis identifies a small set of attention heads, termed addressing heads, that capture this capability across multiple hops in a single forward pass. Building on this finding, we propose HyperAddress, which runs Personalized PageRank on a description-labelled hypergraph, using addressing heads to determine the restart distribution and transition weights, thereby guiding multi-hop retrieval without any generation at query time. On MuSiQue, 2WikiMultiHopQA, and HotpotQA, HyperAddress outperforms the strongest baseline by 3.5 points of Recall@5 and 7.4 points of All@5 on average, with gains that grow with the number of hops. It answers more than twice as fast and builds its index at half the cost.

Explore related subjects

cs.IR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jiate Liu, Liuyi Chen, Xiaofei Yuan, Yifu Tang, Zihan Yang, Xu Zhou, Zhengyi Yang. 2026-10-26. Read the Route: Attention-Steered Hypergraph Retrieval for Multi-hop RAG. https://arxiv.org/abs/2610.35412

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

An Interpretable CF-RL-TOPSIS Fusion Model for Skills-Aware Talent Recommendation

Effective skills-aware talent recommendation must balance behavioral transition patterns, trajectory-sensitive adaptation, and inspectable occupation-level criteria. Evidence from public benchmarks on how these signals interact, however, remains limited. This study proposes CF-RL-TOPSIS, an interpretable late-fusion model that integrates a transition-aware collaborative branch, a compact reinforcement-style occupation-family bandit, and an entropy-weighted TOPSIS branch constructed from six semantic proxies; the validation-selected fusion coefficients remain auditable. The model is evaluated on two frozen public ICT talent-history benchmarks, JobHop and Karrierewege, using repeated chronological top-5 ranking and paired Wilcoxon tests. On JobHop the full hybrid attains NDCG@5 = 0.3040 +/- 0.0073 and significantly surpasses repeat-last, item Markov, transition-aware collaborative filtering, the CF+TOPSIS hybrid, GRU4Rec, and SASRec (p <= 0.0039 across planned comparisons). On Karrierewege the hybrid remains competitive but does not significantly exceed the strongest Markov baseline, revealing a persistence-dominated setting in which the bandit branch appropriately shrinks to near-zero weight. Proxy-sensitivity, family-level deep Q-network, and runtime checks support this interpretation, and a worked user-level case shows how branch scores, criterion weights, and rank shifts can be inspected for an individual recommendation. The contribution is not a benchmark-agnostic superiority claim, but a reproducible account of the conditions under which transparent late fusion adds value beyond simple continuation heuristics. In semantically rich, non-saturating talent-history regimes the three branches reinforce one another; in persistence-dominated regimes the same architecture remains competitive through its collaborative backbone, with the adaptive branch correctly inactive.

cs.IR↗
arXiv2026

UniRec: Cross-stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems

Industrial recommender systems use cascaded stages with different candidate spaces, data distributions, and latency constraints. Each stage include multi-objective prediction and multi-task fusion (MTF) modules. Existing MTF methods typically design and optimize the fusion module independently for each stage, which can lead to inconsistent preferences across stages. Although cross-stage coordination methods have been proposed, they mainly transfer supervision to upstream scoring models rather than jointly optimizing the fusion modules. We propose UniRec, a Unified Cross-stage Recommendation Fusion framework that jointly optimizes pre-ranking and ranking fusion agents. UniRec employs a coupled dual-agent architecture with partially shared embeddings and stage-specific fusion functions. It further introduces Dual-Axis Preference Optimization to combine stage-local preferences with downstream-to-upstream preference transfer. To enable stable and efficient cross-stage training, we address two practical challenges: reward-induced distribution drift and the computational cost of pairwise preference learning. Extensive offline and online experiments on a production short-video system demonstrate consistent gains over single-stage fusion and cross-stage coordination baselines. In a nine-week online A/B test, UniRec improves total watch time by $0.675\%$ and app usage time by $0.616\%$, and active users by $0.189\%$, and has been fully deployed in production.

cs.IR↗
arXiv2026

Geometric Gain Graph: Zero-Token Graph Construction for Multi-Hop RAG

Existing Retrieval-Augmented Generation (RAG) systems struggle with multi-hop reasoning due to a fundamental inability to balance relevance and novelty. Traditional dense retrievers are constrained by surface-level matching, falling into a homogenized "Similarity Trap". Conversely, emerging Graph RAGs attempt to explore novel documents via heuristic entity extraction, but they introduce prohibitive Large Language Model (LLM) overhead and easily drift into off-topic "Novelty Traps". To fundamentally resolve this contradiction, we propose Geometric Gain Graph (G$^3$RAG), the novel zero-token graph construction paradigm for RAG. G$^3$RAG completely discards the expensive heuristic extraction by LLMs, aiming instead to directly quantify the information gain between nodes through the native document feature space. To this end, we designed a concise edge weight criterion, $\cosθ\cdot \sinθ$, which naturally maps the trade-off between relevance and novelty into a geometric measurement of directional consistency and orthogonality in the vector space, thereby driving the token-free topological construction of the graph. Furthermore, we introduce a topological penalty to suppress the excessive connectivity of high-frequency hubs, forcing the transient diffusion process toward long-tail peripheral nodes that carry critical indirect evidence. Extensive experiments demonstrate that, while entirely eliminating LLM token costs during graph construction, G$^3$RAG significantly outperforms state-of-the-art Graph RAG baselines on complex multi-hop QA benchmarks. The code is available at: https://github.com/deeplearning-LI/G3RAG

cs.IR↗