QREF: Improving DeepSearch Agents by Aligning Search with Reasoning via Query Refinement and Evidence Filtering

Yujia Chen; Yuhui Wang; Junzhe Wang; Shihan Dou; Di Liang; Yujiong Shen; Yuhao Zhou; Zhiheng Xi; Ming Zhang; Senjie Jin; Jiayi Chen; Dingwei Zhu; Yajie Yang; Yuan Hua; Chenyao Lu; Tao Gui; Qi Zhang; Xuanjing Huang

arXiv · 2610.21512

QREF: Improving DeepSearch Agents by Aligning Search with Reasoning via Query Refinement and Evidence Filtering

Yujia Chen·Yuhui Wang·Junzhe Wang·Shihan Dou·Di Liang·Yujiong Shen·Yuhao Zhou·Zhiheng Xi·Ming Zhang·Senjie Jin·Jiayi Chen·Dingwei Zhu·Yajie Yang·Yuan Hua·Chenyao Lu·Tao Gui·Qi Zhang·Xuanjing Huang

View original source ↗Read source document ↗Export metadata ↓

Abstract

DeepSearch agents achieve strong performance on complex information-seeking tasks through iterative search and reasoning. However, they often issue intent-misaligned, under-constrained, or redundant search queries, resulting in poor evidence coverage and accumulating retrieval noise. To mitigate these limitations, we propose QREF, a plug-and-play Query Refiner and Evidence Filter. Specifically, the Query Refiner rewrites search queries according to the current reasoning intent while ensuring complementary coverage across each search batch. The Evidence Filter retains evidence relevant to both the current search batch and the reasoning objective, reducing distracting retrieval results. We initialize the Query Refiner with supervised fine-tuning and optimize it through reinforcement learning based on interactions with a real-world web search environment, reducing ineffective queries. Comprehensive experiments demonstrate that our approach improves the performance of DeepSearch agents across four challenging benchmarks, achieving average relative Pass@1 improvements of 33.55% on WebSailor and 11.04% on TongyiDeepResearch without modifying architectures or retraining their reasoning policies. Compared with the original DeepSearch backbone without QREF, the RL-trained refiner reduces the fraction of query batches that lack discriminative retrieval anchors by 95%. Compared with the SFT-trained refiner, it also lowers the filter-based ineffective query ratio.

Explore related subjects

cs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Yujia Chen, Yuhui Wang, Junzhe Wang, Shihan Dou, Di Liang, Yujiong Shen, Yuhao Zhou, Zhiheng Xi, Ming Zhang, Senjie Jin, Jiayi Chen, Dingwei Zhu, Yajie Yang, Yuan Hua, Chenyao Lu, Tao Gui, Qi Zhang, Xuanjing Huang. 2026-10-15. QREF: Improving DeepSearch Agents by Aligning Search with Reasoning via Query Refinement and Evidence Filtering. https://arxiv.org/abs/2610.21512

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Atomic Proximal Policy Optimization for Electric Robo-Taxi Dispatch and Charger Allocation

Pioneering companies such as Waymo have deployed electric robo-taxi services in several U.S. cities. Operating such fleets requires coordinating ride matching, vehicle repositioning, and battery charging in a stochastic environment while accounting for limited charging infrastructure. Understanding both how to design efficient, scalable control algorithms for large fleets and how charger deployment shapes system performance is therefore critical for robo-taxi operators. We model the operations of an electric robo-taxi system as a discrete-time Markov decision process with a long-run average reward objective. To efficiently evaluate system performance under different configurations, we develop a scalable deep reinforcement learning algorithm, Atomic Proximal Policy Optimization (Atomic-PPO), which reduces the action space using atomic action decomposition. The resulting training procedure requires only modest computational resources, making it practical to evaluate a large number of system configurations. We calibrate a simulation environment using real-world NYC for-hire vehicle trip records and benchmark Atomic-PPO against heuristic policies and a fluid-based reward upper bound. The results show that Atomic-PPO produces near-optimal fleet dispatching policies and serves as a reliable tool for system evaluation. We conduct extensive numerical experiments to evaluate how charging infrastructure and fleet size affect system performance. Our findings highlight the importance of ridership-based charger placement, balanced investments in charger quantity and charging speed, and joint planning of charging infrastructure and fleet size.

cs.AI↗

arXiv2026

Generalising from Self-Produced Data: Model Training Beyond Human Constraints

Current large language models (LLMs) are constrained by human-derived training data and limited by a single level of abstraction that impedes definitive truth judgments. This paper introduces a novel framework in which AI models autonomously generate and validate new knowledge through direct interaction with their environment. Central to this approach is an unbounded, ungamable numeric reward - such as annexed disk space or follower count - that guides learning without requiring human benchmarks. AI agents iteratively generate strategies and executable code to maximize this metric, with successful outcomes forming the basis for self-retraining and incremental generalisation. To mitigate model collapse and the warm start problem, the framework emphasizes empirical validation over textual similarity and supports fine-tuning via GRPO. The system architecture employs modular agents for environment analysis, strategy generation, and code synthesis, enabling scalable experimentation. This work outlines a pathway toward self-improving AI systems capable of advancing beyond human-imposed constraints toward autonomous general intelligence.

cs.AI↗

arXiv2026

Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis

Data synthesis for training large reasoning models offers a scalable alternative to limited, human-curated datasets, enabling the creation of high-quality data. However, existing approaches face several challenges: (i) indiscriminate generation that ignores the solver's ability and yields low-value problems, or reliance on complex data pipelines to balance problem difficulty; and (ii) a lack of reasoning in problem generation, leading to shallow problem variants. In this paper, we develop a problem generator that reasons explicitly to plan problem directions before synthesis and adapts difficulty to the solver's ability. We recover problem-design CoT from human-authored multi-part questions, using later subquestions as targets to infer the design rationale. These traces initialize the generator, which is then optimized with a seed-conditioned reward based on solver answer consistency. The reward guides difficulty adjustment relative to each seed while favoring problems near the solver's competence boundary. Across 10 mathematical and general reasoning benchmarks, our method outperforms seed-only training by 2.69 and 3.39 percentage points after one and three rounds, respectively. The framework also extends to vision-language models. Our code is publicly available at https://github.com/WalkerWorldPeace/Reasoning-Synthesis.

cs.AI↗