Ascend-RaBitQ: Heterogeneous NPU-CPU Acceleration of Billion-Scale Similarity Search with 1-bit Quantization

Fujun He; Chuyue Ye; Huaxiang Cai; Zetao Lv; Baolong Cui; Wenru Yan; Chao Zhan; Zigang Zhang; Hao Yi; Jie Xiang; Xiabing Li; Yuhang Gai; Ziyang Zhang; Pengfei Zheng; Yunfei Du

arXiv · 2605.16007

Ascend-RaBitQ: Heterogeneous NPU-CPU Acceleration of Billion-Scale Similarity Search with 1-bit Quantization

Fujun He·Chuyue Ye·Huaxiang Cai·Zetao Lv·Baolong Cui·Wenru Yan·Chao Zhan·Zigang Zhang·Hao Yi·Jie Xiang·Xiabing Li·Yuhang Gai·Ziyang Zhang·Pengfei Zheng·Yunfei Du

View original source ↗Read source document ↗Export metadata ↓

Abstract

Vector similarity search is a critical component of modern AI systems, but traditional CPU-based implementations face fundamental scalability bottlenecks for billion-scale corpora due to prohibitive computational overhead and memory bandwidth limitations. While Neural Processing Units (NPUs) offer orders-of-magnitude higher compute density, existing CPU/GPU-optimized 1-bit RaBitQ quantization implementations cannot be directly ported to NPU architectures due to fundamental hardware mismatches, and homogeneous design paradigms struggle to simultaneously balance accuracy, memory footprint, and performance. This paper presents Ascend-RaBitQ, the first heterogeneous NPU-CPU optimized IVF-RaBitQ system for billion-scale vector search, built on the core insight that decoupling coarse ranking (NPU) from fine ranking (CPU) allows each stage to leverage its optimal hardware, breaking the long-standing accuracy-memory-performance trade-off. We propose a three-stage heterogeneous execution path comprising AI Core-accelerated coarse ranking on 1-bit quantized vectors, on-device AI CPU Top-k processing, and host CPU fine re-ranking on full-precision vectors. We introduce four NPU architecture-native optimizations: fused AIC-AIV operators for parallel distance computation, computation flow restructuring to exploit rotation orthogonality, fine-grained index block-level load balancing that breaks query boundaries, and intra-NPU pipeline parallelism between AI Core and AI CPU to mask Top-k latency. Evaluation on standard datasets shows that Ascend-RaBitQ achieves 3.0X to 62.8X faster index construction than the CPU baseline, up to 11.7X throughput improvement over the fastest CPU IVF-RaBitQ implementation, and over two orders of magnitude over the mathematically equivalent CPU baseline, while demonstrating encouraging scalability on distributed multi-NPU systems.

Explore related subjects

cs.IR

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Fujun He, Chuyue Ye, Huaxiang Cai, Zetao Lv, Baolong Cui, Wenru Yan, Chao Zhan, Zigang Zhang, Hao Yi, Jie Xiang, Xiabing Li, Yuhang Gai, Ziyang Zhang, Pengfei Zheng, Yunfei Du. 2026-06-15. Ascend-RaBitQ: Heterogeneous NPU-CPU Acceleration of Billion-Scale Similarity Search with 1-bit Quantization. https://arxiv.org/abs/2605.16007

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Do Large Language Models Favor Recent Content? A Study on Recency Bias in LLM-Based Reranking

Large language models (LLMs) are increasingly deployed in information systems, including being used as second-stage rerankers in information retrieval pipelines, yet their susceptibility to recency bias has received little attention. We investigate whether LLMs implicitly favour newer documents by prepending artificial publication dates to passages in the TREC Deep Learning passage retrieval collections in 2021 (DL21) and 2022 (DL22). Across seven models, GPT-3.5-turbo, GPT-4o, GPT-4, LLaMA-3 8B/70B, and Qwen-2.5 7B/72B, "fresh" passages are consistently promoted, shifting the Top-10's mean publication year forward by up to 4.78 years and moving individual items by as many as 95 ranks in our listwise reranking experiments. Although larger models attenuate the effect, none eliminate it. We also observe that the preference of LLMs between two passages with an identical relevance level can be reversed by up to 25% on average after date injection in our pairwise preference experiments. These findings provide quantitative evidence of a pervasive recency bias in LLMs and highlight the importance of effective bias-mitigation strategies.

cs.IR↗

arXiv2026

UniRec: Cross-stage Multi-Task Fusion with Preference Alignment for Cascaded Recommender Systems

Industrial recommender systems cascade stages with different objectives, feature spaces, and latency constraints. Optimizing pre-ranking and ranking separately induces cross-stage inconsistency: upstream models may filter out items preferred by downstream rankers, while independently tuned downstream fusion can offset upstream improvements. Most existing multi-task fusion methods target the ranking stage alone, and cross-stage methods often align with a downstream-derived score, leaving joint optimization of fusion modules across cascaded stages largely unexplored. We propose UniRec, a Unified Cross-stage Recommendation Fusion model. First, the two fusion agents partially share input embeddings in a single computation graph, allowing gradients from either stage to propagate through the shared representations. Second, a dual-axis preference alignment objective coordinates the two stages: horizontally, a compact aggregation term reorganizes dozens of pairwise objectives over heterogeneous prior signals into bidirectional preference evidence; vertically, a cross-stage consistency term transfers downstream pairwise preferences to the upstream fusion score. Third, we introduce attribute group-relative regularization, which computes relative advantages and normalizes policy updates within each attribute group, ensuring that uniformly promoting all items in a high-reward group provides no additional optimization gain. Offline experiments demonstrate UniRec consistently outperforms single-stage fusion and cross-stage coordination baselines; online A/B experiments show a 0.616% gain in app usage duration. UniRec has been fully deployed on the Kuaishou platform.

cs.IR↗

arXiv2026

Scaling Articulated Rationales for MLLM-based Recommendation

We presented SARA, an industrial framework that transforms sparse articulated user rationales into scalable recommendation signals. Its data engine curates questionnaire responses into SARA-HQ, providing explicit preference supervision for aligning SARA-7B through SFT and Quality-Refining DPO. This alignment extends rationale generation from $86{,}564$ questionnaire-covered authors to the full $10$M-author space. SARA-Ranker translates the generated positive and negative rationales into features for user--author interaction modeling and negative-feedback history modeling, connecting articulated reasons to production ranking. Evaluation on unseen authors demonstrates that SARA-7B generates more specific, relevant, and grounded rationales than the evaluated general-purpose MLLMs. On top of a strong industrial ranking baseline with multimodal features, separate online A/B tests show that positive-rationale integration increases watch time by $0.99\%$, while negative-rationale integration reduces Hate feedback by $8.16\%$. Daily refresh and more than $30$ days of production deployment further demonstrate the operational feasibility of the approach. These findings establish articulated rationales as a useful complement to behavioral and content signals, and demonstrate a practical role for MLLMs in scaling sparse human explanations into preference information that improves industrial recommendation.

cs.IR↗