Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.17970

AlignPred: Error-Controlled KV Cache Compression via Alignment and Predictive Coding

Xu Yang·Jiapeng Zhang·Zhiwei Zuo·Yuxin Chen·Feiqiang Sun·Chengguang Xu·Feng Jin·Zhuo Tang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Long-running agents retain KV caches across turns and sessions for hours or days. Compressing these persistent caches requires reducing storage while controlling reconstruction error. We present AlignPred, a codec that combines local alignment with predictive coding under a configurable KV reconstruction-error budget. Local orthogonal transforms align related heads or layers into a common coordinate frame, allowing AlignPred to combine token-wise prediction with same-token prediction across heads or layers. Residuals are quantized on an error-bounded lattice and entropy-coded, without learned predictors, encoder-side mode search, or large joint transforms. Across three attention mechanisms, AlignPred requires less storage than the evaluated baselines at comparable measured reconstruction error. Against the evaluated CPU codec implementations, it decodes at least 1.6 times faster while matching or improving encode time. AlignPred achieves near-native task scores at substantially compressed operating points on LongBench-Pro at 128K context. AlignPred's SGLang HiCache integration reduces persistent target-KV storage by 42-72% relative to native HiCache, with comparable latency.

Explore related subjects

cs.PFcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Xu Yang, Jiapeng Zhang, Zhiwei Zuo, Yuxin Chen, Feiqiang Sun, Chengguang Xu, Feng Jin, Zhuo Tang. 2026-10-13. AlignPred: Error-Controlled KV Cache Compression via Alignment and Predictive Coding. https://arxiv.org/abs/2610.17970

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

On the Robustness of Sparse Randomized Sketching to FP16 Rounding on GPUs

Randomized sketching replaces large matrices with smaller surrogates that preserve geometry for downstream numerical linear algebra. Sparse sketches promise faster formation than dense projections, but irregular GPU atomic updates become costly when each input row contributes to several output rows, as in SparseStack. FP16 accumulation halves sketch storage and update traffic, relieving cache pressure. How much does this rounding affect sketch geometry and downstream accuracy? For SparseStack with four nonzeros per column, cuSPARSE's FP16 path is slower than its FP32 baseline. Our custom kernel instead pairs FP32 inputs and signed products with packed FP16 vector atomic accumulation, turning lower precision into substantial gains. On NVIDIA H200, it forms complete sketches up to $1.91\times$ faster than our FP32 baseline and up to $1.58\times$ faster than cuSPARSE FP32. Under stated independence and range assumptions, random sketch signs cancel round-to-nearest conversion-error cross terms, giving expected squared Frobenius error no greater than ideal stochastic rounding. We assess accumulation empirically. At a sketch dimension of twice the number of input columns, the least-squares pipeline is $1.44$-$1.47\times$ faster than FP32, with median paired residual increases below $0.4\%$. Rescaling exposes range limitations and cases where stochastic conversion degrades accuracy.

cs.PF↗
arXiv2026

How Reliable Are Large Language Models for Soil Organic Carbon Assessment? An Australian Benchmark

Soil organic carbon (SOC) is an indicator of agricultural soil condition and carbon storage, but numerical outputs from large language models (LLMs) require validation before use in monitoring or management. We evaluated seven general-purpose LLM configurations on three direct-response tasks: estimating 0--10 cm SOC concentrations for 308 Australian paddocks under two information conditions, calculating SOC stock in 120 matched scenarios, and comparing 300 paddock pairs. References were measured SOC records or deterministic calculations; response validity, repeated-call dispersion and estimated API cost were assessed separately. Additional measured soil attributes reduced concentration mean absolute error by 13--62\% across configurations, although high-SOC underprediction and paddock-specific errors persisted. Six configurations were highly accurate in stock calculations using equation-native units without coarse fragments, but accuracy declined with physically equivalent alternative units. Pairwise accuracy was 90.5--94.6\% overall and 59.1--70.9\% for 52 closely separated pairs; some configurations were sensitive to presentation order. Nearly all outputs were parseable, without necessarily being correct. The benchmark identifies three distinct requirements for AI-mediated SOC assessment: agreement with measured concentrations, unit-consistent stock calculations and reliable discrimination between sites. These findings support task-specific validation and auditable calculations, rather than unverified use of LLM outputs as ecological evidence.

cs.PF↗
arXiv2026

Fast and Flexible Spiking Network Simulations on Massively Distributed On-Chip Memory Accelerators

Studying brain function and dysfunction in spiking networks with realistic synaptic densities pushes simulation technology beyond its current limits. It requires simulations substantially faster than biological real-time to cover long model times and enable efficient iterative exploration, particularly when connectivity is plastic. The modelling community aspires to a $100\times$ speedup. Yet on conventional many-core systems this remains elusive, bounded by inter-node communication and memory access. So far, significant speedup has only been shown on a custom-built FPGA system. We demonstrate the potential of commercial systems with large distributed on-chip memory to accelerate brain research. To exploit the local SRAM, we introduce a group-based mapping and communication strategy tackling spike-routing challenges on massively distributed systems. On Graphcore IPUs, the cortical microcircuit runs at a record $22.7\times$ faster than biological real-time with negligible build times and a reusable compiled graph. By design, the algorithm makes no model- or hardware-specific assumptions, providing a general strategy for advancing neuronal simulation technology.

cs.PF↗