Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.28867

Engineering Low-Latency, High-Throughput Quantum Decoders on NVIDIA GPUs

Stefano Markidis·Luca Pennati·Ivy Peng
View original source ↗Read source document ↗Export metadata ↓

Abstract

We design and optimize a quantum error correction (QEC) decoder for NVIDIA GPUs, combining Relay belief propagation, a state-of-the-art message-passing algorithm, with order-zero ordered statistics post-processing. Our implementation uses CUDA Graphs to control GPU execution, stops updates for converged samples, and collects unresolved samples in queues for post-processing. We optimize sparse traversal through coalesced message accesses, prefetching, and register reuse, and accelerate binary post-processing with warp-level collectives. In addition, mixed FP16-FP32 arithmetic and INT8 quantization reduce message storage and traffic while retaining accuracy using accumulators at higher precision. We evaluate surface, hypergraph product, and bivariate bicycle code workloads on NVIDIA A100 and GH200 data-center GPUs and a commodity RTX 5080. On the bivariate bicycle code at error probability 0.003, INT8 halves amortized decoding time relative to independently tuned FP32 with a negligible logical error rate increase on RTX 5080. With independent syndrome batches submitted back-to-back to concurrent decoder instances, RTX 5080 delivers 91% of GH200's highest measured throughput with an 8.9x shorter median submission-to-correction latency. The combination of performance optimization, mixed precision, INT8 quantization, and concurrent decoder instances opens a path toward QEC decoder clusters built from commodity GPUs.

Explore related subjects

cs.DCquant-ph

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Stefano Markidis, Luca Pennati, Ivy Peng. 2026-10-20. Engineering Low-Latency, High-Throughput Quantum Decoders on NVIDIA GPUs. https://arxiv.org/abs/2610.28867

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

NanoCP: Taming Load Imbalance in Data-Expert Parallel Decoding under Variable-Length Workloads

Modern serving systems for Mixture-of-Experts (MoE) models adopt hybrid data-expert parallelism: expert parallelism (EP) shards experts across GPUs to scale capacity, while data parallelism (DP) replicates attention layers across instances to process independent requests. Existing systems bind each request's attention, MoE communication, and KV cache to a single instance. Because attention latency scales with KV cache size while MoE communication latency scales with batch size, this binding cannot balance both simultaneously, producing EP stragglers; it also fragments KV memory across instances, inflating tail latency under long contexts. While existing context parallelism (CP) mitigates these constraints, existing group-level and uniform CP designs incur prohibitive communication and attention-side overheads. We present NanoCP, which decouples MoE communication from KV cache placement and achieves dual balance through dynamic context parallelism (DCP). DCP assigns each request a context-parallel degree sized to its KV footprint: long requests distribute attention across multiple instances; short requests remain local. This dynamic parallelism effectively liquefies the KV cache across DP instances within each node, balancing both the per-instance KV cache occupancy and batch sizes without unnecessary load-balancing costs. To bridge DCP with static execution, NanoCP introduces an ahead-of-time (AOT) graph engine paired with a custom routing-based communication backend. Experimental results show that, under 99% TPOT-SLO attainment, NanoCP achieves $3.27\times$ and $2.11\times$ the maximum sustainable request rate of the best-performing vLLM baseline on the DeepSeek-V3 Issue 1% and Issue 5% workloads, respectively, while reducing P99 TPOT by $1.94\times$ and $2.01\times$ on the same two workloads, respectively.

cs.DC↗
arXiv2026

EP-NCO: Latency-Aware Service Placement using Neural Combinatorial Optimisers for Edge-Cloud Systems

Latency-sensitive Internet of Things (IoT) applications are increasingly deployed as interdependent microservices across heterogeneous edge-cloud infrastructures. Placing them under resource and network constraints is an NP-hard combinatorial optimisation problem. This study proposes EP-NCO, a graph-based neural combinatorial optimisation framework for service placement in compute continuum platforms. The objective accounts for execution latency, direct-link communication delay, and an approximate sharing of bandwidth among dependency flows that coexist in the resulting placement. Across held-out graph sizes generated by the same synthetic framework, the EP-NCO variant trained on Large instances with hard decoding achieved the lowest mean aggregate latency cost at every scale. It was 6.0%-12.5% lower than a genetic algorithm (GA), 14.4%-20.8% lower than particle swarm optimisation (PSO), and 1.2%-9.4% lower than the strongest reinforcement learning ablation. On Small instances, its mean cost was 10.5% below the best Gurobi solutions found and close to Gurobi's mean lower bound. The observed inference time of the EP-NCO variants was 0.88-1.03 s per instance, compared with 85-97 s for GA and PSO. As a result, the one-off training cost of the EP-NCO variant trained on Large instances with hard decoding is recovered after roughly 1,000 placements.

cs.DC↗
arXiv2026

Weight Placement Is Not Enough: Synchronization in Multi-Node CPU Decoding of Ternary LLMs

Ternary LLMs read only a few gigabytes per decoded token, yet on dual-socket servers with four NUMA nodes the fastest ternary runtime we found uses less than half of the memory bandwidth. Placing each weight on the node of the thread that reads it is necessary but not enough: in a decode engine with node-local weights that reuses ik_llama's kernels, threads on two generations of Xeon servers still spend 20-48% of each token waiting at synchronization points. Per-thread barrier traces show that most of the wait is for the slowest threads of each node; the barriers' own latency (microseconds each, but up to 12% of the token for the smallest model) and serial per-node work are smaller parts. On identical kernels, a Megatron-style column split beats a row-owned split without cross-node reductions in every configuration we measured, and isolating the number of barriers shows that most of its gain comes from its threads waiting less for each other and for their node leaders, and only a minority from having fewer barriers. Where decoding is DRAM-bound, the resulting engine is 1.5-2.1x faster than ik_llama with all cores and 1.5-1.9x faster than ik_llama at its fastest thread count (3.2x and 2.8x for a small model that runs from the L3 cache), at the same accuracy and with less energy per token.

cs.DC↗