Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2607.03105

ORBIT-Q: Dual-axis benchmarking of autonomous agents in scientific quantum programming

Shi-Xin Zhang·Yu-Qin Chen
View original source ↗Read source document ↗Export metadata ↓

Abstract

Autonomous coding agents perform well on many conventional programming tasks, but scientific computing demands a rigorous validation paradigm that extends beyond simple functional test completion: generated code must preserve physical fidelity, differentiable workflows, framework-native semantics, and scalable representations. We introduce Open Research Benchmark for Integrated Tasks in Quantum Computing (ORBIT-Q) to address this gap. At its core, ORBIT-Q contributes a carefully curated suite of complex, research-level quantum workflows that serves as a challenging testbed for modern scientific programming. ORBIT-Q combines a rigorous multi-tier verification pipeline to support two orthogonal comparisons: different agent harness and model configurations at a fixed quantum software framework, and different quantum software frameworks at a fixed agent. In our systematic evaluations, TensorCircuit-NG (TC) exhibits the highest capability and performance efficiency among the evaluated quantum software frameworks under agent-driven programming, and Codex with GPT-5.5 is the strongest tested agent configuration on TC. However, a significant performance and design gap remains between frontier autonomous agents and human expert reference implementations. We further evaluate two efficiency dimensions: agent-side resource use and artifact-side runtime. Together, these results establish ORBIT-Q as a rigorous benchmark for autonomous scientific programming, framework-agent synergy, and quantum software performance.

Explore related subjects

quant-ph

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Shi-Xin Zhang, Yu-Qin Chen. 2026-07-03. ORBIT-Q: Dual-axis benchmarking of autonomous agents in scientific quantum programming. https://arxiv.org/abs/2607.03105

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Iteratively decoded magic state distillation

We present numerical simulation results for the 7-to-1 and 15-to-1 state distillation circuits, constructed using transversal CNOTs acting on multiple surface code patches. The distillation circuits are decoded iteratively using the method outlined in [arXiv:2407.20976]. We show that, with a re-configurable qubit architecture, we can perform fast magic state distillation in $\sim\mathcal{O}(1)$ code cycles. We confirm that both circuits suppress an injected input logical error rate $p$ to $\mathcal{O}(p^3)$ in the presence of additional circuit-level noise. This is done with two types of stabiliser proxies, distilling logical $|-\rangle$ and $|Y\rangle$ states, the latter is the intended state of the 7-to-1 circuit while a stabiliser-proxy for the 15-to-1 circuit. We then also provide numerical evidences for actual $|T\rangle$ state distillation using the 15-to-1 circuit with a faulty-$T$ measurement, leveraging recent near-Clifford simulation tools. Finally, we outline how ZX-calculus and Pauli webs can be used to benchmark stabiliser proxies for these distillation circuits.

quant-ph↗
arXiv2026

Enhanced measurements on quantum computers via the simultaneous probing of non-commuting Pauli operators

Measuring the state of quantum computers is a highly non-trivial task, with implications for virtually all quantum algorithms. A promising avenue is multi-copy schemes, where identical copies of a quantum state are measured jointly so that all Pauli operators within the considered observable can be simultaneously assessed. Here, we present a first implementation of such a two-copy scheme in a measurement protocol. Based on Bayesian statistics, it accurately estimates not only the average of the desired observable but also the error en route. This enables an adaptive shot-allocation algorithm that preferentially samples the most uncertain Pauli terms. In regimes with many non-commuting Pauli operators, this ``double'' scheme can outperform the state-of-the-art measurement protocol in minimizing total shots for a given precision. We also numerically confirm the finding in previous theoretical works that the two-copy scheme incurs an overhead due to the square-root relationship between the variance of measured quantities and the number of measurement shots.

quant-ph↗
arXiv2026

Thermodynamics of a phaseonium-driven optomechanical Otto engine

We study an optomechanical Otto engine whose working medium is a single-mode cavity driven by beams of coherently prepared three-level phaseonium atoms. The atoms are not thermal reservoirs in the Gibbs sense; rather, their populations and ground-state coherence set the detailed-balance ratio of the cavity collision map, so that the field relaxes to a Gibbs state at an operational apparent temperature. We combine the finite-time collision-model dynamics with radiation-pressure work extraction and compare three reservoir preparations: a thermal reference at the same apparent temperatures, an incoherent atomic beam with the same populations, and the coherent phaseonium beam. We show that the phaseonium isochore charges the cavity passively: the cavity ergotropy and energy-basis coherence remain zero up to numerical precision, while the state converges to the Gibbs fixed point selected by the apparent detailed balance. We further estimate lower bounds on the cost of preparing the atomic populations and coherence, showing that the relevant advantage of phaseonium is a resource-preparation tradeoff rather than a cost-free enhancement over a thermal bath at the same temperature. Finally, we assess the finite-time performance of a two-cavity cascade with additive mechanical work accounting. Over the investigated coherence-phase range, the cascade produces approximately $47\%$--$52\%$ more power than the single-cavity engine while requiring only $65\%$--$68\%$ of the hot and cold phaseonium atoms needed by two independent engines, resulting in a $9\%$--$15\%$ enhancement of power per injected atom over a complete cycle.

quant-ph↗