How NOT to Fool the Masses When Giving Performance Results for Quantum Computers

Catherine McGeoch

Home/Archive/arXiv

arXiv · 2411.08860

How NOT to Fool the Masses When Giving Performance Results for Quantum Computers

Catherine McGeoch

View original source ↗Read source document ↗Export metadata ↓

Abstract

In 1991, David Bailey wrote an article describing techniques for overstating the performance of massively parallel computers. Intended as a lighthearted protest against the practice of inflating benchmark results in order to ``fool the masses" and boost sales, the paper sparked development of procedural standards that help benchmarkers avoid methodological errors leading to unjustified and misleading conclusions. Now that quantum computers are starting to realize their potential as viable alternatives to classical computers, we can see the mistakes of three decades ago being repeated by a new batch of researchers who are unfamiliar with this history and these standards. Inspired by Bailey's model, this paper presents four suggestions for newcomers to quantum performance benchmarking, about how not to do it. They are: (1) Don't claim superior performance without mentioning runtimes; (2) Don't report optimized results without mentioning the tuning time needed to optimize those results; (3) Don't claim faster runtimes for (or in comparison to) solvers running on imaginary platforms; and (4) No cherry-picking (without justification and qualification). Suggestions for improving current practice appear in the last section.

Explore related subjects

quant-ph cs.ET

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Catherine McGeoch. 2024-11-13. How NOT to Fool the Masses When Giving Performance Results for Quantum Computers. https://arxiv.org/abs/2411.08860

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Quantum Authenticated Key Expansion with Key Recycling

Data privacy and authentication are two main security requirements for remote access and cloud services. While QKD has been explored to address data privacy concerns, oftentimes its use is separate from the client authentication protocol despite implicitly providing authentication. Here, we present a quantum authentication key expansion (QAKE) protocol that (1) integrates both authentication and key expansion within a single protocol, and (2) provides key recycling property - allowing all authentication keys to be reused. We analyse the security of the protocol in a QAKE framework adapted from a classical authentication key exchange (AKE) framework, providing separate security conditions for authentication and data privacy. We experimentally implemented the protocol with appropriate post-selection. Additional results on the security of pseudorandom basis generation in QAKE and decoy state BB84 are provided.

quant-ph↗

arXiv2026

Entanglement as Difference: Reduction-induced Minimal Partial Entropy Difference

Bipartite mixed-state quantum entanglement (QE) and its measures play a crucial role in both theoretical research and practical quantum applications. Its internal structure is far more complex and less well understood compared with bipartite pure-state QE. Some existing measures involve inherently intractable global optimizations, while others are only applicable to highly limited-dimensional quantum systems. Here based on the inherent feature that bipartite QE systems nonseparable necessarily implies that local reduced density matrix differs from its \textquotedblleft native\textquotedblright density matrix, we propose a more physical and intuitive measure termed Reduction-induced Minimal Partial Entropy Difference to quantify arbitrary bipartite mixed-state QE. Partial Von Neumann Entropy is only a pure-state special case of this method. This measure offers intrinsic structural %perspective insights into bipartite QE characterization, thereby establishing itself as a valuable complementary measure. Its intuitive and clear physical picture, combined with relatively low computational complexity and wide applicability, facilitates exploring its potential quantum information applications, hence its conceptual framework and line of thought deserve to be further developed to describe and quantify multipartite QE in the future.

quant-ph↗

arXiv2026

Non-local mass superpositions and optical clock interferometry in atomic ensemble quantum networks

Quantum networks are emerging as powerful platforms for sensing, communication, and fundamental tests of physics. We propose a programmable quantum sensing network based on entangled atomic ensembles, where optical clock qubits realize mass superpositions arising via mass-energy equivalence, as in atom and atom-clock interferometry. Our approach uniquely combines scalability to large atom numbers with minimal control requirements, relying only on collective addressing of internal atomic states. This enables the creation of both non-local and local superpositions with spatial separations beyond those achievable in conventional matter-wave interferometry with single atoms. Starting from Bell-type seed states distributed via photonic channels, collective operations within atomic ensembles coherently build many-body mass superpositions sensitive to gravitational redshift. The resulting architecture implements a non-local Ramsey interferometer, where gravitationally induced phase shifts are imprinted on non-local entangled states and are read out through local measurements at the network nodes. Beyond extending the spatial reach of mass superpositions, our scheme establishes a scalable, programmable platform to probe the interface of quantum mechanics and gravity, and offers a new experimental pathway to test atom and atom-clock interferometer proposals, e.g. for probing gravitational dephasing, in a network-based quantum laboratory.

quant-ph↗