Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.07035

Repository-Scale Performance Characterization of the IO500 Benchmark

Aasish Kumar Sharma·Anila Ghazanfar·Sepehr Mahmoodianhamedani·Sascha Safenreider·Julian Kunkel
View original source ↗Read source document ↗Export metadata ↓

Abstract

The IO500 benchmark provides a common basis for evaluating high-performance computing (HPC) storage systems, while its growing submission repository also offers an opportunity to study performance behavior across systems and time. This work presents a repository-scale characterization of 294 post-reset IO500 submissions from 116 sites spanning 2019--2025. We combine descriptive and temporal analysis with phase-level correlation, scale-sensitivity analysis, and benchmark-log examination to investigate how performance characteristics represented by individual benchmark phases relate to composite rankings. The results show strong relationships among phases measuring similar I/O behavior, but weaker and scale-sensitive relationships between bandwidth and metadata performance. Temporal and log-level analyses further reveal information that is not apparent from aggregate scores alone. These findings demonstrate the value of analyzing IO500 results beyond leader-board rankings and identify important considerations concerning system scale, benchmark execution, and repository provenance when using community benchmark data for comparative and longitudinal HPC performance analysis.

Explore related subjects

cs.DC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Aasish Kumar Sharma, Anila Ghazanfar, Sepehr Mahmoodianhamedani, Sascha Safenreider, Julian Kunkel. 2026-10-04. Repository-Scale Performance Characterization of the IO500 Benchmark. https://arxiv.org/abs/2610.07035

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Fast Tendermint: Speeding Up a Foundational Consensus Protocol

Tendermint is among the most widely studied and deployed Byzantine fault-tolerant (BFT) consensus protocols, owing in part to its native leader-rotation mechanism that subsumes complex view changes. Like most partially-synchronous BFT protocols, Tendermint tolerates $f < n/3$ Byzantine processes and decides in three communication steps. Motivated by the push for lower-latency blockchains, a recent line of work shows that consensus can be solved in two communication steps when $f < n/5$. We present Fast Tendermint, an adaptation of Tendermint to the $n > 5f$ setting that decides in two communication steps in the good case, while preserving Tendermint's leader-rotation structure. Fast Tendermint collapses Tendermint's prevote and precommit steps into a single voting step and merges the $locked$ and $valid$ state. We give proofs of agreement, validity, and termination, and a formal specification in Quint, a modern surface syntax for TLA+, used to model-check the protocol.

cs.DC↗
arXiv2026

ATTUNE: Agentic Throughput Tuning for LLM Training

The throughput of an LLM training job depends on its recipe: how the model is split across GPUs, how memory is traded for compute, and which kernels and communication overlaps it uses. The best recipe changes with every model, accelerator and framework version. Automated tuners search a fixed space of knobs and check only that a job runs, so they miss levers outside that space and can ship a configuration that silently changes training. ATTUNE is a coding agent that tunes any working recipe unattended within a GPU-hour budget. Guided by a skill of search rules, scripts and lessons from past tuning campaigns, it diagnoses each run and changes one lever at a time. A correctness gate outside the agent refuses any argument known to change the training objective. ATTUNE tunes Amazon's production post-training recipes on H100, H200 and B200 clusters, raising throughput by 1.7x on average. It beats the published throughput of four of NVIDIA's own benchmark recipes on NVIDIA's AutoModel framework by 6-17%. On models held out from its development, it improves the strongest public recipes most where they were written for another setting, by up to 2.6x on MiniMax-M2, and least where the vendor tuned them for the same hardware. Given the same GPU-hour budget, it outperforms existing automated tuners, including Bayesian optimization, random search and OptiCo, a recent LLM-agent tuner. Its gain on GLM-4.5 is more than four times the best alternative's. The same coding agent without ATTUNE's skill gains less and fails half its launches. We release ATTUNE at https://github.com/amazon-science/attune and OpenRecipes, a leaderboard of tuned and verified training recipes, at https://openrecipes.ai.

cs.DC↗
arXiv2026

Parallel Sparse Outputs for Tensor Compilers

Sparse tensor compilers are commonly used high-performance tools that optimize tensor algebra programs over notoriously complex sparse data structures. Despite the ever-increasing size and scale of sparse datasets, there exists astonishingly little research on developing a parallel sparse tensor compiler. Existing approaches fail to parallelize or make assumptions about access patterns or output formats that limit generality. This paper presents Birdseed, the first tensor compiler to support all loop orderings over any standard sparse format in parallel. More specifically, Birdseed supports parallel sparse scatters and directly writes into list, bytemap, hash, and dense levels of fibertree sparse formats. Our approach exhibits asymptotic memory and time improvements over all previous work. We additionally demonstrate empirical speedups on challenging kernels to parallelize, such as histogram construction and SpMSpV.

cs.DC↗