Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.27755

Evaluating Runtime Prediction and Conservative Walltime Policies in HPC Systems

Hector Romero Ugalde·Youssef Faqir-Rhazoui·Marlon Funk·Jesus Gorroñogoitia Cruz
View original source ↗Read source document ↗Export metadata ↓

Abstract

Efficient use of large-scale supercomputing systems depends on accurate runtime estimation and walltime selection. To avoid premature termination, users often overestimate walltime, leading to poor resource utilization and reduced scheduling efficiency. This paper presents a predictor-agnostic framework for safe walltime recommendation in production HPC environments. Rather than proposing a new runtime prediction model, the primary contribution is the SAFE decision layer, which converts predicted runtimes into conservative bounded walltime recommendations suitable for deployment. Runtime prediction is therefore treated as a modular component within the proposed framework. We evaluate the approach using large-scale production job histories from the Fugaku and NREL Eagle supercomputers, comprising approximately 24M and 11M jobs, respectively. Runtime prediction performance is first evaluated using representative predictors, including linear regression, neural networks, and a neural ensemble. The proposed SAFE framework is then evaluated using the NN ensemble predictor. Experimental results indicate that similarity-aware prediction substantially improves runtime estimation for workloads similar to previous executions. When combined with the proposed conservative walltime recommendation policies, these improvements enable walltime gap reductions of approximately 20 - 26% while maintaining job termination rates below 1% across both systems.

Explore related subjects

cs.DC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Hector Romero Ugalde, Youssef Faqir-Rhazoui, Marlon Funk, Jesus Gorroñogoitia Cruz. 2026-10-20. Evaluating Runtime Prediction and Conservative Walltime Policies in HPC Systems. https://arxiv.org/abs/2610.27755

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

NanoCP: Taming Load Imbalance in Data-Expert Parallel Decoding under Variable-Length Workloads

Modern serving systems for Mixture-of-Experts (MoE) models adopt hybrid data-expert parallelism: expert parallelism (EP) shards experts across GPUs to scale capacity, while data parallelism (DP) replicates attention layers across instances to process independent requests. Existing systems bind each request's attention, MoE communication, and KV cache to a single instance. Because attention latency scales with KV cache size while MoE communication latency scales with batch size, this binding cannot balance both simultaneously, producing EP stragglers; it also fragments KV memory across instances, inflating tail latency under long contexts. While existing context parallelism (CP) mitigates these constraints, existing group-level and uniform CP designs incur prohibitive communication and attention-side overheads. We present NanoCP, which decouples MoE communication from KV cache placement and achieves dual balance through dynamic context parallelism (DCP). DCP assigns each request a context-parallel degree sized to its KV footprint: long requests distribute attention across multiple instances; short requests remain local. This dynamic parallelism effectively liquefies the KV cache across DP instances within each node, balancing both the per-instance KV cache occupancy and batch sizes without unnecessary load-balancing costs. To bridge DCP with static execution, NanoCP introduces an ahead-of-time (AOT) graph engine paired with a custom routing-based communication backend. Experimental results show that, under 99% TPOT-SLO attainment, NanoCP achieves $3.27\times$ and $2.11\times$ the maximum sustainable request rate of the best-performing vLLM baseline on the DeepSeek-V3 Issue 1% and Issue 5% workloads, respectively, while reducing P99 TPOT by $1.94\times$ and $2.01\times$ on the same two workloads, respectively.

cs.DC↗
arXiv2026

EP-NCO: Latency-Aware Service Placement using Neural Combinatorial Optimisers for Edge-Cloud Systems

Latency-sensitive Internet of Things (IoT) applications are increasingly deployed as interdependent microservices across heterogeneous edge-cloud infrastructures. Placing them under resource and network constraints is an NP-hard combinatorial optimisation problem. This study proposes EP-NCO, a graph-based neural combinatorial optimisation framework for service placement in compute continuum platforms. The objective accounts for execution latency, direct-link communication delay, and an approximate sharing of bandwidth among dependency flows that coexist in the resulting placement. Across held-out graph sizes generated by the same synthetic framework, the EP-NCO variant trained on Large instances with hard decoding achieved the lowest mean aggregate latency cost at every scale. It was 6.0%-12.5% lower than a genetic algorithm (GA), 14.4%-20.8% lower than particle swarm optimisation (PSO), and 1.2%-9.4% lower than the strongest reinforcement learning ablation. On Small instances, its mean cost was 10.5% below the best Gurobi solutions found and close to Gurobi's mean lower bound. The observed inference time of the EP-NCO variants was 0.88-1.03 s per instance, compared with 85-97 s for GA and PSO. As a result, the one-off training cost of the EP-NCO variant trained on Large instances with hard decoding is recovered after roughly 1,000 placements.

cs.DC↗
arXiv2026

Weight Placement Is Not Enough: Synchronization in Multi-Node CPU Decoding of Ternary LLMs

Ternary LLMs read only a few gigabytes per decoded token, yet on dual-socket servers with four NUMA nodes the fastest ternary runtime we found uses less than half of the memory bandwidth. Placing each weight on the node of the thread that reads it is necessary but not enough: in a decode engine with node-local weights that reuses ik_llama's kernels, threads on two generations of Xeon servers still spend 20-48% of each token waiting at synchronization points. Per-thread barrier traces show that most of the wait is for the slowest threads of each node; the barriers' own latency (microseconds each, but up to 12% of the token for the smallest model) and serial per-node work are smaller parts. On identical kernels, a Megatron-style column split beats a row-owned split without cross-node reductions in every configuration we measured, and isolating the number of barriers shows that most of its gain comes from its threads waiting less for each other and for their node leaders, and only a minority from having fewer barriers. Where decoding is DRAM-bound, the resulting engine is 1.5-2.1x faster than ik_llama with all cores and 1.5-1.9x faster than ik_llama at its fastest thread count (3.2x and 2.8x for a small model that runs from the L3 cache), at the same accuracy and with less energy per token.

cs.DC↗