Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.41160

ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference

Hung-Chun Lin·Ting-Wei Hsu·Chung-En Ho·Ahmed Saeed
View original source ↗Read source document ↗Export metadata ↓

Abstract

Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.

Explore related subjects

cs.NIcs.AIcs.DC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Hung-Chun Lin, Ting-Wei Hsu, Chung-En Ho, Ahmed Saeed. 2026-10-30. ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference. https://doi.org/10.1145/3789240.3828750

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Which Deep Learner? A Systematic Evaluation of Advanced Deep Forecasting Models Accuracy and Efficiency for Network Traffic Prediction

Network traffic prediction is an essential tool for automating modern network management. It is a difficult time series forecasting (TSF) problem that has been addressed by Deep Learning (DL) models due to their ability to capture complex patterns. Recent years have witnessed advances in deep forecasting models, from sophisticated transformer architectures to simple linear models, improving performance across diverse prediction tasks. However, given the diversity of these models, the variability of network traffic across environments and timescales, and the inherent challenges of network traffic prediction, it is essential to identify effective deployment choices and modeling directions. Henceforth, this study systematically identifies 12 deep forecasting models, comprising 8 advanced TSF models and 4 traditional deep learning architectures -- each with unique advantages for network traffic prediction -- against 3 statistical baselines, for a total of 15 forecasting methods and evaluate them on four real traffic datasets, across multiple time scales and horizons, assessing performance, robustness to anomalies, data gaps, external factors, data efficiency, and resource efficiency in terms of time, memory, and energy. Our results provide insights into model behaviour and guidelines for network management, highlighting performance regimes, efficiency thresholds, and promising architectures that balance accuracy, efficiency, and robustness to traffic challenges.

cs.NI↗
arXiv2026

ORCHID: Fairness-Aware Orchestration in Mission-Critical Air-Ground Integrated Networks

Uncrewed Aerial Vehicles (UAVs) provide on-demand wireless coverage for mission-critical Air-Ground Integrated Networks (AGINs). However, DRL-based orchestration faces challenges from multi-agent non-stationarity and the trade-off between energy efficiency (EE) and service fairness. To address these issues, we propose ORCHID (Orchestration of Resilient Coverage via Hybrid Intelligent Deployment), a stability-enhanced two-stage learning framework. First, GBS-aware topology partitioning provides a topology-informed warm start for UAV deployment. Second, a Reset-and-Finetune (R\&F) mechanism is integrated into Multi-Agent Proximal Policy Optimization (MAPPO) to support stable late-stage policy refinement through event-triggered rollback and learning-rate adaptation. To study the efficiency-fairness trade-off, ORCHID employs a Max-Min Fairness (MMF)-oriented bandwidth allocation based on a closed-form Egalitarian Bargaining Game (EBG) solution and is compared with Proportional Fairness (PF). Experiments show stronger served-user bottleneck-rate protection under MMF, together with competitive coverage and user-rate fairness and favorable energy-efficiency and UAV-load-balancing performance under zero-shot scalability and mobility tests. Overall, ORCHID provides an effective balance among user-level service protection, energy efficiency, and policy stability for resilient mission-critical UAV networks.

cs.NI↗
arXiv2026

Halley: A Predictive Control Plane for Optical AI Data Center Networks

A frontier AI model is trained in an AI data center, on tens of thousands of accelerators joined by a dedicated network, and every training step ends with all of them exchanging data at once. Between racks, that network is optical, and each of its four layers, from lasers to switches, runs a fixed or after-the-fact policy. Lasers sharing an enclosure, cooling, and power can fail together. At a fixed spare ratio, correlated failure provably caps a pool's availability however large it grows. In this paper, we present a comprehensive predictive control plane framework, called Halley, that reads each resource's next state and acts before the failure occurs. It has four components, each carrying a formal guarantee: LightReserve that sizes the laser pool to provide a coverage bound, FoldFit that schedules the switch within $11/9$ of the optimal, ThermaLead that holds a microring with one-step-ahead control, and TriageGuard that protects the highest-risk modules first. Halley gives each layer a predictive controller that defers to the deployed policy when unsure. Composed from simulations at that point, effective goodput, the share of ideal throughput kept, rises from 65.2\% to 76.5\% if a window that exhausts its spares is lost. If only the dark channels' bandwidth is lost, it reaches 67.4\%. Per megawatt, that is up to 1.03--1.17 times the training; for the same training, up to 3.3--14.7\% less data center capacity. \textsf{Halley}'s fallback circuits are formally verified and are shown to achieve close timing on a commercial FPGA, and pass a 130~nm signoff.

cs.NI↗