Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1912.03349

Data Replication for Reducing Computing Time in Distributed Systems with Stragglers

Amir Behrouzi-Far·Emina Soljanin
View original source ↗Read source document ↗Export metadata ↓

Abstract

In distributed computing systems with stragglers, various forms of redundancy can improve the average delay performance. We study the optimal replication of data in systems where the job execution time is a stochastically decreasing and convex random variable. We show that in such systems, the optimum assignment policy is the balanced replication of disjoint batches of data. Furthermore, for Exponential and Shifted-Exponential service times, we derive the optimum redundancy levels for minimizing both expected value and the variance of the job completion time. Our analysis shows that, the optimum redundancy level may not be the same for the two metrics, thus there is a trade-off between reducing the expected value of the completion time and reducing its variance.

Explore related subjects

cs.DCcs.ITmath.IT

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Amir Behrouzi-Far, Emina Soljanin. 2019-12-31. Data Replication for Reducing Computing Time in Distributed Systems with Stragglers. https://arxiv.org/abs/1912.03349

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Large Language Model based air quality monitoring and localized alert generation

Poor indoor air quality can cause up to five times more direct health problems to occupants than outdoor air. In particular, it may cause headaches, fatigue, eye/throat irritation, and long-time exposure is linked to respiratory and heart as well as some forms of cancer. Despite the importance of indoor health and well-being, most current monitoring devices and systems (usually for offices and workspaces) are passive. The Environmental Quality Monitor (EnQyMo) platform is a generic Internet of Things (IoT) middleware designed to process several sensor data related to air quality in indoor spaces and correlate this data with health exposure risks of users/workplace employees. Using Bluetooth Low Energy (BLE) beacons and a mobile IoT middleware it is able to identify the (smartphone) users exposed to these polluted air or high CO2 (carbon dioxide) levels, and generate location-specific alarms only to the users at the places with the unhealthy air conditions. At the core of EnQyMo is an agency of Large Language Models (LLMs) capable of interpreting regulatory standards and scientific literature to automatically identify critical health exposure levels.

cs.DC↗
arXiv2026

SSD-LLaMA: SSD-Native Inference for Trillion-Parameter MoE at 1+ Token/s on a Consumer PC

Frontier open-weight language models increasingly use Mixture-of-Experts (MoE) architectures to expand model capacity while activating only a small subset of experts per token. Local inference must nevertheless keep the complete expert pool available, which remains far beyond consumer-grade RAM and VRAM capacity even after quantization. SSDs provide practical capacity at this scale, but turning that capacity into executable model memory requires efficient expert delivery, coordinated management of SSD, RAM, and VRAM, and CPU--GPU hybrid execution under bounded bandwidth. We present \textit{SSD-LLaMA}, an SSD-native local MoE inference system that addresses these challenges with an SSD I/O pipeline optimized for expert delivery, a native three-tier storage hierarchy that delivers and retains experts dynamically, and balanced CPU--GPU hybrid execution. \textit{SSD-LLaMA} executes every selected expert without pruning or substitution. Across three frontier MoE model families, \textit{SSD-LLaMA} improves prefill token rate by 1.52$\times$--4.19$\times$ and decode token rate by 2.10$\times$--15.58$\times$ over the evaluated baselines. We also achieve higher than 1 token/s for running trillion-parameter model with a single RTX 5090 and no more than 32GB RAM.

cs.DC↗
arXiv2026

Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving

LLM serving is typically offered as a shared, multi-tenant service, where high-demand workloads from one client can cause latency SLO violations for others. Existing solutions for performance isolation equalize client throughput in the long run, for example through queueing and batching fairness. However, these approaches do not provide latency isolation guarantees; as a result, well-behaved clients can still experience significant degradation to their token-level latencies. In this paper, we present FairInference, which provides the novel δ-token fairness guarantee: for a well-behaved client, if a token is generated in d time units in isolation, it will be generated within d + δ time units in multi-tenant execution, providing strong latency isolation guarantees for LLM serving. To achieve this, FairInference addresses a key challenge of LLM serving: bounding delays from sharing GPU resources without support for fine-grained scheduling or resource allocation. In FairInference, the scheduler enforces per-token deadlines, while bounding the delays from GPU compute sharing and accounting for the additional delays introduced by the shared KV caching in GPU memory. We show that FairInference effectively bounds token-level latency spikes for well-behaved clients and improves overall throughput compared to state-of-the-art LLM serving systems.

cs.DC↗