GHOST Driver

Azhar Shaikh

Home/Archive/arXiv

arXiv · 2611.03649

GHOST Driver

Azhar Shaikh

View original source ↗Read source document ↗Export metadata ↓

Abstract

AI inference workloads running on GPU clusters exhibit a well-documented utilization paradox: monitoring tools such as nvidia-smi regularly report GPU utilization figures above 80%, yet effective compute throughput is substantially lower. A significant fraction of reported utilization time is consumed by fence waits, driver submission overhead, DMA scheduling latency, and interrupt coalescing delays, none of which is visible to userspace profiling tools. These hidden latency components are embedded deep inside the Linux kernel GPU driver stack, primarily within the Direct Rendering Manager (DRM) subsystem, GEM buffer object lifecycle, and dma_fence synchronization infrastructure. Existing vendor profiling tools like CUPTI operate at the CUDA userspace boundary and provide no visibility into kernel-side driver paths. We present GHOST-Driver (GPU Host-Side Observability and Tracing for Drivers), a non-intrusive eBPF-based framework that attaches to stable DRM kernel tracepoints and select kprobes to expose the full latency decomposition of the GPU driver submission path with sub-microsecond resolution and CPU overhead below 2% in production. GHOST-Driver tracks six key submission stages (ioctl entry, GEM BO validation, dma_fence dependency resolution, command ring emission, DMA engine scheduling, and interrupt fence signaling) and correlates them with per-tenant vGPU context IDs for multi-tenant attribution in SR-IOV and MIG environments, exporting trace metrics to Prometheus and OpenTelemetry.

Explore related subjects

cs.OS cs.DC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Azhar Shaikh. 2026-07-27. GHOST Driver. https://arxiv.org/abs/2611.03649

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

UMLoad: Fast On-Demand Model Loading on Unified Memory Architecture

On-demand model loading dominates response latency for interactive on-device AI, yet unified memory architectures (UMAs), now common on edge devices, should make it cheap by placing weights in memory the GPU already sees. However, frameworks still use the loading flow built for discrete GPUs, leaving significant inefficiencies. This paper presents UMLoad, a system-level model-loading accelerator that decouples a framework's logical allocations from their physical page backing. UMLoad uses on-demand reconstruction to eliminate the weight copy while preserving copy independence, and GPU-assisted page zeroing to accelerate runtime allocation while preserving its security guarantee. Implemented on Intel, AMD, and NVIDIA UMA platforms and evaluated with llama.cpp, PyTorch, and vLLM across six models, UMLoad reduces average loading time by 36.6%-71.9% over vanilla and 19.9%-62.7% over pipelined loaders, and cuts mean LLM-routing time to first token (TTFT) by 51.3%-59.3% and model-backed tool completion latency by 10.2%-31.6%.

cs.OS↗

arXiv2026

ToolSched: Resource-Efficient Tool Scheduling for Shared-Host LLM Agents

Local LLM agents often work on shared hosts. Unlike multi-tenant cloud clusters, these hosts often have tightly bounded memory, CPU, and I/O resources. Concurrent agents frequently suffer from resource contention, leading to failures such as OOM and reduced resource efficiency. Considering that agents' resource demands are bursty and shell command delay is imperceptible to agents, there is a huge scheduling opportunity to arrange when and in what order commands are actually executed. Thus, we present ToolSched, a Shell MCP Server for shared local hosts that serves multiple agents and schedules their operations. ToolSched embeds a resource model into the MCP interface: agents estimate resource demands along with command requests. These hints, together with real-time host resource conditions, determine when commands are actually executed. Internally, ToolSched applies a scheduling algorithm to stagger commands, reducing contention and improving resource utilization. Prediction correction and checkpoint-based recovery mitigate estimation bias and resource errors. Our evaluation on Terminal Bench shows that, compared with unscheduled execution, ToolSched improves resource utilization by 148.4% and end-to-end command throughput by 29.4%, demonstrating its effectiveness and practicality.

cs.OS↗

arXiv2026

High-Throughput Private Information Retrieval with on-CPU Accelerators

Private information retrieval (PIR) hides which record a client retrieves by scanning the entire database, and batching is an effective technique to improve throughput by sharing this scan across queries. However, existing CPU-based solutions gain little from batching because each query still requires its own multiplications, and vector instructions perform only a few at a time. GPUs and other discrete accelerators can accelerate these computations, but require extra hardware and can constrain database size due to their limited memory capacity. In this paper, we present AltoPIR, the first PIR system that leverages on-CPU matrix accelerators such as Intel AMX. AltoPIR keeps the accelerator fed from the CPU caches, reading each database byte from memory once for all queries in a batch and never writing intermediate products to memory. This enables high-performance PIR through efficient batching without requiring discrete accelerators. Our evaluation shows that AltoPIR achieves up to 50.1x higher throughput than existing CPU-based solutions and serves databases larger than a GPU's memory.

cs.OS↗