Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2611.07198

Capability Gain from ISA to Shared Execution on AMD MI300A and MI250X

Aaron Jarmusch·Sunita Chandrasekaran
View original source ↗Read source document ↗Export metadata ↓

Abstract

Each new GPU generation is showcased by its peak instruction rates, yet users hardly ever run instructions directly: they reach the hardware's performance through vendor libraries, applications and devices shared with other jobs. How much of a theoretical speedup survives these layers is rarely measured, so a team deciding whether to move to a new generation, or to adopt FP8 or 2:4 sparsity, lacks adequate guidance. AMD's MI300A (CDNA3) adds new FP8 matrix instructions and 2:4 structured-sparse instructions that the predecessor, the MI250X (CDNA2), does not have. We analyze this theoretical speedup gap on AMD's MI300A (CDNA3) against the MI250X (CDNA2) by defining a new metric, capability gain. Capability gain is the fraction of a feature's instruction-level speedup that it keeps at a higher level, such as a library, an application or a shared device. We trace it from ISA microbenchmarks through hipBLASLt and hipSPARSELt to LLM inference and mixed-precision HPL, and onto a device shared by up to eight concurrent streams. FP8 keeps most of its speedup through the library, completing an $8192^3$ GEMM \genCapPkgEndToEndGainEightK{} faster than the MI250X with FP16, but unfused quantization turns it into a per-layer slowdown in an LLM, and 2:4 sparsity keeps at most about half of its speedup in hipSPARSELt. Under queue sharing, FP8's GEMM capability gain rises above 1. How fairly streams share the device depends on the number of hardware queues, not on the numeric precision, yet shared LLM serving gains nothing from FP8.

Explore related subjects

cs.DC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Aaron Jarmusch, Sunita Chandrasekaran. 2026-11-04. Capability Gain from ISA to Shared Execution on AMD MI300A and MI250X. https://arxiv.org/abs/2611.07198

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

FedDyMem: Efficient Federated Learning with Dynamic Memory and Memory-Reduce for Unsupervised Image Anomaly Detection

Unsupervised image anomaly detection (UAD) has become a critical process in industrial and medical applications, where models are trained on normal samples without requiring labeled anomalies. However, even normal images may contain sensitive product designs, manufacturing information, or institution-specific population characteristics, making centralized data collection difficult in practical deployments. Federated UAD provides a promising decentralized solution, but it remains challenging because clients use only normal training samples, their feature distributions remain highly heterogeneous within the same task, and communication should be efficient while limiting the exposure of sensitive client information. To address these challenges, we propose FedDyMem, an efficient federated learning method with dynamic memory and memory-reduce for unsupervised image anomaly detection. Instead of exchanging full model parameters, FedDyMem facilitates knowledge sharing between the clients and server through reduced dynamic memory banks. In the local clients, a memory generator and a metric loss are employed to improve the consistency of the feature distribution for normal samples, leveraging the local model to update the memory bank dynamically. For efficient communication and data privacy, a memory-reduce method based on weighted averages is proposed to significantly decrease the scale of memory banks. This representation reduces communication overhead and empirically lowers reconstruction fidelity under inversion attacks. On the server, global memory is constructed and distributed to individual clients through k-means aggregation. Experiments on six federated UAD tasks constructed from eleven public industrial and medical datasets show that FedDyMem consistently improves anomaly detection and localization while reducing communication cost.

cs.DC↗
arXiv2026

Twig: Parallel and Memory-Efficient Tree Speculative Decoding for Hybrid-Attention LLMs

Emerging LLMs increasingly interleave full attention with recurrent linear attention to reduce long-context serving costs. Tree speculative decoding could further accelerate these hybrid models, but recurrent linear-attention layers carry parent-dependent states. Existing engines therefore verify proposal trees sequentially and snapshot the full state per node, causing latency and memory to grow with tree and batch sizes. We present Twig, a kernel--runtime co-design for tree speculative decoding of hybrid LLMs. Twig derives a tree-structured factorization of the linear-attention recurrence and realizes it with a resource-efficient GPU kernel that verifies all nodes in parallel, accelerating linear-attention tree verification by 3.4--7.7$\times$. It losslessly represents speculative state updates as token-level factors and reconstructs only the accepted state, reducing transient state memory by 82--99$\times$ and freeing GPU memory for KV caches. We integrate Twig into SGLang, a widely deployed production LLM serving engine, and further optimize its runtime through batch-wide verification budgeting. Across four models, two GPU platforms, and diverse datasets, Twig achieves up to $4.72\times$ the offline decode throughput of autoregressive decoding and up to $2.03\times$ that of the strongest tree-speculative baseline. Under online agent workloads, Twig reduces TTFT and TPOT by up to $67.6\%$ and $49.9\%$, respectively, over the strongest tree-speculative baseline.

cs.DC↗
arXiv2026

Lantern: Finding Committable Transactions via Back-Propagation on DAGs

Existing concurrency control protocols either introduce nondeterminism, resulting in a serial execution-replay dependency between primary and replica nodes, or rely on impractical prior knowledge of transaction read-write sets. In this paper, we present Lantern, a deterministic concurrency control protocol tailored for high-performance transaction processing systems operating without prior knowledge. The key insight of Lantern is that all zero-out-degree transaction vertices in the local dependency graph can be safely committed in ascending order using an overwrite-permissive strategy. We further introduce a novel Back-Propagation mechanism that iteratively propagates dependency states from sink to source vertices to identify additional committable transactions. We also propose Conflict-Free Batch Selection (CFBS) for read-modify-write intensive scenarios. We integrate Lantern into the open-source blockchain platform ChainMaker. Extensive evaluations on YCSB and SmallBank benchmarks demonstrate that Lantern achieves up to a 4.2x throughput speedup over Aria and improves the throughput of ChainMaker's execution layer by at least 2.2x.

cs.DC↗