Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.32279

TranSAGE: Automating Spatial Accelerator Generation for Large-Scale Transformer Models

Chi Zhang·Yi Sun·Weichuang Zhang·Quan Chen·Jieru Zhao
View original source ↗Read source document ↗Export metadata ↓

Abstract

Large-scale Transformers have achieved remarkable success across NLP, computer vision, and robotics, but their growing computational and memory demands challenge efficient inference. Spatial accelerators offer advantages in on-chip bandwidth and energy efficiency, yet designing them for large models remains costly and time-consuming. Existing accelerator generators typically focus on single-operation support, overlooking the benefits of multi-operation pipelining, which limits throughput and energy efficiency. We propose TranSAGE, a compiler-based framework that jointly optimizes scheduling for large-scale Transformers under multi-operation pipelining and generates corresponding accelerator implementations. Our approach integrates a design space exploration (DSE) engine with multi-operation dataflow optimization and a dynamic programming-based graph partitioning algorithm to efficiently identify pipeline-friendly operation clusters and their optimal dataflow parameters. We further design spatial architecture primitives and lowering passes to automatically generate efficient RTL implementations. Evaluations on four representative LLMs demonstrate that TranSAGE achieves up to 2.41x higher throughput and 13.04x higher energy efficiency than commercial GPU/NPU baselines. Compared with state-of-the-art accelerator generators, TranSAGE achieves up to 11.33x higher throughput and 6.07x higher energy efficiency. Our code is available at https://github.com/sjtu-zhao-lab/TranSAGE.

Explore related subjects

cs.AR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Chi Zhang, Yi Sun, Weichuang Zhang, Quan Chen, Jieru Zhao. 2026-10-23. TranSAGE: Automating Spatial Accelerator Generation for Large-Scale Transformer Models. https://arxiv.org/abs/2610.32279

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

HAWK: Heterogeneity-Aware KV Cache Transfer for Efficient GPU-NPU Phase-Disaggregated LLM Inference

Large language model (LLM) inference exhibits distinct computational characteristics across its two phases: prefill is compute-intensive, whereas decode is more memory-bound. As a result, AI accelerators achieve varying levels of efficiency across the two phases. GPUs are well suited to prefill, but are often underutilized during decode. By contrast, NPUs can utilize their compute resources more effectively during decode. Motivated by this observation, we explore a heterogeneous architecture that integrates GPUs and NPUs, mapping prefill to GPUs and decode to NPUs to achieve high performance and efficiency. However, realizing such heterogeneous integration is challenging because LLM phase disaggregation requires KV cache transfer from prefill to decode (that is, from GPUs to NPUs), which is complicated by differences in inter-device communication mechanisms and KV cache tensor layouts. To address this, we propose HAWK, a transfer mechanism that enables rapid KV cache transfer across heterogeneous devices. HAWK establishes a remote direct memory access (RDMA) path for direct cross-device transfers while handling inter-device tensor layout discrepancies as part of the transfer process. In particular, HAWK allocates tensor-layout-aware RDMA buffers on the NPU side to match the target tensor layout. Leveraging these buffers, HAWK orchestrates data transfer from a conventional contiguous GPU RDMA buffer to the corresponding NPU buffers, allowing transferred tensors to be consumed directly without additional data rearrangement while maintaining transfer efficiency. We implement and evaluate HAWK on a system built with commercially available GPUs and NPUs, demonstrating performance within 7.88% of a GPU-only system under the same SLO constraints while achieving up to 1.26$\times$ higher power efficiency and 1.72$\times$ higher cost efficiency.

cs.AR↗
arXiv2026

PARIS: Parallel Arithmetic Reduction and Integrated Selection for MSDF Inner Products

Existing most-significant-digit-first (MSDF) accelerators serialize input-term processing, causing inner-product latency to increase with fan-in. This paper presents PARIS, an MSDF architecture that eliminates input-term serialization through parallel bit-plane reduction and shared carry-save residual selection. For $B$-bit operands, PARIS processes $B^2$ bit-position pairs, independent of fan-in within the supported tile, while generating exact MSDF outputs. Synthesized in 45\,nm at 100\,MHz, the 784-lane INT8 core occupies 0.008701\,mm$^2$ and completes an inner product in 88 cycles. Evaluation on two MNIST multilayer perceptrons demonstrates arithmetic-workload speedups of 240.44$\times$ and 180.47$\times$ over a matched reused-MSDF implementation, reducing arithmetic energy by 98.89\% and 98.58\%, respectively. A synthesized 16-engine accelerator occupies 0.90273\,mm$^2$ and meets the 100\,MHz timing constraint. Assuming immediate weight availability, it delivers a projected throughput of 24.33\,GOPS and an estimated energy efficiency of 1,581.14\,GOPS/W at unchanged mapped power. Bias-aware ReLU detection further reduces modeled arithmetic cycles by 5.52\% and 4.86\%. A separate 256-engine RTL implementation completes Model A in 259 cycles, demonstrating output-parallel scalability.

cs.AR↗
arXiv2026

Automated Multi-Line Bug Repair Using Model Checker Aided LLMs: A Case Study on RISC-V Processors

Functional verification of Register-Transfer Level(RTL) designs has become a critical bottleneck in modern semiconductor development, often consuming the majority of project resources. Debugging, in particular, remains an especially time-consuming task, requiring careful analysis of simulation traces or counterexamples. Recent work has explored applying Large Language Models (LLMs) to RTL debug and repair with great promise. However, the existing approaches typically are for single-line bugs, operate in open-loop, rely heavily on prompts or retrieval, lack scalability, and often produce incorrect fixes. In this paper, we propose a novel automated RTL bug repair methodology that combines LLMs, assertions, and a model checker to provide scalable, trustworthy semantic grounding and formal guarantees. Concretely, the methodology leverages failing assertions to guide the LLM in generating patches for multi-line bugs. Each candidate patch is formally verified using a model checker (JasperGold), ensuring only valid repairs are accepted. We evaluate the efficacy and scalability of our methodology on the BRISC-V RISC-V core using mutation testing, introducing several mutation types, and demonstrate that our methodology can repair up to 98% of single-line RTL bugs and 93% of the multi-line bugs. To the best of our knowledge, this is the first work to show automated, LLM-assisted RTL multi-line bug repair results on a complex RISC-V processor design in a scalable manner.

cs.AR↗