Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2309.04011

CXLMemUring: A Hardware Software Co-design Paradigm for Asynchronous and Flexible Parallel CXL Memory Pool Access

Yiwei Yang·Yusheng Zheng·Kexin Chu·Jianchang Su·Jesun Firoz·Andi Quinn
View original source ↗Read source document ↗Export metadata ↓

Abstract

CXL-attached memory lets servers add more memory while keeping the standard load/store programming model. The main drawback is latency. CXL memory accesses are too slow for normal CPU mechanisms to hide reliably, especially when each access depends on the result of a previous one. At the same time, they are too fast for traditional software techniques, such as context switches or interrupt-based asynchrony, to manage one load at a time. On a real Granite Rapids CXL platform, we find that placing GAPBS graph workloads in CXL memory slows execution by 2.44x on average compared with local DRAM. A state-of-the-art software prefetcher still leaves a 2.21x slowdown. This paper presents our system, a hardware/software co-designed approach for hiding CXL latency using larger units of asynchronous work. The system creates regions: parts of the original program that include one or more CXL-resident memory operations together with the nearby address-generation and memory-orchestration logic needed to run them. The host CPU launches these regions asynchronously on a near-memory accelerator built on a commodity CXL Type 2 FPGA, then continues useful work while the device runs ahead and prepares future memory accesses. A compiler identifies candidate regions from unmodified source programs, and an online JIT refines region boundaries and execution parameters based on workload behavior. We implement the system as a prototype compiler, runtime, and Vortex-based CXL-side accelerator. Across GAPBS, MCF, Spatter, and NAS Parallel Benchmark workloads, it improves end-to-end performance by 1.45x to 1.75x, with a 1.59x geometric mean speedup, compared with native CXL-memory execution.

Explore related subjects

cs.AR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yiwei Yang, Yusheng Zheng, Kexin Chu, Jianchang Su, Jesun Firoz, Andi Quinn. 2026-06-22. CXLMemUring: A Hardware Software Co-design Paradigm for Asynchronous and Flexible Parallel CXL Memory Pool Access. https://arxiv.org/abs/2309.04011

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A System Architecture for Low Latency Multiprogramming Quantum Computing

As quantum systems scale, multiprogramming quantum computing (MPQC) provides a practical way to improve device utilization and throughput. However, because quantum executables are device-dependent, non-portable across qubit regions, and highly susceptible to noise and crosstalk, current MPQC pipelines rely on expensive online compilation to co-optimize concurrently running programs. This online step dominates runtime and impedes low-latency deployments for practical, real-world workloads in the future, such as repeatedly invoked quantum neural network (QNN) services. We present FLAMENCO, a fidelity-aware multi-version compilation system that enables independent offline compilation and low-latency multiprogramming at runtime. \textbf{At the architecture level}, the system abstracts devices into compute units to reduce the search space of region allocation. \textbf{At compile time}, it generates diverse executable versions for each program---each bound to a distinct qubit region---allowing dynamic region selection at runtime and overcoming non-portability. \textbf{At runtime}, it employs a lightweight orchestrator that uses post-compilation fidelity metrics to avoid conflicts and mitigate crosstalk, supporting conflict-free co-execution without online co-optimization. Evaluations show that FLAMENCO achieves over 5$\times$ runtime speedup in post-scheduling execution while maintaining comparable execution fidelity on common-success workloads. When integrated into existing scheduler-coupled systems, it raises workload-level conflict-free orchestration ratio from 0.183 to 1.000 for HyperQ and from 0.050 to 0.400 for QOS.

cs.AR↗
arXiv2026

Microflow: Microarchitectural Causal Observability for Deep Cross-Layer Analysis and Optimization

Modern computer architecture relies heavily on simulation to identify bottlenecks and evaluate optimizations. However, existing microarchitectural performance analysis methods are fundamentally limited by an instruction-centric paradigm that captures only downstream symptoms while leaving the true microarchitectural root cause opaque. Because modern processors are governed by complex interactions across non-instruction entities like prefetchers, replacement policies, and shared queue occupancies, instruction-centric frameworks miss the mechanisms that dictate performance. To eliminate this blind spot, we present Microflow, a framework that achieves causal observability in microarchitectural simulation. To address this, we introduce the Microflow intermediate representation (MFIR), which models execution through microarchitecture-tailored core abstractions such as flows, resource tenancies, and causal edges. By compiling simulation runs into a relational causal database, Microflow decouples tracing from analytical processing. This transforms complex diagnostics into expressive queries, enabling architects to trace performance symptoms directly to hardware root causes without developing bespoke analysis scripts or running costly re-simulations for every new question. We demonstrate that Microflow solves pathologies opaque to conventional tools. Across CVP-1 benchmarks, Microflow decomposes a 22% prefetcher oracle headroom by attributing 61.4% of stall mass to specific hardware prefetcher decisions, yielding a 2.31% average speedup (peaking at 25.11%). Furthermore, it exposes the hidden pipeline-blocking residue of wrong-path execution with high portability and precision across simulators.

cs.AR↗
arXiv2026

On the Limits of Machine-Learned Ranking for Modern Microarchitectural Policies

Machine-learning predictors estimate processor performance far faster than cycle-level simulation. For design-space exploration, however, the valuable test is not merely reproducing the usual hardware ordering, but identifying how different hardware configurations rank on individual program phases. We evaluate four ML-predictors in two design regimes: \emph{Structural Parameters} (SP), varying hardware resources such as issue width, ROB size, and cache capacity; and \emph{Behavioral Policies} (BP), varying prefetching and replacement algorithms. In the SP regime, aggregate ranking is strong, yet counter-intuitive windows(CIW)---where the configuration expected to be slower is faster---constitute $22.4\%$ of non-tied windows across five pairs with a clear architectural prior. CIW match across these pairs is only $23.3$--$39.9\%$; every point estimate is below the $50\%$ random strict-ordering reference. The BP regime presents a different failure: ground-truth ties cover $37.8\%$ of pair-windows, most strict pairs have margins of only a few cycles, and no model family reliably beats a feature-free majority baseline. NeuroScalar and SimNet fall below that baseline, Concorde is statistically tied with it, and the best selected OneDSE head improves by only $2.1$ percentage points. Accuracy rises mainly at large margins. We further show that this failure is not a matter of model capacity: an information-theoretic analysis reveals that when ranking outcomes depend on hidden microarchitectural state absent from the instruction stream, no trace-based predictor can exceed the Bayes accuracy determined by observable inputs alone. Thus high cycle or aggregate ranking accuracy can reflect mastery of easy, high-margin cases while missing the local reversals that carry the most architectural insight and for which cycle-level simulation remains indispensable.

cs.AR↗