A systolic update scheme to overcome memory bandwidth limitations in GPU-accelerated FDTD simulations

Abstract

The exponential growth of artificial intelligence has fueled the development of high-bandwidth photonic interconnect fabrics as a critical component of modern AI supercomputers. As the demand for ever-increasing AI compute and connectivity continues to grow, the need for high-throughput photonic simulation engines to accelerate and even revolutionize photonic design and verification workflows will become an increasingly indispensable capability for the integrated photonics industry. Unfortunately, the mainstay and workhorse of photonic simulation algorithms, the finite-difference time-domain (FDTD) method, because it is a memory-intensive but computationally-lightweight algorithm, is fundamentally misaligned with modern computational platforms which are equipped to deal with compute intensive workloads instead. This paper introduces a systolic update scheme for the FDTD method, which circumvents this mismatch by reducing the need for global synchronization while also relegating the need to access global memory to the case of boundary values between neighboring subdomains only. We demonstrate a practical implementation of our scheme as applied to the full three-dimensional FDTD algorithm that achieves a performance of roughly 0.15 trillion cell updates per second (TCUPS) on a single Nvidia H100 GPU. Our work paves the way for the increasingly efficient, cost-effective, and high-throughput photonic simulation engines needed to continue powering the AI era.