CuAD: Source-to-Source Automatic Differentiation of CUDA Kernels for Discrete Adjoints in Computational Fluid Dynamics

Abstract

Discrete adjoint methods give the sensitivities that aerodynamic design and data assimilation require, but they remain hard to obtain for flow solvers that run on graphics processors. This paper presents CuAD, a source-to-source automatic differentiation tool that reads CUDA kernels and writes their tangent and adjoint versions as ordinary CUDA source. Its reverse mode is organised around the GPU thread. It adopts the convention of CUDA kernels that inputs and outputs do not alias, recomputes intermediate values instead of storing them, differentiates pure accumulations without a tape and keeps any remaining tape in a fixed thread-local array. We evaluate CuAD on four kernels from simple flow models and on a two-dimensional Euler solver. All generated derivatives pass dot-product tests to round-off, and the adjoint kernels cost two to six times the primal kernels. On a Biot-Savart summation compiled by one compiler, the CuAD adjoint costs about four times the primal at every size, as does Enzyme when the source declares the absence of aliasing with __restrict__. On the unchanged source Enzyme caches the loop in the device heap and costs 9 to 240 times the primal, while the heap tape of Clad costs 60 to 4400 times the primal and fails beyond 8192 vortices. Exact Jacobians assembled from the generated tangents also matter for accuracy. In single precision, the adjoint gradient of a finite-difference Jacobian reaches errors of 45 to 75 per cent on a stiff Burgers problem and of $5\times10^{-4}$ to $4\times10^{-2}$ on a transonic airfoil, whereas the generated Jacobian keeps them below $5\times10^{-5}$ and $3\times10^{-6}$. The applications include the shape sensitivities of lift and drag of a transonic NACA 0012, for which the residual, the boundary conditions and the wall force are differentiated by CuAD and the adjoint agrees with the tangent to $10^{-13}$.