Compressing radio interferometric visibility data into a probabilistic model using sparse Gaussian processes

Abstract

Next-generation radio interferometers will produce massive data volumes, making it impractical to store original visibility measurements and later combine observations in $uv$ spatial frequency space. Visibility measurements at similar $uv$ locations measure the same signal but different noise realizations. In principle, these measurements can therefore be compressed by storing only the inferred mean visibility and its uncertainty. We propose modeling the visibility with a sparse Gaussian process (GP) and storing the resulting compact probabilistic model rather than raw visibilities. Using simulated Atacama Large Millimeter/submillimeter Array (ALMA) observations, we demonstrate that the sparse GP is flexible enough to represent the visibilities and recover images with high fidelity. We estimate compression factors of $10^3-10^5$ for an 8-hour Square Kilometre Array (SKA)-Mid observation, with further gains expected by extending the GP input space to include the spectral axis. Beyond data compression, the model exploits correlations in $uv$ space, boosting the signal-to-noise ratio compared with independent grid averaging. Once trained, the model can predict visibility and its uncertainty at any desired $uv$ coordinates, allowing imaging with arbitrary fields of view and image resolutions. The model may also be incrementally updated with new observations while filtering outliers based on the prediction.