XRISM Pre-Pipeline and Singularity: Container-Based Data Processing for the X-Ray Imaging and Spectroscopy Mission and High-Performance Computing

Abstract

The X-Ray Imaging and Spectroscopy Mission (XRISM) is the seventh Japanese X-ray observatory whose development and operation are in collaboration with universities and research institutes in Japan, the United States, and Europe, including JAXA, NASA, and ESA. The telemetry data downlinked from the satellite are reduced to scientific products using pre-pipeline (PPL) and pipeline (PL) software running on standard Linux virtual machines (VMs) for the JAXA and NASA sides, respectively. OBSIDs identified the observations, and we had 80 and 161 OBSIDs to be reprocessed at the end of the commissioning period and performance verification and calibration period, respectively. The combination of the containerized PPL utilizing Singularity of a container platform running on the JAXA's "TOKI-RURI" high-performance computing (HPC) system and working disk images formatted to ext3 accomplished a 33x speedup in PPL tasks over our regular VM. Herein, we briefly describe the data processing in XRISM and our porting strategies for PPL in the HPC environment.