Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1905.04029

The Data Processing Pipeline for the Herschel-HIFI Instrument

K. Edwards·R. F. Shipman·D. Kester·A. Lorenzani·M. Melchior
View original source ↗Read source document ↗Export metadata ↓

Abstract

The HIFI data processing pipeline was developed to systematically process diagnostic, calibration and astronomical observations taken with the HIFI science instrumentas part of the Herschel mission. The HIFI pipeline processed data from all HIFI observing modes within the Herschel automated processing environment, as well as, within an interactive environment. A common software framework was developed to best support the use cases required by the instrument teams and by the general astronomers. The HIFI pipeline was built on top of that and was designed with a high degree of modularity. This modular design provided the necessary flexibility and extensibility to deal with the complexity of batch-processing eighteen different observing modes, to support the astronomers in the interactive analysis and to cope with adjustments necessary to improve the pipeline and the quality of the end-products. This approach to the software development and data processing effort was arrived at by coalescing the lessons learned from similar research based projects with the understanding that a degree of foresight was required given the overall length of the project. In this article, both the successes and challenges of the HIFI software development process are presented. To support future similar projects and retain experience gained lessons learned are extracted.

Explore related subjects

astro-ph.IM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

K. Edwards, R. F. Shipman, D. Kester, A. Lorenzani, M. Melchior. 2019-05-10. The Data Processing Pipeline for the Herschel-HIFI Instrument. https://doi.org/10.1016/j.ascom.2019.04.003

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Scale-Vector Alignment: A Scale-Aware Framework for Spatially Resolved Morphological Similarity in Astronomical Images

Astronomical maps made with different tracers are not expected to have identical morphology. Excitation, optical depth, chemistry, radiation, and ISM phase alter the response of a tracer, and the resulting differences can depend on both position and spatial scale. We propose scale-vector alignment, a scale-aware method based on Constrained Diffusion Decomposition (CDD). CDD decomposes an image into localized scale components; at each position, their amplitudes define a scale vector that describes how the measured intensity is distributed over spatial scale. We define the pixel-wise similarity $\Spix(x,y)$ as the normalized alignment of two local scale vectors. The normalization removes the overall amplitude, so $\Spix$ compares relative scale composition rather than absolute flux. We also define the scale-wise similarity $\Sscale(l)$ by comparing the two CDD component maps at each spatial scale. Spatial shifts are used to construct an empirical shifted reference distribution for $\Spix$. In Orion~A, the tracer with the highest similarity to the dust-derived column-density map changes from $^{12}$CO to $^{13}$CO to C$^{18}$O toward higher column density. In NGC~6334I(N), the line--continuum similarity decreases locally around the brightest compact structures, where radiative-transfer effects can alter the observed line morphology. In NGC~3627, CO is most similar to 21~$μ$m emission, and $\Sscale$ reaches its maximum at an intermediate sub-kpc scale. The method measures where two tracers have similar multiscale structure and at which scales their spatial distributions agree. The implementation is publicly available at https://github.com/meng-ke/Scale-Vector-Alignment.

astro-ph.IM↗
arXiv2026

Fast and accurate astronomical source deblending with Density-Peak Clustering

Source deblending is a fundamental challenge for current and forthcoming astronomical surveys, where increasing source density and image depth lead to a growing number of overlapping detections. Accurate deblending is essential for reliable measurements of source morphology and photometry, as well as for cosmological analyses. We present a redesign of the Advanced Density Peak (ADP) clustering algorithm, tailored to the identification and separation of blended astronomical sources within detection regions. We develop a validation framework combining realistic image simulations, automatically generated ground-truth segmentation, and label-invariant metrics. ADP is assessed against ASTErIsM, an established density-based astronomical deblender, using pairwise simulations, synthetic multi-source images, and Euclid Q1 public data. In pairwise simulations, the methods show comparable performance across a broad range of source separations and flux ratios, with photometric differences typically below 1% and reaching 5-7% in the most challenging cases, without systematic bias. In multi-source simulations, ADP recovers approximately 8% more ground-truth sources, while the positions of sources identified by both methods agree at the sub-pixel level. On Euclid Q1 public data, the methods show strong agreement in segmentation area, ellipticity, position angle, and photometry, with the largest differences for the smallest and faintest sources. ADP also provides a substantial computational advantage: end-to-end benchmarks on 19200 x 19200 pixel Euclid images require approximately 16-200 s, corresponding to speedups of 12-156x relative to ASTErIsM, with median and mean improvements of 36x and 54x, respectively. These results show that ADP provides scientifically competitive deblending at substantially lower computational cost, making it a promising approach for large-scale astronomical imaging surveys.

astro-ph.IM↗
arXiv2026

IceCube Upgrade status and perspectives

The IceCube Neutrino Observatory instruments one cubic kilometer of deep-glacial ice between 1450 m and 2450 m below the surface at the geographic South Pole to detect neutrinos via Cherenkov radiation of relativistic charged particles produced in their interactions. This detector is responsible for a number of key observations in neutrino astrophysics, which include the discovery of a high-energy astrophysical neutrino flux and, more recently, the galactic plane. During the austral summer of 2025/26, five new strings equipped with new photosensor designs were deployed as a dense infill in the middle of the existing detector. The science goals of this detector are twofold: Firstly, given the higher photocathode density, an improved atmospheric neutrino event selection and reconstruction at a few GeV can be achieved for enhanced capabilities to study neutrino oscillations. Secondly, novel calibration devices will improve the knowledge of the optical properties of the glacial ice and the detector response. These new calibration results will be applied to archival IceCube data, improving angular and spatial resolution of all detected astrophysical neutrino events. The IceCube Upgrade also serves as a first step towards the next-generation neutrino telescope at the South Pole, called IceCube-Gen2.

astro-ph.IM↗