Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2208.02781

From Data to Software to Science with the Rubin Observatory LSST

Katelyn Breivik·Andrew J. Connolly·K. E. Saavik Ford·Mario Jurić·Rachel Mandelbaum·Adam A. Miller·Dara Norman·Knut Olsen·William O'Mullane·Adrian Price-Whelan·Timothy Sacco·J. L. Sokoloski·Ashley Villar·Viviana Acquaviva·Tomas Ahumada·Yusra AlSayyad·Catarina S. Alves·Igor Andreoni·Timo Anguita·Henry J. Best·Federica B. Bianco·Rosaria Bonito·Andrew Bradshaw·Colin J. Burke·Andresa Rodrigues de Campos·Matteo Cantiello·Neven Caplar·Colin Orion Chandler·James Chan·Luiz Nicolaci da Costa·Shany Danieli·James R. A. Davenport·Giulio Fabbian·Joshua Fagin·Alexander Gagliano·Christa Gall·Nicolás Garavito Camargo·Eric Gawiser·Suvi Gezari·Andreja Gomboc·Alma X. Gonzalez-Morales·Matthew J. Graham·Julia Gschwend·Leanne P. Guy·Matthew J. Holman·Henry H. Hsieh·Markus Hundertmark·Dragana Ilić·Emille E. O. Ishida·Tomislav Jurkić·Arun Kannawadi·Alekzander Kosakowski·Andjelka B. Kovačević·Jeremy Kubica·François Lanusse·Ilin Lazar·W. Garrett Levine·Xiaolong Li·Jing Lu·Gerardo Juan Manuel Luna·Ashish A. Mahabal·Alex I. Malz·Yao-Yuan Mao·Ilija Medan·Joachim Moeyens·Mladen Nikolić·Robert Nikutta·Matt O'Dowd·Charlotte Olsen·Sarah Pearson·Ilhuiyolitzin Villicana Pedraza·Mark Popinchalk·Luka C. Popović·Tyler A. Pritchard·Bruno C. Quint·Viktor Radović·Fabio Ragosta·Gabriele Riccio·Alexander H. Riley·Agata Rożek·Paula Sánchez-Sáez·Luis M. Sarro·Clare Saunders·Đorđe V. Savić·Samuel Schmidt·Adam Scott·Raphael Shirley·Hayden R. Smotherman·Steven Stetzler·Kate Storey-Fisher·Rachel A. Street·David E. Trilling·Yiannis Tsapras·Sabina Ustamujic·Sjoert van Velzen·José Antonio Vázquez-Mata·Laura Venuti·Samuel Wyatt·Weixiang Yu·Ann Zabludoff
View original source ↗Read source document ↗Export metadata ↓

Abstract

The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) dataset will dramatically alter our understanding of the Universe, from the origins of the Solar System to the nature of dark matter and dark energy. Much of this research will depend on the existence of robust, tested, and scalable algorithms, software, and services. Identifying and developing such tools ahead of time has the potential to significantly accelerate the delivery of early science from LSST. Developing these collaboratively, and making them broadly available, can enable more inclusive and equitable collaboration on LSST science. To facilitate such opportunities, a community workshop entitled "From Data to Software to Science with the Rubin Observatory LSST" was organized by the LSST Interdisciplinary Network for Collaboration and Computing (LINCC) and partners, and held at the Flatiron Institute in New York, March 28-30th 2022. The workshop included over 50 in-person attendees invited from over 300 applications. It identified seven key software areas of need: (i) scalable cross-matching and distributed joining of catalogs, (ii) robust photometric redshift determination, (iii) software for determination of selection functions, (iv) frameworks for scalable time-series analyses, (v) services for image access and reprocessing at scale, (vi) object image access (cutouts) and analysis at scale, and (vii) scalable job execution systems. This white paper summarizes the discussions of this workshop. It considers the motivating science use cases, identified cross-cutting algorithms, software, and services, their high-level technical specifications, and the principles of inclusive collaborations needed to develop them. We provide it as a useful roadmap of needs, as well as to spur action and collaboration between groups and individuals looking to develop reusable software for early LSST science.

Explore related subjects

astro-ph.IM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Katelyn Breivik, Andrew J. Connolly, K. E. Saavik Ford, Mario Jurić, Rachel Mandelbaum, Adam A. Miller, Dara Norman, Knut Olsen, William O'Mullane, Adrian Price-Whelan, Timothy Sacco, J. L. Sokoloski, Ashley Villar, Viviana Acquaviva, Tomas Ahumada, Yusra AlSayyad, Catarina S. Alves, Igor Andreoni, Timo Anguita, Henry J. Best, Federica B. Bianco, Rosaria Bonito, Andrew Bradshaw, Colin J. Burke, Andresa Rodrigues de Campos, Matteo Cantiello, Neven Caplar, Colin Orion Chandler, James Chan, Luiz Nicolaci da Costa, Shany Danieli, James R. A. Davenport, Giulio Fabbian, Joshua Fagin, Alexander Gagliano, Christa Gall, Nicolás Garavito Camargo, Eric Gawiser, Suvi Gezari, Andreja Gomboc, Alma X. Gonzalez-Morales, Matthew J. Graham, Julia Gschwend, Leanne P. Guy, Matthew J. Holman, Henry H. Hsieh, Markus Hundertmark, Dragana Ilić, Emille E. O. Ishida, Tomislav Jurkić, Arun Kannawadi, Alekzander Kosakowski, Andjelka B. Kovačević, Jeremy Kubica, François Lanusse, Ilin Lazar, W. Garrett Levine, Xiaolong Li, Jing Lu, Gerardo Juan Manuel Luna, Ashish A. Mahabal, Alex I. Malz, Yao-Yuan Mao, Ilija Medan, Joachim Moeyens, Mladen Nikolić, Robert Nikutta, Matt O'Dowd, Charlotte Olsen, Sarah Pearson, Ilhuiyolitzin Villicana Pedraza, Mark Popinchalk, Luka C. Popović, Tyler A. Pritchard, Bruno C. Quint, Viktor Radović, Fabio Ragosta, Gabriele Riccio, Alexander H. Riley, Agata Rożek, Paula Sánchez-Sáez, Luis M. Sarro, Clare Saunders, Đorđe V. Savić, Samuel Schmidt, Adam Scott, Raphael Shirley, Hayden R. Smotherman, Steven Stetzler, Kate Storey-Fisher, Rachel A. Street, David E. Trilling, Yiannis Tsapras, Sabina Ustamujic, Sjoert van Velzen, José Antonio Vázquez-Mata, Laura Venuti, Samuel Wyatt, Weixiang Yu, Ann Zabludoff. 2022-08-04. From Data to Software to Science with the Rubin Observatory LSST. https://arxiv.org/abs/2208.02781

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Multi-Scale Contrastive Attention for Light-Curve Representation Learning

Current and next-generation time-domain surveys demand automated techniques capable of analyzing millions of light curves, observed in multiple filters, without relying on exhaustive human annotation or scarce spectroscopic follow-up. We present Astra-CLR, an attention-based, self-supervised contrastive learning framework which enables the representation of raw light curves into a highly discriminative latent space. Pre-trained on $\sim$2.1 million unlabeled Zwicky Transient Facility light curves, the framework utilizes partial light curves as input sequences to generate asymmetric, multi-scale temporal views (explicitly contrasting shorter sequences against longer ones) forcing the network to learn a robust "local-to-global" mapping strategy. Furthermore, we introduce a novel multi-view late fusion architecture that extends the model to efficiently handle longer light curves with larger numbers of observations while accommodating the different cadences associated with each filter. The discriminatory power of the resulting representations was evaluated by using them as input to a Multinomial Logistic Regression classifier, trained to identify 12 broad classes of variability. Final accuracy achieved $\sim 0.70$. When applying a label-efficient, partial top-layer fine-tuning strategy, the topological structure of the latent space is significantly refined, boosting results to $\sim$0.77. Astra-CLR is the first publicly available multi-filter time-series Transformer trained exclusively on real ZTF light curves. Results presented here demonstrate that it provides an ideal foundation for the development of end-to-end pipelines, taking into account color evolution and respecting the inhomogeneous nature of astronomical light curve sampling.

astro-ph.IM↗
arXiv2026

Combining astrometry with pulsar timing: the first joint analysis of very low frequency gravitational waves

The pHz to sub-nHz GW regime remains largely unexplored but is crucial for mapping the early inspiral stage of SMBHBs and probing early-Universe physics. Astrometry and pulsar timing offer orthogonal and deeply complementary secular observables to investigate this frequency band. We aim to present the first joint data analysis combining real astrometric proper motions with binary pulsar timing, to search for and constrain continuous gravitational waves (CWs) sourced by SMBHBs in the ultra-low-frequency regime ($10^{-12} \le f_{GW} \le10^{-9} Hz$). We employ a Bayesian model selection and upper limit estimation framework to combine apparent proper motion displacements of $\sim 1.5 \times 10^6$ quasars from the Gaia CRF3 catalog with the line-of-sight orbital period derivatives ($\dot{P}_b$) of 11 high-precision binary pulsars. To prevent spurious detections, we heavily model instrumental and astrophysical systematics: we propagate the Galactic potential uncertainty for pulsars via Monte Carlo simulations and perform a Vector Spherical Harmonics (VSH) decomposition up to the octupole order (l=3) for quasars. We find no statistically significant evidence for a CWs signal in the joint analysis ($\ln B_{joint} = -0.42 \pm 0.03$). In the absence of detection, we set the tightest constraints to date on CW strain in the pHz band, yielding a 95% upper limit of $h_0\le6.4x10^{-11}$ at a reference frequency of $f_{ref} = 4 \times 10^{-10}$ Hz. The combined dataset achieves full sky coverage and improves single-dataset upper limits by 20%-30%. Combining orthogonal observables successfully breaks spatial degeneracies intrinsic to isolated searches. Furthermore, forecasts from inj.-rec. indicate that with the extended temporal baseline and reduced uncertainties of the upcoming Gaia DR4, this joint framework is poised to break the $h_0 < 10^{-11}$ upper limit barrier for sub-nHz CWs.

astro-ph.IM↗
arXiv2026

Bayesian classification of astronomical spectra with class uncertainties

Context: We developed a probabilistic machine learning method with the aim of performing the O(10)-way classification of low- and high-resolution spectra of stellar and extragalactic targets for the upcoming 4MOST survey. In fulfilment of the survey requirements, this method should be able to express uncertainty in the input data as well as uncertainty introduced in its prediction. Aims: Four different methods are explored: (1) convolutional neural networks (CNNs), (2) the Dirichlet distribution, (3) Monte Carlo dropout (MCD), (4) Bayesian neural Networks (BNNs) + variational inference (VI). Training and validation was performed using labelled spectra from the SDSS database and a custom 4MOST mock dataset. All the methods were compared in terms of the same metrics: accuracy, area under the curve (AUC), expected calibration error (ECE), Shannon entropy, negative log-likelihood (NLL), Brier score, training time, and inference time. Methods: A CNN with simple architecture and about 20,000 parameters was trained to achieve classification accuracies of 91.5% on SDSS data and 92.8% on 4MOST mock data. The direct Dirichlet prediction and VI models tested provide uncertainties on class membership probabilities, but they confuse classes more often. The MCD on a CNN is found to be the most suitable; it boosts the point-estimate accuracies to 92.6% and 93.9%, while still providing fast training and sufficiently fast inference. Compared to a standard CNN, the method additionally provides well-calibrated uncertainties at marginal extra cost.

astro-ph.IM↗