arXiv subjects

Christopher David Roberts

Publications and source records attributed to Christopher David Roberts.

4 recordsLinked to original sources

arXiv2026

Estimating ensemble-mean anomaly correlations and the ratio of predictable components using an unbiased estimate of the predictable signal variance

The correlation ($r$) between ensemble mean forecasts and observations is a frequently-used metric of forecast quality in the weather and climate prediction communities. However, uncorrected estimates of $r$ are biased towards zero at finite ensemble size due to the influence of unpredictable noise that contaminates the sample ensemble mean. This note summarises established ensemble-size adjustment methods and characterises the behaviour of $r_{\infty}$, a correlation estimator that is constructed from an unbiased and ensemble-size-independent estimate of the signal variance. The only requirement is that forecast members are exchangeable with one another such that they can be interpreted as independent draws from a case-specific predictive distribution. The estimator $r_{\infty}$ does not require forecasts and observations to be statistically exchangeable, only that the unpredictable component of each member is uncorrelated with the observed truth, and thus works for both reliable and unreliable forecasts. In some situations, $r_{\infty}$ may be undefined (i.e. the estimated signal variance is negative) or fall outside the admissible range $-1 \leq r_{\infty} \leq 1$ when the predictable signal is small relative to the sampling uncertainty. These inadmissible values can be interpreted as evidence that the true predictable signal cannot be resolved with the available ensemble size and number of cases. The same unbiased signal variance can also be used to estimate the correlation between the predictable signal and individual members to yield an ensemble-size-invariant estimator of the ratio of predictable components (RPC). In contrast, variance-based RPC estimates, including forms that correct only the model predictable component, are biased low by a factor that depends only on the ensemble size and model signal-to-noise ratio.

physics.ao-ph↗

arXiv2026

Ensemble-size-dependence of deep-learning post-processing methods that minimize an (un)fair score: motivating examples and a proof-of-concept solution

Fair scores reward ensemble forecast members that behave like samples from the same distribution as the verifying observations. They are therefore an attractive choice as loss functions to train data-driven ensemble forecasts or post-processing methods when large training ensembles are either unavailable or computationally prohibitive. The adjusted continuous ranked probability score (aCRPS) is a fair and ensemble-size-unbiased scoring rule when applied to ensembles with forecast members that are interpretable as conditionally independent draws from an underlying predictive distribution. However, distribution-aware post-processing methods that introduce structural dependency between members can violate this assumption, such that aCRPS no longer functions as a fair scoring rule. We demonstrate this effect using two approaches designed to minimize the expected aCRPS of a finite ensemble: (1) a linear member-by-member calibration, which couples members through a common dependency on the sample ensemble mean, and (2) a deep-learning method, which couples members via transformer self-attention across the ensemble dimension. In both cases, the results are sensitive to ensemble size and apparent gains in aCRPS can correspond to systematic unreliability characterized by over-dispersion. We introduce trajectory transformers as a proof-of-concept that ensemble-size independence can be achieved. This approach is an adaptation of the Post-processing Ensembles with Transformers (PoET) framework and applies self-attention over lead time while preserving the conditional independence required by aCRPS. When applied to weekly mean $T_{2m}$ forecasts from the ECMWF subseasonal forecasting system, this approach successfully reduces systematic model biases whilst also improving or maintaining forecast reliability regardless of the ensemble size used in training or real-time forecasts.

physics.ao-ph↗

arXiv2026

The role of the oceans for subseasonal prediction: insights from eddy-permitting and eddy-rich coupled forecast systems

The oceans play a fundamental role in Earth's climate system, redistributing heat and influencing global and regional climate variability and predictability across weather and climate timescales. The benefits of ocean-atmosphere coupling for initialised predictions depend on the balance between improvements associated with more realistic air-sea interactions and dynamics, and degradations arising from the development of systematic biases at the coupling interface. Here, we draw on recent developments in modelling and data assimilation at ECMWF to revisit the role of ocean-atmosphere coupling in subseasonal predictions. In particular, we evaluate the impact of ocean-atmosphere coupling in 46-day reforecasts produced with the ECMWF Integrated Forecasting System (IFS) and explore the potential for improvements through increased horizontal resolution and a better representation of the ocean mesoscale. We find that ocean-atmosphere coupling significantly enhances ensemble forecast skill in the tropics, with positive effects increasing at longer lead times. In particular, Madden-Julian Oscillation (MJO) forecasts are substantially improved, with forecast skill extended by approximately 5 days compared to the uncoupled configuration. In contrast, ocean-atmosphere coupling has a more limited impact on the extratropical atmosphere at subseasonal timescales, with marginal impacts on the predictability of major tropospheric and stratospheric circulation indices. Finally, we present selected results from an experimental eddy-rich coupled configuration of the IFS, with a horizontal ocean resolution of approximately 8 km. We find that a better-resolved representation of the ocean mesoscale has a limited impact on atmospheric forecasts at subseasonal lead times, which suggests that many of the known deficiencies of the eddy-permitting reference configuration are mitigated by accurate initialisation.

physics.ao-ph↗

arXiv2026

Ensemble reliability and the signal-to-noise paradox in ECMWF subseasonal forecasts

Ensemble forecasts can exhibit counterintuitive statistical properties such that the correlation between ensemble means and observations ($r_{mo}$) exceeds the correlation between ensemble means and individual members ($r_{mm}$). This behaviour has been interpreted as a `signal-to-noise paradox' (SNP), which is commonly diagnosed using the ratio of predictable components ($\textnormal{RPC} = \sqrt { r_{mo}^2 / r_{mm}^2 } $). Here, we emphasise the links between ensemble-size-invariant estimates of RPC and other metrics of ensemble reliability and derive a general closed-form expression for RPC in terms of $r_{mo}$, the spread-error ratio (SER), and total variance ratio (VR). Physical constraints on the admissible solutions provide a mechanism to identify statistically paradoxical sample estimates of RPC, $r_{mo}$, SER, and VR that correspond to combinations that are not possible without sampling uncertainty. We evaluate three atmospheric circulation indices in ECMWF subseasonal reforecasts. Large-ensemble NAO forecasts evaluated over 80 start dates satisfy reliability criteria within our estimated sampling uncertainties but exhibit high RPC values at some lead times. These lead times coincide with paradoxical combinations of correlation and reliability metrics that are impossible in the large-sample limit, indicating an important role for sampling uncertainties. Nevertheless, wintertime NAO indices averaged over days 16-45 exhibit more robust evidence for unreliability characterised by RPC$approx1.5$ suggesting that SNP-like behaviour observed in daily data during the period 2001-2020 is not solely attributable to sampling artefacts. However, these results do not generalise to other configurations of the same IFS model evaluated over 3120 start dates for the period 1959-2023. In these extended reforecasts, daily NAO indices are well-calibrated and RPC$\approx1$ for all lead times.

physics.ao-ph↗