Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2611.03331

Leveraging Continuous Trade Data for Intraday Imbalance Price Forecasting

Klimis Stylpnopoulos·Jethro Browell·Janine Illian·Ciaran Gilbert·Wayne Jones
View original source ↗Read source document ↗Export metadata ↓

Abstract

Forecasts of imbalance prices are essential for participants in short-term electricity markets. This study proposes an approach to integrate trade data from continuous intraday markets into forecasting methods and considers the production of both parametric and non-parametric density forecasts. Extensive feature engineering is required to convert irregular trade data into a suitable tabular structure. Our findings demonstrate that intraday trade data can enhance state-of-the-art forecasts based on quantile regression and LASSO. We also propose a model based on Natural Gradient Boosting to predict the means and variances of a bimodal Gaussian mixture, which offers both improved predictive performance and computational efficiency. In a case study based on multiple years of data from Great Britain, our proposed approach reduced the Pinball Loss of state-of-the-art models by 20% for forecasts made 30 seconds before gate closure, and by 7.5% for forecasts made one hour before delivery.

Explore related subjects

stat.AP

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Klimis Stylpnopoulos, Jethro Browell, Janine Illian, Ciaran Gilbert, Wayne Jones. 2026-11-02. Leveraging Continuous Trade Data for Intraday Imbalance Price Forecasting. https://arxiv.org/abs/2611.03331

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

On the non-uniformity of the 2026 FIFA World Cup draw

The group stage of a sports tournament is often made more appealing by introducing additional constraints in the group draw that promote an attractive and balanced group composition. For example, the number of intra-regional group matches is minimised in several world championships. Under such constraints, the traditional draw procedure may become non-uniform, meaning that the feasible allocations of the teams into groups are not equally likely to occur. Our paper quantifies this non-uniformity of the 2026 FIFA World Cup draw for the official draw procedure, as well as for 47 reasonable alternatives implied by all permutations of the four pots and two group labelling policies. We show why simulating with a recursive backtracking algorithm is intractable, and propose a workable implementation using integer programming. The official draw mechanism is found to be optimal based on two of five non-uniformity measures, including the average distortion of the 771 non-zero assignment probabilities. Nonetheless, non-uniformity can be more than halved if the organiser aims to treat the best teams drawn from the first pot equally.

stat.AP↗
arXiv2026

State Space Modeling of the Ornstein-Uhlenbeck Process with Measurement Error: An Application to Microbiome Data

Understanding the temporal dynamics of the human microbiome is critical for deciphering host-microbe interactions and identifying responses to environmental or physiological disruptions. In this study, we develop a likelihood-based framework grounded in Ornstein-Uhlenbeck (OU) state-space modeling to analyze longitudinal microbiome data. By applying log-ratio transformations to genus-level relative abundances, we accommodate the compositional nature of microbiome profiles and enable biologically interpretable inference. We estimate the mean reversion rate of the OU process using maximum likelihood estimation and construct profile likelihood confidence intervals to assess temporal stability. Biologically, higher estimated mean-reversion rates correspond to stronger regulatory feedback and thus greater temporal stability of the gut microbiome, whereas lower rates reflect weaker stabilization and slower recovery from perturbations. In the datasets analyzed here, most estimated mean reversion rates were between 0.12 and 0.36, corresponding to a half-life in the range 2-6 days for the system to return to the equilibrium level. These mean-reversion rates are substantially lower than previous estimates obtained from OU models that did not account for measurement error. To capture structural shifts in microbial dynamics, we propose a statistical testing procedure that compares a single-parameter model to a split-parameter model across disrupted time periods. We also propose an OUSSM to jointly model multiple microbes, taking into account the special measurement error covariance structure for log-ratio transformed data.

stat.AP↗
arXiv2026

Integrating Machine Learning and Uncertainty Quantification for Real-Time Decision-Making in Digital Twins: A System-Centric Review and Perspective

Digital Twins are evolving from passive virtual representations into closed-loop systems that continuously synchronize with physical assets and drive real-time decisions. Yet predictive modeling, machine learning (ML), uncertainty quantification (UQ), and decision-making have been studied as separate capabilities, leaving limited understanding of their integration and maintenance during online operation. This narrative review, scoped by the functional requirements of real-time decision-making rather than by Digital Twin terminology alone, examines how such decisions are generated and sustained in engineering Digital Twins. We first examine foundational decision-making architectures and representative engineering applications, followed by methods that address their limitations across the decision loop. We identify optimization-based, policy-based, and knowledge-based approaches as the major pathways from virtual representation to action. Across them, ML plays three roles in decision-making: enabling efficient surrogate modeling, transforming heterogeneous physical feedback into decision-relevant information, and accelerating or directly generating decisions. UQ supports uncertainty-aware decisions and model adaptation, while continual validation, drift detection, and adaptation help maintain reliable operation as the physical system evolves. Together, these findings underscore the need for a shift from a model-centric to a system-centric perspective on Digital Twins, where its performance depends on the interactions with sensing, state estimation, computation, decision-making, and adaptation rather than on their standalone performance. Therefore, future research should shift from isolated algorithm development toward system-level co-design, scalable decision-making, and approaches to design, and data-efficient continual adaptation and validation for AI-enabled autonomous systems.

stat.AP↗