arXiv subjects

Wael Mohammed

Publications and source records attributed to Wael Mohammed.

3 recordsLinked to original sources

arXiv2026

Global tree forecasters collapse at the hierarchical aggregate: a five-panel failure characterization

Global forecasting models pool many series and learn one shared function. Gradient-boosted trees are their most common form. We measure a failure of this design that has not, to our knowledge, been documented. Train a global tree on the individual series of a hierarchy, then ask it for the hierarchical aggregate. The aggregate sits far outside the model's training range, and the forecast collapses. The model under-predicts the total by 30-50x in our production deployment, and by up to 496x in a public M5 reconstruction. The mechanism is known: beyond its training range, a tree predicts a constant. It surfaces at the aggregate because the total dwarfs every training series. The cure is not new. Per-series scaling, the preprocessing step that Montero-Manso and Hyndman (2021) recommend, prevents the collapse. So do a weighted aggregate-level training row and seasonal differencing. Our contribution is the characterization. The collapse reproduces on five panels: a production business-to-business marketplace, a synthetic hierarchy, M5, Australian Tourism, and a public business-buyer panel. It holds on three tree libraries, is invariant across training seeds, and is statistically significant. Its onset is immediate and tracks a simple support bound: a scale gap of only 1.15x already costs a third of the total. No standard configuration change prevents it: pooling every hierarchy level into training fails at scale, and the one knob that fits linear models in the leaves softens it without curing it. Rolling the forecasts forward recursively separates the cures: the aggregate-row cure re-collapses, per-series scaling degrades but stays low, and only seasonal differencing keeps its one-step accuracy unchanged. We close with a three-step procedure for diagnosing and preventing the failure in deployed systems.

cs.LG↗

arXiv2026

Evaluation Choices Decide the Forecasting Leaderboard: Evidence from a Production Marketplace Panel

A forecasting benchmark reports which method won. We show that the answer is set by the evaluator's choices before any model is fitted. We benchmark 24 forecasting methods and one textbook reference, including six 2025-era time series foundation models, on a production marketplace panel of 1,887 business customers over 67 months. We hold the data, the horizon and the period fixed, and vary only the evaluation design. Three choices each reverse or dissolve a headline conclusion. Changing the unit of analysis from the market total to the individual customer moves our production baseline from second of nineteen, beaten by nothing, to twenty-third of twenty-five. Nineteen of its twenty-four challengers beat it there. Changing how much error is pooled decides whether a Diebold-Mariano test finds anything at all. Scoring prediction intervals rather than point forecasts reorders the field almost completely, with a rank correlation of 0.02 on intermittent demand. We then measure what the deployed system gets from this. Its selection rule captures 55% of the distance between doing nothing and choosing with hindsight. The reversal is not a quirk of our data. We ran the released protocol, unchanged, on the public M5 retail panel. The same baseline shape places first at the market total and last per series, beaten by everything, and a replayed selection rule closes 64.7% of the same floor-to-ceiling distance there. Adding five zero-shot foundation models to that roster changes who wins at the total, not the shape. The bands' blind spot travels too: conformal bands under-cover most on the spikiest items. Splitting our own panel into ever smaller groups turns the contrast into a curve: the baseline's rank worsens at every level of disaggregation. We release the evaluation protocol and report an error of our own that inverted a result before we caught it.

cs.LG↗

arXiv2026

Seasonal false alarms in customer churn and decline early-warning systems: adjacent-window labels confound seasonality with decline, and a year-over-year correction

Customer decline early-warning systems feed account-manager action lists, and every flagged account consumes intervention capacity. In a deployed business-to-business marketplace system, one action-list slot in three went to flags that dissolve under a seasonally aligned label. The standard target in non-contractual churn prediction compares an entity's next k months of activity with its trailing k months. The two windows cover different calendar months, so for seasonal entities the threshold-ratio construction confounds seasonality with decline, and the event rate depends on the label's anchor calendar month. We formalize the mechanism and measure it on three public panels and the production panel (the public arms pre-specified). Of the adjacent-window decay events, 37--69% on the public panels and 28--50% in production have no counterpart under a seasonally aligned definition. Pooling anchors, the standard remedy, balances the training mixture but corrects no individual label. Measured alternatives repair the curve only partially or change the detection horizon. Aligning the baseline to the same k calendar months one year prior -- advice practitioners already state, here formalized, measured, and costed -- flattens the curve at the source. With the classifier held fixed, production holdout ROC-AUC rises from 0.767 to 0.864 for decline (different targets; the gap closes on synthetic ground truth and a production hindsight referee). The served action list shrinks by a third, 119 to 79 accounts. The measured price is extra history, a blind spot to decline-then-stabilization, and a stricter cut under sustained growth.

stat.AP↗