Tabular foundation models are not a shortcut to calibrated clinical risk prediction: a pre-registered simulation against the true risk

Ebrahim Khaled Ebrahim

Home/Archive/arXiv

arXiv · 2610.39065

Tabular foundation models are not a shortcut to calibrated clinical risk prediction: a pre-registered simulation against the true risk

Ebrahim Khaled Ebrahim

View original source ↗Read source document ↗Export metadata ↓

Abstract

Tabular foundation models are pretrained networks that predict risk from a table without fitting a model. Whether their risks are correct (calibrated) has been judged on real data, where true risk is unknown. In a pre-registered simulation with known true risks, we compared TabPFN-3.5, TabICLv2 and Mitra-v2 with Firth logistic regression with intercept correction (FLIC) in 58 scenarios and 1,000 random designs, then on four clinical data sets. Foundation models were not a shortcut to calibration. Their calibration slope matched FLIC's within the pre-registered margin in 36 or 37 of 48 core scenarios and was better only at n = 200 with a 3% event rate. Even 3,000 patients were often not enough: with FLIC, TabPFN-3.5 or Mitra-v2, 63%-83% of samples gave risks close to the truth in average, intercept and slope at event rates of 10% and 30%, and 35%-50% at 3%. TabICLv2 did worse (8%-39% and 1%-3%). Across the random designs, foundation models had a lower true calibration error than FLIC beyond the margin in 25%-38% and a higher one in none, though small losses were predicted at n = 3000. On real data, no model resisted a change in event rate between populations; recalibration on 500 local patients largely repaired it.

Explore related subjects

stat.ME cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Ebrahim Khaled Ebrahim. 2026-10-28. Tabular foundation models are not a shortcut to calibrated clinical risk prediction: a pre-registered simulation against the true risk. https://arxiv.org/abs/2610.39065

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

On the closed-loop Volterra method for analyzing and forecasting time series

This paper studies the closed-loop Volterra method for analyzing and predicting stationary time series. The method represents the dependence of a current observation on past observations through polynomial terms, including powers of individual lags and interactions between different lags. Since the number of coefficients increases rapidly with the memory length and polynomial degree, we use the theory of reproducing kernel Hilbert spaces to estimate these coefficients by regularized least squares through kernel evaluations. We establish conditions on marginal moments, temporal dependence, model complexity and regularization under which the prediction risk converges to the minimum mean squared prediction error as the sample size increases. We also consider data-driven selection of tuning parameters by a rolling-origin procedure based on one-step-ahead prediction errors, and show that the resulting selector asymptotically attains the prediction risk of the best candidate in the considered grid. Simulation studies and a real-data application examine the finite-sample performance of the method and its comparison with linear and nonlinear prediction methods.

stat.ME↗

arXiv2026

Boosting e-BH via conditional calibration

The e-BH procedure is an e-value-based multiple testing procedure that provably controls the false discovery rate (FDR) under any dependence structure between the e-values. Despite this appealing theoretical FDR control guarantee, the e-BH procedure often suffers from low power in practice. In this paper, we propose a general framework that boosts the power of e-BH without sacrificing its FDR control under arbitrary dependence. This is achieved by the technique of conditional calibration, where we take as input the e-values and calibrate them to be a set of "boosted e-values" that are guaranteed to be no less -- and are often more -- powerful than the original ones. Our general framework is explicitly instantiated in three classes of multiple testing problems: (1) testing under parametric models, (2) conditional independence testing under the model-X setting, and (3) model-free conformalized selection. Extensive numerical experiments show that our proposed method significantly improves the power of e-BH while continuing to control the FDR. We also demonstrate the effectiveness of our method through an application to an observational study dataset for identifying individuals whose counterfactuals satisfy certain properties.

stat.ME↗

arXiv2026

Differentially Private Covariate Balancing Causal Inference

Differential privacy is the leading mathematical framework for privacy protection, providing a probabilistic guarantee that safeguards individuals' private information when publishing statistics from a dataset. This guarantee is achieved by applying a randomized algorithm to the original data, which introduces unique challenges in data analysis by distorting inherent patterns. In particular, causal inference using observational data while preserving privacy is challenging because it requires a good covariate balance between treatment groups, but checking the covariate balance is challenging when privacy is a primary concern because releasing any statistic (e.g., t-statistic) for the balance check compromises privacy. Additionally, the performance of the privatized estimator critically depends on the choice of differential privacy mechanisms, and it remains unexplored how privacy-protecting causal inference could be conducted with observational data. In this article, we present a differentially private two-stage covariate balancing weighting estimator to infer causal effects from observational data. Our algorithm produces both point and interval estimators with statistical guarantees, such as covariate balance, rate optimality, and oracle budget allocation, under a given privacy budget.

stat.ME↗