Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.34521

Population-Level Decision Curve Analysis May Mislead the Evaluation of Prediction Model Usefulness under Subgroup Utility Heterogeneity

Junfeng Wang·Kim Zhipei Wang·Ben Van Calster·Nan van Geloven·Ewout Steyerberg·Laure Wynants
View original source ↗Read source document ↗Export metadata ↓

Abstract

Background Decision curve analysis (DCA) evaluates prediction model usefulness using net benefit (NB). Population-level NB is often interpreted as a proxy for population-level expected utility, but this assumes comparability of utility across subgroups. We aimed to characterize when subgroup utility heterogeneity may invalidate population-level DCA conclusions. Methods We compared prediction-driven treatment with default strategies in populations containing subgroups with different utility values. We derived an inconsistency region: combinations of subgroup-specific ΔNB values for which population-level NB and utility favor different strategies. We also developed a practical robustness framework. Results Opposite signs of subgroup-specific ΔNB provide a warning signal for possible inconsistency. The inconsistency region is larger when subgroup sizes are more similar and subgroup-specific \(a-c\) values are more different, where \(a-c\) is the incremental utility of a true positive relative to a false negative. When \(a-c\) differs across subgroups, population-level NB combines quantities on different implicit utility scales and may conflict with population utility. A real-world case study illustrates the problem. Conclusions Using population-level NB as a proxy for population utility implicitly assumes homogeneous \(a-c\) across subgroups. Subgroup DCA and our framework can identify and assess when utility heterogeneity may invalidate population-level conclusions.

Explore related subjects

stat.ME

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Junfeng Wang, Kim Zhipei Wang, Ben Van Calster, Nan van Geloven, Ewout Steyerberg, Laure Wynants. 2026-09-28. Population-Level Decision Curve Analysis May Mislead the Evaluation of Prediction Model Usefulness under Subgroup Utility Heterogeneity. https://arxiv.org/abs/2609.34521

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Univariate-Guided Interaction Modeling

We propose a procedure for sparse regression with pairwise interactions, by generalizing the Univariate Guided Sparse Regression (UniLasso) methodology. A central contribution is our introduction of TripletScan, which screens a pair $(j,k)$ using the coefficient of $X_jX_k$ in the local regression of the response on $1$, $X_j$, $X_k$, and $X_jX_k$. The retained products are incorporated either jointly with the main effects through UniLasso, yielding uniPairs, or after a first-stage main-effects fit, yielding uniPairs-2stage. For the UniLasso components of the procedures, we prove false-positive exclusion and uniform coefficient-error bounds. In simulations and an HIV drug-resistance application, the proposed procedures produce substantially smaller fitted models than competing interaction methods while retaining competitive predictive performance.

stat.ME↗
arXiv2026

Factoring A-Optimality into D-Optimality and Sphericity

The D criterion measures the volume of the joint confidence ellipsoid for the linear model coefficients and ignores its shape, so designs with the same D value can estimate individual effects with different variances. Meanwhile, A-optimality minimizes average coefficient variance. With both criteria expressed as information values, A equals D multiplied by a sphericity index for the same ellipsoid. In a fixed coefficient basis, sphericity further factors into coefficient-variance balance and a determinant-based correlation component. In five published screening comparisons, the A-optimal design has a larger correlation component despite slightly poorer variance balance; in three it also has a smaller D value. In a seven-run family of designs that all tie under D, the two with equal coefficient variances have the lowest A value. Both sphericity components can be calculated directly from standard errors and estimate correlations available in statistical software. After whitening by a prediction moment matrix, the same determinant-sphericity factorization applies to the I-criterion.

stat.ME↗
arXiv2026

Testing the equality of parameters in fixed and increasing dimension

This paper proposes a general and unified framework for testing the equality of a broad class of parameters, defined as a smooth function of expectations of symmetric kernels, across multiple independent populations. We consider two test statistics, a Wald-type statistic and an ANOVA-type statistic. The asymptotic distribution of the first one is derived under a fixed-dimension regime, whereas the second one is studied under both fixed and increasing-dimension regimes, where the parameter dimension diverges with the sample size. Based on these limiting distributions, we construct test procedures enabling asymptotically exact inference without parametric assumptions. Additionally, an alternative null distribution estimator based on a weighted bootstrap approximation is studied, which is applicable to the ANOVA-type statistic under a fixed-dimension regime. The finite-sample performance and computational efficiency of the proposed procedures are evaluated through an extensive simulation study and a real dataset application.

stat.ME↗