Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.22930

Conformal Policy Selection: When to Deploy AI Decisions for Targeted Intervention

Junyu Cao·Ruijiang Gao·Maytal Saar-Tsechansky·Mingzhang Yin
View original source ↗Read source document ↗Export metadata ↓

Abstract

Artificial-intelligence (AI) policies are increasingly used to make consequential decisions in medicine, finance, education, and public services, yet deploying them responsibly requires knowing not only whether a policy helps on average, but for whom. Existing tools for off-policy evaluation and safe policy learning provide population-level guarantees, but some individuals may experience adverse treatment effects. We recast targeted deployment as a multiple-testing problem: for each candidate unit, test whether the outcome under the AI policy would exceed a user-specified threshold, and control the false discovery rate (FDR) among the selected set. Our method, Conformal Policy Selection, combines weighted conformal $p$-values that account for the distributional shift from behavior to target policy with a two-step selection-and-pruning procedure. We prove finite-sample FDR control and establish a doubly robust asymptotic guarantee, where FDR remains controlled if either the outcome model or the propensity model is correctly specified. Synthetic experiments verify these guarantees across four misspecification regimes, and a semi-synthetic study built on real electronic-health-record covariates illustrates how the same FDR control can act as a deployment safety gate, bounding the harmful treatments that a globally trained policy would otherwise administer to a high-risk subpopulation where it misgeneralizes. The framework offers a principled method for safe AI deployment in high-stakes applications.

Explore related subjects

stat.MEcs.LGstat.ML

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Junyu Cao, Ruijiang Gao, Maytal Saar-Tsechansky, Mingzhang Yin. 2026-10-16. Conformal Policy Selection: When to Deploy AI Decisions for Targeted Intervention. https://arxiv.org/abs/2610.22930

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Closed-form Covariance Functions in Space and Time from B-spline Spectral Densities

The autocovariance function (ACF) is central to the analysis of time series, spatial, and spatio-temporal data. Classical parametric models are often too restrictive for multivariate and space-time data, which motivates non-parametric descriptions of the ACF. Bochner's Theorem guarantees that any positive spectral measure yields a valid ACF via the inverse Fourier transform; however, existing non-parametric approaches in the spectral domain rarely return closed-form expressions for the ACF itself. We develop a closed-form class of non-parametric ACFs by deriving the inverse Fourier transform of B-spline spectral bases with arbitrary degree and knot placement. This yields a family of closed-form stationary covariance models with three properties: (i) it is uniformly dense for the ACF, among the ACFs of weakly stationary processes that possess a spectral density; (ii) it accommodates univariate, multivariate, and multidimensional processes; and (iii) it is non-separable in general, with separability a special case. Jackson-type approximation bounds establish convergence rates, and simulation studies show accurate recovery of known univariate and bivariate ACFs. Applied to output from a high-resolution ocean simulation, the fit space-time ACF is non-separable and demonstrates accurate process recovery.

stat.ME↗
arXiv2026

2D Stability Selection: Design Jittering to Exclude Correlated Irrelevant Features

Stability Selection resamples the observations but leaves the design untouched, so it cannot reject an irrelevant feature that is correlated with the relevant ones in a way that violates the irrepresentable condition. Such a feature (a ``confounder'') is selected in a substantial fraction of resamples, and the exchangeability behind the false-positive bound of Stability Selection fails. We propose 2D Stability Selection (2D-SS), which aggregates selection frequencies over half-samples and over a grid of injected design noise, and reduces to Stability Selection at zero noise. Because input noise acts, in expectation, as Tikhonov regularisation, a jittered Lasso fit is a noisy elastic net whose support recovery is governed by a noise-indexed irrepresentable condition. Jittering can therefore repair designs in which confounders violate the Lasso condition, although it can also hurt. We give the noise level at which a violated feature's statistic vanishes (for uncorrelated relevant features), exact-recovery guarantees for a single fit, conditions under which aggregation over noise levels recovers the support, and a complementary-pairs bound on the expected number of false positives, carried over from Stability Selection to the noise-aggregated procedure, for a fixed threshold or a size-matched selector. The procedure takes no measurement-error covariance as input, although the guarantees assume i.i.d.\ Gaussian measurement error. In simulations 2D-SS excludes confounders that Stability Selection includes, and its size-matched version stays below the nominal closed-form false-positive bound in all 240 simulated cells, whereas Stability Selection exceeds it in 21; it does not generally improve on Stability Selection in $F_1$ and is lower at low noise. It is useful when stable but wrong selections are the concern.

stat.ME↗
arXiv2026

CVaR anchor regression protects against rare shifts

We study prediction in new environments when the training data contains rare, large distribution shifts. Anchor regression penalizes the average of the squared mean residual across environments. It protects against shifts in an ellipsoid determined by the second moment of the training shifts. Covering rare shifts may therefore require a large penalty $γ\gg 1$, expanding the ellipsoid in every direction and reducing accuracy on common environments. We propose CVaR anchor regression, which replaces the average of the squared mean residuals with a tail average. We prove an exact worst-case risk guarantee under a linear structural model that allows for heteroscedastic noise. CVaR anchor regression protects against shifts in a union of ellipsoids determined by second moments of the shifts in the worst $α$-fraction of environments. For $α$ small enough, this is the scaled convex hull of the training shifts and their negatives. It therefore covers all training shifts when $γ\geq 1$. Unlike CVaR or GroupDRO applied directly to prediction risks, CVaR anchor regression does not give environments more weight solely because their noise levels are high. Examples show how the method can improve protection against rare shifts while retaining accuracy on common environments. We illustrate the method on the New York City taxi dataset.

stat.ME↗