Search arXiv⌕ Search

subject

stat.AP

stat.AP: explore 41 source-linked works published from 2022 to 2026, with original documents and citations.

This collection is a preview while coverage and quality are evaluated.

Search within this collection

Coverage and selection

Includes records with this source-supplied label or an explicit phrase match in their metadata. Matches indicate a mention, not proof that a paper uses a method or tests a material. Source versions are consolidated by DOI.

Sources: arxiv. Collection updated 2026-09-16. Counts describe this index, not the complete source archives.

Generalized Sequential Monte Carlo Sampling for Redistricting Simulation

Simulation methods have become important tools for quantifying partisan and racial bias in redistricting plans. We generalize the Sequential Monte Carlo (SMC) algorithm of McCartan and Imai (2023), one of the commonly used approaches. First, our generalized SMC (gSMC) algorithm can split off regions of arbitrary size, rather than a single district as in the original SMC framework, enabling the sampling of multi-member districts with a varying number of representatives. Second, the gSMC algorithm can operate over various sampling spaces, providing additional computational flexibility. Third, we derive optimal-variance incremental weights and show how to compute them efficiently for each sampling space, leading to more efficient sampling. Finally, we propose a hybrid gSMC-MCMC algorithm by incorporating Markov chain Monte Carlo (MCMC) steps to handle large-scale redistricting applications without changing the target distribution. We demonstrate the effectiveness of the proposed methodology through analyses of the Irish Parliament, which uses multi-member districts of varying sizes, and the Pennsylvania House of Representatives, which has more than 200 single-member districts.

stat.AP↗

On the Capacity of Distinguishable Synthetic Identity Generation under Face Verification

Synthetic face generators can produce many nominal identities, but nominal count does not determine how many are jointly distinguishable under a specified verification rule. We define finite-dimensional capacity as the supremum of codebook sizes over distinct latent identity codes whose induced identity-conditional embedding distributions satisfy per-identity genuine acceptance and pairwise impostor non-match constraints. For deterministic view-invariant pipelines, fixed-code capacity equals the spherical-code cardinality over the realizable embedding set and reduces to the classical spherical-code cardinality when every sphere direction is realizable. For stochastic identity-conditional embedding distributions concentrated with probability at least $1-η$ in spherical caps of angular radius $ρ$, we derive a sufficient center-separation condition, spherical-code capacity lower bounds under full angular expressivity, and positive asymptotic lower-bound exponents for dimension-indexed pipeline families. We also derive prior-constrained random-code lower bounds from pairwise center-separation failure probabilities. When each identity-conditional embedding distribution has support equal to a spherical cap of angular radius $ρ$, we derive necessary zero-error geometric conditions and, for $2ρ<\arccos(τ)$ under full $ρ$-cap angular expressivity, show that the restricted zero-error capacity equals the classical spherical-code cardinality at minimum angle $\arccos(τ)+2ρ$. For finite repeated-view samples, a maximum clique in the resulting compatibility graph identifies the largest sampled subset satisfying all empirical genuine and pairwise impostor constraints. We evaluate this sample-restricted quantity on a deterministically selected DigiFace-1M subset under three fixed recognizers with identity-disjoint in-domain threshold calibration.

cs.IT↗

Spending Scarce Confirmatory PET Measurements: Target-Aligned Validation in A4/LEARN

Anti-amyloid therapies and blood-based biomarkers are changing Alzheimer disease workups into a two-stage measurement workflow: screen broadly with cheaper information, then spend scarce confirmatory amyloid measurements where they support the decision that will be reported. Amyloid positron-emission tomography (PET) remains one such protocol measurement for amyloid burden, but PET slots, trial budgets, and payer-facing evidence packages are finite. This paper asks a deliberately operational question: when is simple transparent PET validation enough, and when is a fitted residual-uncertainty score worth the added complexity? For a weighted protocol target, the first-order value of validating subject i is the product of target influence and residual protocol uncertainty. Generic uncertainty sampling uses only the second factor and can spend PET measurements on subjects that are hard to predict but weak for the scientific, clinical, or commercial claim. We apply this rule to the A4/LEARN PET archive, treating observed PET as a design laboratory for scarce-confirmation studies. For the primary APOE4 carrier versus non-carrier contrast in Centiloid 24-or-higher PET positivity, simple APOE4-balanced validation recovers nearly all of the target-specific gain: at PET budget 200, the confidence-interval width ratio relative to random validation is 0.923 for APOE4 balancing and 0.914 for target-specific scoring, while generic uncertainty sampling is 0.980. Other targets behave differently: target-specific scoring gives larger gains for an age-slope analysis and for cutoff-indexed PET positivity. The practical message is simple: spend scarce protocol measurements according to the claim being validated, not only according to prediction uncertainty.

stat.AP↗

Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modeling

Technical-service LLM agents are entering production workflows, where value depends on whether engineers adopt generated replies. Service tickets hide decision logic, contain noisy single-reference responses, and make reward evaluation costly, making standard post-training brittle. Existing post-training and LLM-as-a-Judge approaches improve grounding or feedback, but do not jointly model latent decision logic, response diversity, and reward cost. We address this gap by coupling latent logic augmentation, robust noise reduction, and hybrid reward modeling. The framework augments supervised fine-tuning data with Planning-Aware Trajectory Modeling and Reasoning Augmentation, builds dual-filtered Multiple Ground Truths, and trains the policy with a hybrid reward that combines a Reranker with an LLM-as-a-Judge. On real Cloud technical-service tasks, the adapted Qwen3-4B achieves the highest Multi-ECS (0.441), lower reward cost, and the highest production adoption rate (46.63%).

cs.LG↗

Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and Tight Coordinatewise Rates

In high-dimensional online prediction, sparse comparators motivate regret bounds that depend on sparsity rather than ambient dimension. Feature priming seeks such adaptation by reweighting features using past data and refitting a minimum-norm predictor. At COLT 2023, Warmuth and Amid posed the open problem of whether the univariate, Pearson, or multivariate priming rules admit competitive online regret guarantees. Under the natural past-only Moore--Penrose protocol, we establish sparse-regret lower bounds that refute the corresponding sparse-logarithmic guarantee. The key obstruction is cheap nuisance interpolation, which permits exact interpolation of the history while assigning insufficient weight to the truly predictive coordinate. An exact target-mass identity and a two-sign argument convert this obstruction into clipped prediction loss. Hadamard constructions yield $Ω(\min\{T,\sqrt d\})$ clipped regret for each of the three unit-power rules against a zero-loss one-sparse comparator. For every fixed power $α\ge1$, one shared paired construction further yields linear regret simultaneously for all three powered rules and selectors among them in sufficiently high dimension. A rank upper bound is tight for powered univariate priming, even with Euclidean-unit inputs, and for unit-power Pearson priming with coordinatewise bounded inputs and target-preserving totalization. A separate algebraic construction gives $Ω(\min\{T,d^{1/4}\})$ regret for unit-power multivariate priming under Euclidean-unit inputs. The univariate lower bound persists under any nonnegative second-stage ridge schedule, while a paired ridge construction yields linear lower bounds for all three powered rules. Exploratory diagnostics on frozen language-model activations are consistent with the same qualitative mechanism. The exact multivariate frontier remains open.

stat.ML↗

Distributed Lag Neural Additive Models

We introduce Distributed Lag Neural Additive Models (DLNAMs), neural-additive analogues of Distributed Lag Non-linear Models (DLNMs) for learning nonlinear effects distributed over lags. DLNAMs replace a prespecified spline cross-basis with neural components that learn exposure--lag response surfaces, avoiding choices of basis family, dimension, and knot placement while preserving additive interpretability and familiar distributed-lag summaries. Exp-centered input layers, smooth activations, and learned subnetwork mixtures produce smooth, locally adaptive representations; pointwise uncertainty combines a conditional last-layer Laplace approximation with between-member ensemble variation. In simulations, DLNAMs generally outperformed DLNM comparators, including penalized and treed variants, in recovering known response functions, with lower bias, stronger boundary recovery, and better-calibrated cumulative intervals; gains were largest for more demanding functions. The architecture performed consistently across sample sizes, outcome families, lag horizons, and jointly fitted multi-exposure settings, retaining recovery performance as exposures were added; fit-specific changes were largely confined to optimization, and applications recovered established empirical patterns.

stat.ML↗

Statistical versus machine learning-based spatial interpolation of post-processed ensemble weather forecasts

Statistical post-processing improves ensemble weather forecasts, but generating calibrated predictions at locations without observations remains challenging. This study compares statistical and machine-learning-based methods for post-processing ECMWF 2-m temperature and 10-m wind speed forecasts at observed and unobserved stations in Germany. We consider EMOS-based approaches, distributional regression networks, Transformers, and graph neural networks under both limited and extended predictor settings. For temperature, we also investigate linear forecast combinations and propose an altitude-aware linear pool (ALP). The results show that post-processing improves upon the raw ensemble in most settings, but no single method performs best across all variables, station groups, and evaluation metrics. The proposed ALP provides a small but significant improvement over the standard linear pool at unobserved locations.

cs.LG↗

Translation of Black-Box Clinical Prediction Models into Standalone Transparent Nomograms: Temporal External Validation in Heart Transplantation

We convert black-box clinical prediction models for tabular data into standalone nomograms that can be audited term by term. PRiSM (Partial Responses in Structured Models) takes the shape of each effect and interaction from the source model, not merely which variables mattered, and lets the outcome select and weight them. We tested this in 50,356 heart transplant recipients, with validation in a later era than training. Nomograms from all 5 source models - a public clinical risk score, logistic regression, neural networks, random forests and extreme gradient boosting - met a prespecified noninferiority criterion for discrimination before any further simplification, and generally preserved calibration and clinical net benefit. Those from the 3 machine-learning models showed no detectable difference in discrimination from de novo generalized additive and explainable boosting models, exceeded neural additive models, and carried fewer terms than the explainable boosting model. PRiSM is released as an open-source Python package.

cs.LG↗

Neural Posterior Estimation for Tomographic Weak Lensing Mass Mapping

Weak gravitational lensing shear and convergence trace the distribution of baryonic and dark matter across space, making them a powerful probe of cosmic structure. Inferring shear and convergence from images is a challenging inverse problem. The prevailing approach to this task estimates shear from weighted averages of galaxy ellipticities, calibrates these estimates to account for systematic biases, and transforms them to reconstruct convergence, a multistage procedure that requires substantial computational resources and meticulous handling of statistical uncertainties. As an alternative, we propose a probabilistic approach to field-level weak lensing inference in which we train a deep neural network to directly map a multiband image to a variational distribution over the underlying tomographic shear and convergence fields. This neural posterior estimation (NPE) procedure implicitly marginalizes over nuisance variables in the cosmological forward model and does not require evaluating the likelihood function. It is also amortized, so it enables rapid posterior inference for astronomical surveys once the neural network is trained. When evaluated on synthetic images from the LSST-DESC DC2 Simulated Sky Survey, NPE produces well-calibrated variational distributions for shear and convergence that are consistent with the ground truth. We describe how maps sampled from these variational distributions could be used in a subsequent simulation-based inference procedure to approximate the posterior distribution over cosmological parameters.

astro-ph.IM↗

When Can LLM Digital Twins Reduce Human Measurement? From Behavioral Fidelity to Statistical Substitutability

LLM-based digital twins promise to reduce repeated human data collection by generating person- specific responses, yet existing evaluations provide little evidence about whether they can reduce human measurement while preserving valid inference. To address this, we introduce statistical substitutability, an inferential criterion that evaluates the extent to which twin predictions can reduce human measurement for a particular estimand while preserving valid inference. We develop a framework, grounded in mixed-subject and prediction-powered inference, that evaluates statistical substitutability along four dimensions: aggregate fidelity, paired respondent-level signal, finite-sample human-label recovery, and stability across populations. Across two empirical evaluations spanning behavioral experiments, multiple models, and alternative respondent representations, we find that digital twins can reproduce average human effects while providing little information about which individuals differ from those averages. Newer models and richer respondent information improve some dimensions of performance but do not reliably translate into human-data savings. Human calibration can reduce aggregate prediction error, yet limited labeled samples often fail to produce stable precision gains. Importantly, these findings demonstrate that behavioral fidelity is neither necessary nor sufficient for statistical substitutability. More broadly, they suggest that AI-generated evidence should be evaluated based on its ability to support valid scientific inference rather than its ability to reproduce human outcomes alone. Digital twins should therefore be judged for confirmatory use by whether they reduce uncertainty about human quantities, not merely by whether they reproduce human means, distributions, or effects.

cs.AI↗

Moment-Matching Probabilistic Data Association for Optimization-Based SLAM

Optimization-based simultaneous localization and mapping (SLAM) makes it possible to reduce accumulated navigation errors of sensing platforms by returning to known areas (loop closure). In this paper, we present an approach to combine probabilistic data association (PDA) with optimization-based SLAM. Instead of associating a single measurement with each landmark, we follow the PDA paradigm from the multiobject tracking community. In particular, in a processing stage performed in addition to the nonlinear least-squares solver of optimization-based SLAM, our method (i) assigns multiple measurements to landmarks probabilistically, (ii) computes the mean and covariance of landmark distributions via moment matching by taking multiple measurement-to-landmark associations into account, and (iii) establishes a virtual landmark measurement and a corresponding linear-Gaussian measurement model that leads to the mean and covariance matrix as moment-matching PDA in (ii). By converting the PDA update step into an equivalent linear-Gaussian measurement update step, PDA can be performed effectively within any optimization-based SLAM method. Our preliminary numerical evaluation in a scenario with false negatives and false positives indicates that incremental smoothing and mapping 2 (iSAM2), combined with the proposed PDA approach, can improve agent localization performance compared to conventional iSAM2.

cs.RO↗

Machine Learning for Pre-Culture ESBL Risk Stratification to Guide Empiric Antibiotic Selection: A 12-Hospital Study of Enterobacteriaceae Cultures

Empiric antibiotic therapy for suspected ESBL-producing Enterobacteriaceae must be selected 48-72 hours before culture results, forcing clinicians to choose between undertreating resistant infections and overusing carbapenems that drive further resistance. We developed a cost-sensitive XGBoost model predicting an ESBL phenotype (resistance to ceftriaxone, ceftazidime, cefepime or piperacillin-tazobactam) at culture ordering using 45 pre-culture EHR features across 132,955 cultures from 72,217 patients at 12 hospitals (14.41% with the ESBL phenotype). Cultures were partitioned at the patient level. At 90% sensitivity, the model achieved 95.8% NPV, reducing post-test ESBL probability to 4.2%, a threshold that may support safe carbapenem-sparing in non-ICU settings, while sparing 307 of every 1,000 cultures an unnecessary broad-spectrum course at the cost of 14 missed ESBL cases per 1,000. SHAP analysis identified prior ESBL colonization as the dominant predictor, ahead of prior organism burden and neighborhood deprivation; removing deprivation features caused minimal performance loss ($Δ\text{AUROC} = -0.020$), enabling equitable bedside deployment. Discrimination was unchanged under a strict IDSA ESBL-E definition (AUROC 0.766), with specimen type added as a predictor (0.764) and without any class-imbalance correction (0.762), and ranged from 0.71 to 0.78 across organism strata.

cs.LG↗

Forecast Skill Is Not Decision Skill: Evidence from Weather-Dependent Decision Tasks

Standard weather forecast evaluations focus on the forecaster's perspective and on a statistical assessment comparing forecasts and observations. In practice, however, forecasts are used to make decisions, so it seems natural to take the decision-maker's perspective and quantify the value of a forecast by its ability to improve decision-making. Decision calibration provides a novel framework for evaluating probabilistic forecast performance at the decision level rather than the forecast level. We evaluate decision calibration to compare a Machine Learning and a classical numerical weather prediction model on various weather-dependent decision tasks, though the framework is applicable to any set of forecast models. We find that model performance at the forecast level does not reliably translate to performance in downstream decision-making: some performance differences only become apparent at the decision level, and even among seemingly similar decision tasks, model rankings can change. Our results confirm that typical forecast evaluations are insufficient for selecting the optimal forecast model for a specific decision task.

cs.LG↗

A Location-Invariant Estimator of Extremal Quantile Treatment Effects for Heavy-Tailed Distributions

Quantile treatment effects (QTEs) measure the effect of a treatment on the distribution of an outcome, and their estimation at extreme quantile levels is of central interest in applications where the target quantiles lie far beyond the range of the data. For heavy-tailed potential outcomes, existing extremal QTE estimators rely on extrapolation combined with a causal extreme value index (EVI) estimator, but the resulting estimator is not invariant under a common location shift of the potential outcome distributions, even though the population QTE is. We address this issue in two steps. First, we adapt the location-invariant Fraga estimator of the EVI to the causal setting using inverse propensity score weighting. Second, we replace the original extrapolation formula with a difference-based scheme, under which the location parameter cancels when quantile differences are taken. The resulting QTE estimator is therefore location invariant. We establish the consistency and asymptotic normality of the proposed extremal QTE estimators, and provide a consistent variance estimator, leading to asymptotically valid inference. A simulation study confirms the location invariance, the stability with respect to the threshold, and the coverage of the proposed methods.

cs.LG↗

SafeRestore: Detector-Relative Risk Certificates for Selective Industrial Image Restoration

Industrial inspection pipelines often restore a measured image before a detector acts on it, yet restoration can suppress detector-supported defect structure or create clean-region activations. We formulate restoration as a selective action problem over the measured display, five restored candidates, and review. SafeRestore ranks candidates with action-specific fitted scores, chooses a gate on threshold-tuning data, and evaluates the fixed gate on a disjoint certification sample with two one-sided exact binomial bounds: one for the positive-conditional evidence-loss incident rate and one for the all-accepted excess-activation incident rate. The guarantee is marginal for one policy fixed before its certification outcomes are observed, under an image-level i.i.d. working model. In a retrospective split-sample study of 4,591 public Carinthia-S images, the protocol yields auditable risk-coverage behavior. The primary all-action policy passes in one of five training repetitions (12.0% +/- 26.9% pass-gated test coverage when failures count as zero), whereas fixed bicubic and reduced-complexity variants pass more often. On reserved morphologies, evidence-loss incidence rises to 81.1-90.3%, and KolektorSDD lacks both detector competence and enough positive certification images for the stated target. The contribution is therefore an auditable, detector-relative framework for deciding when a transformed image may be returned automatically and when review remains necessary -- not a claim that adaptive routing outperforms simpler policies on the present evidence.

cs.CV↗

Big data, differential privacy, and national statistical organisations

Differential privacy (DP) has emerged in the computer science literature as a measure of the impact on an individual's privacy resulting from the publication of a statistical output such as a frequency table. This paper provides an introduction to DP for official statisticians and discuss its relevance, benefits, and challenges from a National Statistical Organisation (NSO) perspective. We motivate our study by examining how privacy is evolving in the era of big data and how this might prompt a shift from traditional statistical disclosure techniques used in official statistics--which are generally applied on a cell-by-cell or table-by-table basis--to formal privacy methods, like DP, which are applied from a perspective encompassing the totality of the outputs generated from a given dataset. We identify an important interplay between DP's holistic privacy risk measure and the difficulty for NSOs in implementing DP, showing that DP's major advantage is also DP's major challenge. This paper provides new work addressing two key DP research areas for NSOs: DP's application to survey data and its incorporation within the Five Safes framework.

stat.AP↗

Dynamic Operational Reserve Margin Assessment from Risk-Constrained Unit Commitment States

We propose Dynamic Reserve Margin (DRM) as a time-varying operational adequacy metric derived from risk-constrained unit commitment (RCUC) states. DRM quantifies reserve adequacy using the additional generation capacity that committed generators can provide within a 5-minute response window relative to uncertainty and contingency reserve requirements. We introduce a complementary Reserve Risk Envelope (RRE) metric to quantify operational reserve headroom in directly interpretable MW terms. A low-margin duration metric is further developed to quantify the persistence of reserve stress over an operating horizon. We use an IEEE 14-bus example to illustrate these concepts, followed by large-scale RCUC case studies under multiple operating scenarios. Results demonstrate that reserve requirements and ramp-accessible reserve capability can vary substantially across operating conditions, and that commitment decisions adapt to maintain reserve adequacy under changing system conditions. The proposed DRM and RRE metrics provide an interpretable operational characterization of reserve adequacy, reveal reserve accessibility and stress persistence that are not captured by conventional reserve margin metrics.

eess.SY↗

Modeling Information Blackouts in Missing Not-At-Random Time Series Data

Traffic forecasting systems rely on fixed sensor networks that frequently exhibit contiguous blackouts. Such outages are usually treated as ignorable missingness, although dropout can depend on unobserved traffic conditions. We study this possibility with an MNAR-aware latent state-space model that combines linear traffic dynamics with a Bernoulli missingness channel whose probability depends on the latent state. Inference uses an Extended Kalman Filter (EKF) followed by Rauch-Tung-Striebel (RTS) smoothing, and parameters are learned by approximate EM. We evaluate Seattle using a leakage-free, month-balanced set of 300 unique all-horizon-aligned blackout windows. On this benchmark, MAR-LDS attains 4.264 mph pooled imputation RMSE and MNAR-LDS improves it to 4.177 (difference -0.086); the detector-cluster bootstrap 95% interval is [-0.182,-0.002]. A causal one-step predicted latent representation raises missingness ROC-AUC from 0.685 using observed-only features to 0.784. We further test whether this compact probabilistic model remains competitive with substantially larger neural time-series architectures under the identical masked-imputation protocol. MNAR-LDS ranks second in pooled RMSE and outperforms 8 of 9 evaluated neural architectures; it is within 1.22% of the best neural result, with no statistically resolved difference under detector-cluster bootstrap, while achieving lower P95 error, lower long-blackout RMSE, and orders of magnitude fewer stored scalar entries. MNAR roughly doubles end-to-end training time relative to MAR and increases EKF+RTS inference time by 41%, making the accuracy-complexity-cost tradeoff explicit. Controlled state-dependent blackouts further show larger gains when dropout is genuinely informative, including a 6.34% reduction in 30-minute forecast RMSE relative to MAR.

stat.ML↗
Compare source metadata on this page
WorkPublishedSource identifierSource
Generalized Sequential Monte Carlo Sampling for Redistricting Simulation2026-09-082603.22188arxiv
On the Capacity of Distinguishable Synthetic Identity Generation under Face Verification2026-09-082604.10641arxiv
Spending Scarce Confirmatory PET Measurements: Target-Aligned Validation in A4/LEARN2026-09-172608.22223arxiv
Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modeling2026-09-072603.18074arxiv
Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and Tight Coordinatewise Rates2026-09-072608.17573arxiv
Distributed Lag Neural Additive Models2026-09-072609.07381arxiv
Statistical versus machine learning-based spatial interpolation of post-processed ensemble weather forecasts2026-09-072609.07512arxiv
Translation of Black-Box Clinical Prediction Models into Standalone Transparent Nomograms: Temporal External Validation in Heart Transplantation2026-09-072609.07610arxiv
Neural Posterior Estimation for Tomographic Weak Lensing Mass Mapping2026-09-072609.07833arxiv
When Can LLM Digital Twins Reduce Human Measurement? From Behavioral Fidelity to Statistical Substitutability2026-09-072609.07987arxiv
Moment-Matching Probabilistic Data Association for Optimization-Based SLAM2026-09-052609.05941arxiv
Machine Learning for Pre-Culture ESBL Risk Stratification to Guide Empiric Antibiotic Selection: A 12-Hospital Study of Enterobacteriaceae Cultures2026-09-052609.05970arxiv
Forecast Skill Is Not Decision Skill: Evidence from Weather-Dependent Decision Tasks2026-09-042512.14779arxiv
A Location-Invariant Estimator of Extremal Quantile Treatment Effects for Heavy-Tailed Distributions2026-09-042609.04018arxiv
SafeRestore: Detector-Relative Risk Certificates for Selective Industrial Image Restoration2026-09-032609.03475arxiv
Big data, differential privacy, and national statistical organisations2026-09-022609.02495arxiv
Dynamic Operational Reserve Margin Assessment from Risk-Constrained Unit Commitment States2026-09-022609.03066arxiv
Modeling Information Blackouts in Missing Not-At-Random Time Series Data2026-09-012601.01480arxiv

These are bibliographic comparisons, not experimental rankings. Follow the original document for methods and conditions.