Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.23131

Information-Geometric Blind Spots of Conformal Test Martingales

Johan Hallberg Szabadváry
View original source ↗Read source document ↗Export metadata ↓

Abstract

Conformal test martingales (CTMs) are e-processes for sequentially testing exchangeability via p-values produced by a conformity function $A$. However, any distribution shift that preserves the push-forward $P\circ A^{-1}$ leaves the p-value distribution invariant, rendering it invisible to every CTM based on $A$. We formalise this blind spot through the \emph{$A$-equivalence class} $[P]_A$ and show that, for any target distribution $T$, there is a unique closest element $Q^\ast\in[P]_A$ in KL divergence---the \emph{$A$-cryptic I-projection}---with explicit density $q^\ast(z) = t(z)\,p_S(A(z))/t_S(A(z))$ and a Pythagorean decomposition into detectable and cryptic components. When $A$ is the Neyman-Pearson likelihood ratio, the cryptic component vanishes, connecting our result to optimal e-value design. We demonstrate the theory on high-dimensional image data by empirically constructing this I-projection. By conditionally resampling EMNIST letters to identically match the conformity scores of MNIST digits, we demonstrate that a distribution can be shifted to a disjoint support while leaving the test martingale mathematically blinded.

Explore related subjects

stat.MLcs.LGmath.STstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Johan Hallberg Szabadváry. 2026-10-16. Information-Geometric Blind Spots of Conformal Test Martingales. https://arxiv.org/abs/2610.23131

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

What is causal about causal models and representations?

Causal Bayesian networks are often described as causal because they support reasoning about interventions. Yet interventions are defined in the mathematical model, whereas the resulting model predictions concern the effects of actions in the real world. Consequently, whether and how a causal model makes falsifiable predictions about the effects of actions depends on how actions are interpreted as interventions. This requires specifying which actions correspond to which interventions, a step that is frequently left implicit. We develop a formal framework that separates a causal model from the data-generating process it aims to describe and formalizes interpretations connecting actions to interventions. Within this framework, we prove an impossibility result: no interpretation can simultaneously satisfy a collection of desirable properties while remaining non-circular. Under a circular interpretation, every model that fits the observational distribution is interventionally valid, so no experiment can falsify it. We analyze several non-circular alternatives and discuss implications for falsification, causal discovery, causal representation learning, and causal abstraction. More broadly, our framework provides a way to make assumptions about the connections between actions and interventions explicit and to reason formally about the empirical content of causal models.

stat.ML↗
arXiv2026

Reinforcement Learning in MDPs with Information-Ordered Policies

Many operational Markov decision processes provide structured information where a trajectory generated under one policy may also reveal how other policies would have performed. We formalize this through an information order over a parameterized policy class, writing $π_{θ'} \preceq π_θ$ when data collected under $π_θ$ can be used to estimate the long-run average cost of $π_{θ'}$. We develop an epoch-based reinforcement learning algorithm for infinite-horizon average-cost MDPs with continuous state and action spaces that exploits this order through counterfactual inference. For a $d$-dimensional policy class whose information order has width $w$, the algorithm achieves $\widetilde O(H\sqrt{wdT/α})$ regret relative to the optimal policy in the class, where $H$ is a bias span upper bound and $α$ denotes an "information ratio". Thus, the width captures the effective exploration complexity of the problem, interpolating between full counterfactual feedback and unstructured policy learning. We instantiate the framework in lost-sales inventory systems with positive lead times, dual-sourcing systems governed by dual-index policies, and queues with state-dependent service rates. These applications yield new regret guarantees without requiring global convexity of the average-cost function or specialized arrival-rate structure. Numerical experiments show that exploiting the information order substantially improves learning performance over existing general-purpose and problem-specific benchmarks.

stat.ML↗
arXiv2026

Measuring Differences between Conditional Distributions using Kernel Embeddings

Comparing conditional distributions is a fundamental challenge in statistics and machine learning, with applications across a wide range of domains. While proposed methods for measuring discrepancies using kernel embeddings of distributions in a reproducing kernel Hilbert space (RKHS) provide powerful non-parametric techniques, the existing literature remains fragmented and lacks a unified theoretical treatment. This paper addresses this gap by establishing a coherent framework for studying kernel-based methods to measure divergence between conditional distributions through what we refer to as conditional maximum mean discrepancy (CMMD). The CMMD consists of a family of metrics which we call levels, with three special cases each using a different type of RKHS embedding: CMMD$_0$ (conditional mean operators), CMMD$_1$ (conditional mean embeddings), and CMMD$_2$ (joint mean embeddings). We additionally introduce a general level $s$ CMMD, clarifying the required assumptions, and establishing mathematical connections between the levels through the lens of operator-based smoothing. In addition to reviewing previously proposed estimators, we introduce a novel doubly robust estimator for the CMMD that maintains consistency provided the product of errors of the underlying models reduces sufficiently fast. We provide numerical experiments demonstrating that the CMMD effectively captures complex conditional dependencies for statistical testing.

stat.ML↗