arXiv subjects

Sophia Sklaviadis

Publications and source records attributed to Sophia Sklaviadis.

4 recordsLinked to original sources

arXiv2026

The Fenchel-Young Generalization of Bayesian Principles

Conjugate Bayes plays a key role in machine learning, ensuring computationally simple posterior updates. We generalize the variational principle behind this computation by using the Fenchel-Young (FY) inequality, which lets us define a generalized evidence lower bound beyond the KL geometry, and at equality, characterizes new generalized posteriors. A generalized notion of conjugacy follows, defined as closure of prior-potential families that are affine under translation by the loss. We show that such families not only preserve additive canonical updating, but also posterior inference with parameter-dependent supports and tuneably heavy tails. This combination is not feasible with standard conjugate Bayes. We derive a new FY generalization of expectation maximization (EM), which recovers classical and Hard EM, and amortized versions. We show examples, including generalized Bayesian linear regression with closed-form bounded support and heavy-tailed posteriors, and a Gaussian mixture with sparse cluster assignments.

cs.LG↗

arXiv2026

The Bayesian Learning Rule Beyond KL Geometry

The Bayesian learning rule (BLR) exploits the relation between Bayesian updating and the Kullback-Leibler divergence (KLD) to derive efficient additive updates for exponential family variational posteriors. The KL geometry is powerful but restrictive: combined with posterior families with parameter-dependent support it can lead to undefined objectives. We ask which non-KL geometries preserve the additive BLR computation. Previous works that use $f$-divergences suggest that this is only possible with the KLD. Here, we show that this is not true. We show that additive information-geometric optimization algorithms can be derived for the class of mixed-space Bregman or Fenchel-Young discrepancies. These discrepancies induce generalized maximum-entropy families, dual coordinates, and generalized-BLR canonical gradient updates, recovering the classical BLR. The generalized BLR leads to new algorithms that \textit{cannot} be derived from the KL-based BLR. For the Tsallis negentropy with quadratic statistics, the generalized BLR yields bounded-support and heavy-tailed $q$-Gaussian posteriors and an easy Newton-like implementation. Experiments up to CIFAR-100 illustrate controllable support and tail behavior and competitive performance.

cs.LG↗

arXiv2026

A Stein Identity for q-Gaussians with Bounded Support

Stein's identity is a fundamental tool in machine learning with applications in generative models, stochastic optimization, and other problems involving gradients of expectations under Gaussian distributions. Less attention has been paid to problems with non-Gaussian expectations. Here, we consider the class of bounded-support $q$-Gaussians and derive a new Stein identity leading to gradient estimators which have nearly identical forms to the Gaussian ones, and which are similarly easy to implement. We do this by extending the previous results of Landsman, Vanduffel, and Yao (2013) to prove new Bonnet- and Price-type theorems for q-Gaussians. We also simplify their forms by using escort distributions. Our experiments show that bounded-support distributions can reduce the variance of gradient estimators, which can potentially be useful for Bayesian deep learning and sharpness-aware minimization. Overall, our work simplifies the application of Stein's identity for an important class of non-Gaussian distributions.

cs.LG↗

arXiv2025

Sparse Activations as Conformal Predictors

Conformal prediction is a distribution-free framework for uncertainty quantification that replaces point predictions with sets, offering marginal coverage guarantees (i.e., ensuring that the prediction sets contain the true label with a specified probability, in expectation). In this paper, we uncover a novel connection between conformal prediction and sparse softmax-like transformations, such as sparsemax and $γ$-entmax (with $γ> 1$), which may assign nonzero probability only to a subset of labels. We introduce new non-conformity scores for classification that make the calibration process correspond to the widely used temperature scaling method. At test time, applying these sparse transformations with the calibrated temperature leads to a support set (i.e., the set of labels with nonzero probability) that automatically inherits the coverage guarantees of conformal prediction. Through experiments on computer vision and text classification benchmarks, we demonstrate that the proposed method achieves competitive results in terms of coverage, efficiency, and adaptiveness compared to standard non-conformity scores based on softmax.

cs.LG↗