Optimal aggregation in multiclass classification

Felix Abramovich; Or Davidovitch-Servadio

Home/Archive/arXiv

arXiv · 2610.28025

Optimal aggregation in multiclass classification

Felix Abramovich·Or Davidovitch-Servadio

View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper studies the problem of aggregation in multiclass classification, generalizing existing results for binary classification. Given a finite dictionary of score functions F={f_1,...,f_M} and a training sample, the goal is to construct a data-driven aggregate whose risk is as close as possible to that of the best aggregate (oracle) in a prescribed subset of span(F) with respect to a given loss. In particular, we investigate model selection and convex aggregation under the 0/1 and multinomial logistic losses. For the 0/1 loss, we show that the optimal aggregation rates remarkably do not depend on the number of classes K, and the corresponding empirical risk minimization (ERM) aggregates are rate-optimal for model selection and nearly-optimal for convex aggregation (up to an additional sqrt(ln(K)) factor). The situation under the multinomial logistic loss is more subtle. Although the optimal aggregation rates remain independent of K, restricting aggregation to proper aggregates in conv(F) leads to a deterioration of the achievable rates with K in certain regimes. This reveals a fundamental limitation of proper aggregation in multiclass classification. To overcome it, we propose improper probability-mixture aggregates that average the induced class probabilities rather than the original score functions and map the result back to score-function space. These aggregates achieve optimal or nearly optimal convergence rates independent of K. Finally, we introduce a different aggregation framework in which both the aggregation class and the oracle are defined directly in probability space.

Explore related subjects

math.ST stat.ME stat.ML stat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Felix Abramovich, Or Davidovitch-Servadio. 2026-10-20. Optimal aggregation in multiclass classification. https://arxiv.org/abs/2610.28025

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Extreme change-point detection

We examine rules for predicting whether a point in $\mathbb{R}$ generated from a 50-50 mixture of two different probability distributions came from one distribution or the other, given limited (or no) information on the two distributions, and, as clues, one point generated randomly from each of the two distributions. We prove that nearest-neighbor prediction does better than chance when we know the two distributions are Gaussian densities without knowing their parameter values. We conjecture that this result holds for general probability distributions and, furthermore, that the nearest-neighbor rule is optimal in this setting, i.e., no other rule can do better than it if we do not know the distributions or do not know their parameters, or both.

math.ST↗

arXiv2026

High-dimensional linear regression inference via $\ell^2$ weak convergence

We prove weak convergence in a separable Hilbert space for estimators of high-dimensional regression coefficients, which yields asymptotic normality and enables direct use of standard asymptotic tools such as the continuous mapping theorem. The approach permits diverging sparsity with a growing number of small nonzero coefficients, while requiring that only finitely many have moderate magnitude. As applications, we develop a test for finitely many linear hypotheses and, via a Scheffé-type approach, simultaneous inference for infinitely many linear hypotheses, yielding both a global test and simultaneous confidence bands for the regression function. The limiting distributions are given by weighted sums of independent chi-squared variables, and plug-in critical values achieve asymptotically correct size.

math.ST↗

arXiv2026

Third and Fourth Moment Phenomena for Monochromatic Edges and Applications to Graph-Based Two-Sample Tests

Given a graph sequence $\{G_n\}_{n\in\mathbb{N}}$, denote by $T_n$ the number of monochromatic edges when the vertices of $G_n$ are independently colored red or blue with probabilities $p$ and $1-p$, respectively. We characterize the asymptotic normality of $T_n$, after centering and scaling to unit variance, in terms of subgraph (motif) counts and standardized moments. One of the highlights of our results is a {\it third moment phenomenon}: for every fixed $p\in(0,1)\backslash\{\frac12\}$, asymptotic normality holds if and only if the standardized third moment converges to zero. The more familiar {\it fourth moment condition}, which requires the standardized fourth moment to converge to $3$, gives an equivalent characterization except at the two exceptional probabilities $p=\frac{1\pm1/\sqrt3}{2}$. At these values, the standardized fourth moment converges to $3$ for every graph sequence with a diverging number of edges, even when Gaussian convergence fails. In contrast, in the balanced case $p=\frac12$, the fourth moment condition characterizes asymptotic normality, while the third moment condition is insufficient. These results identify three distinct regimes for moment characterizations of Gaussian convergence, determined by the coloring probability. We further show that the third moment condition remains sufficient in the sparse regime, where $p=p_n \to 0$. As an application, we derive the limiting null distribution of two-sample statistics based on undirected $K$-nearest-neighbor graphs in the Poissonized setting, identifying sharp growth conditions on $K$ for asymptotic normality.

math.ST↗