Agentic Distributional Regression: Conditional Density Estimation through Agent-Written Programs

Lorenzo Tedesco

Home/Archive/arXiv

arXiv · 2610.37611

Agentic Distributional Regression: Conditional Density Estimation through Agent-Written Programs

Lorenzo Tedesco

View original source ↗Read source document ↗Export metadata ↓

Abstract

In this paper we propose a method for estimating conditional densities from an iid sample using a language-model agent. The agent writes the estimator as a short program that, for each covariate value, returns the estimated conditional distribution of the response. The program is built from a library of probability distributions, so the returned law can be a parametric distribution, a mixture or a multivariate factorization, with parameters given by explicit functions of the covariates. Since the number of operations and constants in the program is limited, the fitted conditional density is a formula that can be inspected and evaluated independently of the agent, while the choice of the model structure and of the fitting procedure is left to the agent. We call this approach agentic distributional regression (ADR). Treating the returned program as a random conditional density estimator, we derive a finite-sample error bound under explicit complexity and optimization assumptions, and an oracle inequality for selection using independent validation data. For a stably selected regular structure with sufficiently accurate fitting, we obtain asymptotic normality of the fitted constants. In simulation studies, on twelve real datasets and on a new dataset collected after the agents' training data, ADR with Claude Opus 5.5 has the best average performance among the methods considered.

Explore related subjects

stat.ME stat.ML

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Lorenzo Tedesco. 2026-10-27. Agentic Distributional Regression: Conditional Density Estimation through Agent-Written Programs. https://arxiv.org/abs/2610.37611

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Principal Decomposition with Nested Submanifolds

Over the past decades, the increasing dimensionality of data has increased the need for effective data decomposition methods. Existing approaches, however, often rely on linear models or lack sufficient interpretability or flexibility. To address this issue, we introduce a novel nonlinear decomposition technique called the principal nested submanifolds, which builds on the foundational concepts of principal component analysis. This method exploits the local geometric information of data sets by projecting samples onto a series of nested principal submanifolds with progressively decreasing dimensions. It effectively isolates complex information within the data in a backward stepwise manner by targeting variations associated with smaller eigenvalues in local covariance matrices. The fitted structures are smooth nested manifolds rather than linear or special-shape subspaces. Simulation studies and an analysis of single-cell RNA-sequencing data illustrate how the method summarizes locally varying nonlinear structure while retaining a compatible hierarchy across dimensions. It provides flexible subspace constraints that facilitate the extraction of data components and noise reduction. This approach advances the non-Euclidean statistical analysis of data with low-dimensional intrinsic structure within Euclidean spaces and offers a complementary perspective for high-dimensional noisy data sets in fields such as bioinformatics and machine learning.

stat.ME↗

arXiv2026

Two-sample comparison through additive tree models for density ratios

The ratio of two densities provides a direct characterization of their differences. We consider the two-sample comparison problem by estimating this ratio given i.i.d. observations from two distributions. To this end, we propose additive tree models for density ratio estimation along with efficient algorithms using a new loss function, the balancing loss. The loss allows tree-based models to be trained using several algorithms originally designed for supervised learning, such as forward-stagewise optimization and gradient boosting. Moreover, the balancing loss resembles an exponential family kernel, and it can serve as a pseudo-likelihood with conjugate priors. This property enables generalized Bayesian inference on the density ratio using backfitting samplers designed for Bayesian additive regression trees (BART). Our Bayesian strategy provides uncertainty quantification for the inferred density ratio, which is critical for applications involving high-dimensional and data-limited distributions with potentially substantial uncertainty. We further show connections of the balancing loss to the exponential loss in binary classification and to the variational form of f-divergence, particularly the squared Hellinger distance. Numerical experiments demonstrate that the proposed boosting algorithms achieve high accuracy at low computational cost, and that the Bayesian additive tree is competitive in accuracy while additionally providing uncertainty quantification, which the compared estimators do not offer. Finally, we demonstrate its application to assessing the quality of generative models for microbiome compositional data.

stat.ME↗

arXiv2026

Learning Sequential Decisions from Multiple Sources via Group-Robust Markov Decision Processes

Learning a common policy from multiple source environments requires accounting for cross-source heterogeneity, and the offline setting introduces an additional challenge of limited data coverage. We formulate this problem as a distributionally robust Markov decision process under a shared linear MDP representation with source-specific transition and reward components. The uncertainty set is constructed through feature-wise mixtures of these components, yielding a $d$-rectangular structure that captures cross-source heterogeneity and admits a robust Bellman recursion. Building on this formulation, we develop Group-Robust Pessimistic Value Iteration (GR-PEVI), an offline policy-learning algorithm that combines feature-wise worst-case aggregation with a data-dependent pessimism penalty. Under a robust partial coverage condition, we establish finite-sample suboptimality guarantees for GR-PEVI and extend the analysis to clustered sources, characterizing the trade-off between the statistical gain from pooling similar sources and the approximation error induced by within-cluster heterogeneity. Simulation studies demonstrate the effectiveness of GR-PEVI, with empirical behavior consistent with the theoretical analysis. We apply GR-PEVI to a multi-source electronic health record study of multiple sclerosis treatment decisions to illustrate its use in a clinical setting.

stat.ME↗