Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.33901

The Computational Assessment Stack: An Architecture for AI-Native Measurement

Alina A. von Davier
View original source ↗Read source document ↗Export metadata ↓

Abstract

Test development is shifting toward automation and agentic artificial intelligence (AI), in which autonomous systems draft items at scale, assemble test forms, administer adaptive assessments, score complex responses, and monitor test quality. As autonomous systems take on more of this work, the architecture underlying test development must change to preserve assessment quality under the new paradigm. This paper proposes the Computational Assessment Stack (CAS), a four-layer architecture for AI-native measurement where every stage of the assessment lifecycle shares a unifying computational construct representation as its base layer. This representation is a machine-readable, modular, and versioned knowledge graph. Three layers are built above it: computational content generation that builds on the knowledge graph; computational psychometrics that derives substantive structure directly from the knowledge graph; an orchestration layer in which agentic systems operationalize the stack under human governance. The paper's core contribution is architectural. Building on foundational psychometric frameworks, the CAS provides a persistent representation that unifies the four layers. Unlike the descriptive prose traditionally used to define constructs in test development, the proposed architecture renders measurement assumptions inspectable and their consequences empirically testable.

Explore related subjects

stat.APcs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Alina A. von Davier. 2026-10-24. The Computational Assessment Stack: An Architecture for AI-Native Measurement. https://arxiv.org/abs/2610.33901

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A multistate approach to disability insurance reserving with information delays

Disability insurance claims are often affected by lengthy reporting delays and adjudication processes. The classic multistate life insurance modeling framework is ill-suited to handle such information delays since the cash flow and available information can no longer be based on the biometric multistate process determining the contractual payments. We propose a new individual reserving model for disability insurance schemes which describes the claim evolution in real-time. Under suitable independence assumptions between the available information and the underlying biometric multistate process, we show that these new reserves may be calculated as natural modifications of the classic reserves. For estimation of the model constituents, we employ the procedure proposed in [Buchardt, K., Furrer, C., & Sandqvist, O. L. (2025). Estimation for multistate models subject to reporting delays and incomplete event adjudication. The Annals of Applied Statistics, 19(4), 2782-2802. https://doi.org/10.1214/25-AOAS2068]. A real data application illustrates how the proposed estimation and reserving methods can be implemented and compares the resulting reserves with simpler reserving approaches.

stat.AP↗
arXiv2026

Improving causal inference in interrupted time series analysis: the triple difference design

Background: Interrupted time series analysis (ITSA) is widely used to evaluate health policy and intervention effects. While multiple-group ITSA (MG-ITSA) improves causal inference by incorporating a control group, residual confounding from unmeasured time-varying factors may remain. The triple-difference interrupted time series (DDD-ITSA) design extends this approach by adding a second control group to further isolate treatment effects, but it remains underutilized and lacks formal guidance. Methods: We formalize the DDD-ITSA framework, specify the regression model, define key parameters for estimating level and trend effects, and clarify interpretation of the triple-difference estimand. We illustrate the approach using a worked example evaluating California's Proposition 99 cigarette tax and its impact on per-capita cigarette sales. Results: In the example, all groups were balanced on pre-intervention level and trend. The triple-difference estimand indicated a statistically significant annual reduction of -1.76 per-capita cigarette packs in California relative to the secondary control (P = 0.020; 95 percent CI: -3.24, -0.28), consistent with results from the primary comparison. Differences between control groups were not significant. Conclusions: DDD-ITSA strengthens causal inference when two-group comparisons may be confounded by leveraging an additional control group to remove remaining biases and assess heterogeneity. Implementation is facilitated by updates to the itsa Stata package. Careful attention to control selection, baseline balance, and autocorrelation remains essential.

stat.AP↗
arXiv2026

Leveraging Large Language Models to Improve Precision in Randomized Controlled Trials

Large language models (LLMs) are increasingly used in statistical research and applications. However, they are also notorious for unreliable or biased information. Here, we explore whether LLMs can be used to improve the precision of randomized controlled trials (RCTs) in a safe and rigorous way. We consider two pathways for incorporating LLM assistance: (1) using LLMs to transform non-standard covariates, such as text, into numerical representations, and (2) leveraging the general knowledge embedded in LLMs to generate predicted outcomes. Although external data sources have been leveraged to improve precision before, the value added by LLM-generated predictions through these pathways remains an open question. We develop a pipeline for generally leveraging LLMs in this context and apply it to three different case studies. We find that we are able to safely improve precision, primarily through transforming non-standard covariates, but that LLMs also possess general knowledge that is useful in some contexts.

stat.AP↗