Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2508.02966

Measuring Human Leadership Skills with Artificially Intelligent Agents

Ben Weidmann·Yixian Xu·David J. Deming
View original source ↗Read source document ↗Export metadata ↓

Abstract

We show that the ability to lead groups of humans is predicted by leadership skill with Artificially Intelligent agents. In a large pre-registered lab experiment, human leaders worked with AI agents to solve problems. Their performance on this 'AI leadership test' was strongly correlated with their causal impact on human teams, which we estimate by repeatedly randomly assigning leaders to groups of human followers and measuring team performance. Successful leaders of both humans and AI agents ask more questions and engage in more conversational turn-taking; they score higher on measures of social intelligence, fluid intelligence, and decision-making skill, but do not differ in gender, age, ethnicity or education. Our findings indicate that AI agents can be effective proxies for human participants in social experiments, which greatly simplifies the measurement of leadership and teamwork skills.

Explore related subjects

econ.GNq-fin.ECstat.OT

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Ben Weidmann, Yixian Xu, David J. Deming. 2025-08-05. Measuring Human Leadership Skills with Artificially Intelligent Agents. https://arxiv.org/abs/2508.02966

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Decomposing Wage Stagnation: Employment Reallocation, Wage Structure,and Demographics

Japan's average log real hourly wages rose until the mid-1990s, declined through the mid-2010s, and partially recovered thereafter. This paper decomposes these changes over 1980-2024 into four components: demographic change across worker types, changes in relative employment shares across job types, changes in relative log wages across job types, and unweighted mean wage growth. The framework combines a shift-share decomposition across worker types with an extension of the Olley-Pakes decomposition across job types within worker types, separating employment reallocation from changes in relative wage structure. The four components vary across periods. Before 1996, unweighted mean wage growth and changes in relative wage structure contribute positively, while demographic change and employment reallocation contribute negatively. During 1996-2014, all four components are negative. After 2014, the recovery mainly reflects unweighted mean wage growth. Employment reallocation and changes in relative wage structure contribute differently across dimensions of job type.

econ.GN↗
arXiv2026

Complements or Substitutes? Technology Adoption and Clinical Care Utilization: Evidence from Automated Insulin Delivery

Whether medical technology reduces or increases demand for professional care is central to understanding its effects on healthcare utilization and costs. We study automated insulin delivery (AID) adoption among 1,608 adults with type 1 diabetes treated in four specialist clinics of the Italian National Health Service, 283 of whom adopt AID. Because adoption is clinically targeted and staggered, we combine risk-set coarsened exact matching with a staggered event-study design. Matching retains 181 adopters with comparable untreated controls. We further adjust for differential pre-adoption trends by extrapolating the untreated trajectory in event time. After adjustment, AID adoption is associated with greater routine outpatient engagement. The estimated effect on the probability of a diabetologist visit is 16.3 percentage points one semester after adoption and 39.3 percentage points four semesters after adoption, relative to a 55.2% visit probability in the semester before adoption. HbA1c testing and a broader process-of-care index show similar positive patterns, although the evidence is less precise. The visit estimates remain sensitive to the trend-extrapolation assumption and should therefore be interpreted conditionally on that restriction. The findings are consistent with a task-based view in which automation substitutes for routine dosing decisions while complementing clinical labor through interpretive, adjustment, and supervisory tasks. Patient-facing automation may therefore reorganize rather than simply reduce healthcare use.

econ.GN↗
arXiv2026

Mining Meaning: Measurement Error in AI-Assisted Literature Reviews

Researchers increasingly use generative AI, particularly large language models (LLMs), to automate tasks across the research pipeline. We study the reliability of these tools at the reading, classification, and synthesis of large bodies of academic literature. We frame LLM-assisted literature reviews as a measurement problem, treating models as measurement systems and tracing how their errors affect downstream conclusions. As a test case, we use three different implementations of ChatGPT to identify and extract metadata from economics papers that use rainfall as an instrumental variable. We benchmark each implementation against a subset of human-labeled evaluation data, and then deploy those implementations to extract metadata from the full corpus. The LLMs perform well on binary classification, but performance deteriorates as tasks demand greater contextual interpretation. More importantly, how much researchers can rely on model outputs depends not only on the complexity of the reading task but also on the type of claims the data is asked to support. The same amount of measurement error substantially affects paper-level claims while having little effect on broader claims about the literature. Measurement error in LLM-generated data is thus most consequential at precisely the level of detail that constitutes an LLM's principal value added over human reviewers. We conclude that standard model performance metrics are informative about the quality of generated data but do not by themselves establish the credibility of downstream inference. Researchers must also evaluate whether substantive claims are robust to the measurement system used to generate the underlying data.

econ.GN↗