Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.28929

Scriptorium: System Design and Implementation of a Multi-Human, Multi-AI Collaborative Digital Library Search Platform

Abbas Pirmoradi·Milad Momeni·Anna Chu·Orland Hoeber
View original source ↗Read source document ↗Export metadata ↓

Abstract

This technical report documents the design and implementation of Scriptorium, a digital library search interface designed to support multi-human, multi-AI collaboration: full cross-user transparency of search activity combined with a Manager--Worker architecture of seven role-specialized generative AI agents. A companion short demonstration paper describes Scriptorium's interaction design; because a demonstration format is constrained to a few pages, it necessarily omits most implementation detail. This report documents Scriptorium's complete system architecture, data model, collaborative-interaction design rationale, and AI orchestration mechanism, including verbatim agent prompts, the deterministic guardrails that constrain them, the language model and retrieval configuration, and the steps needed to run the system, so that the design can be inspected, critiqued, reproduced, and reused by other researchers. We also discuss the system's current limitations and the user study we are planning to evaluate its central hypothesis: that combining collaborative transparency with scoped multi-agent AI assistance increases serendipitous cross-pollination between collaborators' research threads.

Explore related subjects

cs.HCcs.IR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Abbas Pirmoradi, Milad Momeni, Anna Chu, Orland Hoeber. 2026-10-20. Scriptorium: System Design and Implementation of a Multi-Human, Multi-AI Collaborative Digital Library Search Platform. https://arxiv.org/abs/2610.28929

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Knowledge-Certainty Diversity Predicts Human-Human and Human-AI Open-Ended Discussion Synergy and Reveals Pure-AI Discussion Shortfalls

Conversations transform individual knowledge into collective insight, enabling collaborators to solve problems more accurately than they could alone. Whether dialogues among large language models (LLMs) can replicate the synergistic gains observed in human discussion remains unclear. We systematically compared four interaction settings, LLM-LLM pairs, LLM trios, human trios, and human-LLM pairs, using validated medical multiple-choice questions. Agents answered individually, engaged in open-ended discussion, then re-answered, allowing us to quantify conversational gains. Interactions that included humans consistently yielded synergy (post-discussion accuracy increased for both stronger and weaker participants), whereas purely LLM groups did not improve and often declined. To explain and anticipate when open-ended dialogue helps, we introduce an agent-agnostic knowledge-certainty framework that models each participant by performance (accuracy) and confidence. This framework quantifies knowledge-certainty diversity, the degree to which one agent tends to be correct when another is uncertain, and yields an upper bound on the gain available from choosing between the agents' answers on the basis of confidence, which we term Potential Conversation Synergy. Computed before any discussion, this metric was low in both settings where discussion did not help and high in both where it did: When knowledge-certainty diversity is low (as in LLM-only groups), discussion doesn't improve performance; when it is present (as in human or human-LLM groups), open-ended dialogue reliably lifts accuracy. These findings suggest a diagnostic for AI collaboration: knowledge-certainty diversity can be measured from pre-discussion responses and provides a configuration-level indicator of conversational gains across the settings studied.

cs.HC↗
arXiv2026

The Feature Understandability Scale for Human-Centred Explainable AI: Assessing Tabular Feature Importance

As artificial intelligence becomes increasingly pervasive and powerful, the ability to audit AI-based systems is growing in importance. However, explainability for artificial intelligence systems is not a one-size-fits-all solution; different target audiences have varying requirements and expectations for explanations. While various approaches to explainability have been proposed, most explainable artificial intelligence methods for tabular data focus on explaining the outputs of supervised machine learning models using the input features. However, a user's ability to understand an explanation depends on their understanding of such features. Therefore, it is in the best interest of the system designer to try to pre-select understandable features for producing a global explanation of an ML model. Unfortunately, no measure currently exists to assess the degree to which a user understands a given input feature. This work introduces two psychometrically validated scales that quantitatively seek to assess users' perceived understanding of tabular input features for supervised classification problems. Specifically, these scales, one for numerical and one for categorical data, each with two factors and comprising 8 and 9 items, aim to assign a score to each input feature, effectively producing a rank and allowing for the quantification of feature prioritisation. A confirmatory factor analysis demonstrates a strong relationship between such items and a good fit of the two-factor structure for each scale. This research presents a novel method for assessing understanding and outlines potential applications in the domain of explainable artificial intelligence.

cs.HC↗
arXiv2026

Large Language Models Can Explain Experts Better Than Experts Themselves

Capturing tacit knowledge, or the "know-how" embedded in individual experiences and contexts, is a central challenge that limits the preservation and transfer of valuable expertise. Traditional methods rely on experts' ability to articulate their own reasoning, even though much of tacit knowledge is difficult to verbalize. Here, we show that Large Language Models (LLMs) can uncover tacit knowledge from expert behavior and support knowledge transfer from experts to novices. We first identify prompting strategies that improve LLMs' ability to externalize tacit knowledge (i.e., transform tacit into explicit knowledge), and then evaluate the downstream utility of the resulting LLM-externalized tacit knowledge. Our analysis reveals that LLM-externalized tacit knowledge incorporating expert behavior improves decision quality and helps reduce the expert-novice score gap, outperforming knowledge articulated by human experts. We further show that LLMs meaningfully learn and extract knowledge from expert behavior rather than simply mimicking expert responses, and that these findings generalize across different LLMs and information retrieval methods. These findings reaffirm Polanyi's Paradox---that "we can know more than we can tell"---and suggest that LLMs can help overcome human experts' articulation bottleneck that has long challenged tacit knowledge externalization.

cs.HC↗