Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools

Bryce Cai; Geetha Jeyapragasan; Samira Nedungadi; Jake Yukich; Seth Donoughe

Home/Archive/arXiv

arXiv · 2609.05818

Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools

Bryce Cai·Geetha Jeyapragasan·Samira Nedungadi·Jake Yukich·Seth Donoughe

View original source ↗Read source document ↗Export metadata ↓

Abstract

We introduce ABLE, a benchmark for evaluating LLM agents' ability to use biological AI models (BAIMs), such as ProteinMPNN and AlphaFold3, in dual-use protein design workflows. ABLE assesses agent performance through a set of tasks spanning structure retrieval, sequence generation, and design validation. We evaluate 15 frontier models and find that seven refuse all tasks, while the remaining models exhibit substantial performance differences. Claude Sonnet 4 and Gemini 3 Pro achieve the highest scores across information retrieval, tool selection, and tool use. We further compare model performance on a subset of tasks against an expert human baseline. Our results suggest that current LLMs can substantially lower barriers to protein design, but remain inconsistent in planning, strategy generation, and integrating biological knowledge with tool use.

Explore related subjects

cs.AI cs.CY q-bio.QM

Keep this discovery

BibTeX RIS

Bryce Cai, Geetha Jeyapragasan, Samira Nedungadi, Jake Yukich, Seth Donoughe. 2026-09-05. Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools. https://arxiv.org/abs/2609.05818

Cite the original work for its findings. Save a collection to share your selection of sources.

Discover connections

cs.AI subject cs.CY subject q-bio.QM subject

Connections use source metadata and explicit phrase matches, not verified experimental comparisons.

KEEP EXPLORING

Related papers

arXiv2026

OmniBioTwin: A System-of-Twinned-Systems Framework for Health Digital Twins

Health digital twins (HDTs) promise patient-specific modeling and decision support but current approaches remain structurally fragmented: monolithic models that address a single organ or task lack cross-scale fidelity, while system-level twins lack generalizable architectural frameworks. We propose OmniBioTwin, a System-of-Twinned-Systems (SoTS) framework that organizes HDTs as modular computational entities coupled through explicit interaction operators within a multi-layer network architecture. The framework comprises seven coordinated layers - spanning data integration, autonomous twin modeling, cross-scale coupling, temporal synchronization, and human-in-the-loop decision support. We demonstrate OmniBioTwin by instantiating a multiscale twin for glucagon-like peptide-1 (GLP-1) signaling pathways in Alzheimer's disease, illustrating how molecular, cellular, and organ-level twins can be composed and coupled within a unified system.

q-bio.QM↗

arXiv2026

SCALE:Scalable Conditional Atlas-Level Endpoint transport for virtual cell perturbation prediction

Virtual-cell models aim to predict how cell populations respond to perturbations, but control and treated cells are measured as unpaired populations, complicating the learning of perturbation-specific effects. We present SCALE, a conditional transport model that represents cells as unordered sets and predicts treated populations without cell-level matching. A shared set-aware encoder and conditional DiT backbone learn latent transport, making endpoint supervision directly delta-aligned without an auxiliary delta objective. Across genetic, chemical, developmental and immune perturbations, SCALE recovered gene-expression changes, response directions and population structure. In CRISPR data with dominant cell-line effects, SCALE outperformed competing methods across seven metrics and maintained separation among gene-target representations rather than collapsing them into a shared region. SCALE further prioritized cytokines predicted to produce distinct immune activation and inflammatory responses. Experiments using matched PBMC samples from three donors confirmed these predicted differences. Together, SCALE enables perturbation-specific prediction from unpaired populations and supports experimental prioritization.

cs.LG↗

arXiv2026

Score-Based Generative Data Assimilation for Integrating Aggregated Surveillance Data into Agent-Based Models in Epidemic Tracking

Reliable epidemic monitoring often requires inferring regional infection burden and transmission heterogeneity from noisy, spatially aggregated, and potentially sparse surveillance data. Agent-based models (ABMs) are attractive for this task because they represent individual behavior, contact heterogeneity, and localized interventions, but these same features make them difficult to calibrate online. We develop a generative AI-based data-assimilation (GenDA) framework for partially observed epidemic ABMs that estimates both the epidemic state and a heterogeneous parameter field while respecting the gap between observable macrostates and latent agent-level microstates. GenDA combines a training-free, score-based generative update for macrostate correction with a direct parameter update based on macrostate discrepancies, followed by a macro-micro reassignment step that restores consistency with the ABM. In controlled and geographically explicit synthetic experiments, the framework recovers regional epidemic burden, dominant hotspot structures, and effective transmission heterogeneity from aggregated observations, while improving post-assimilation forecasts relative to state-only assimilation.

math.NA↗