Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2505.01259

Digital Pathway Curation (DPC): a pipeline able to assess the reproducibility, consensus and accuracy in biomedical search retrieval by comparing Gemini, PubMed, and Scientific Reviewers

Flavio Lichtenstein·Daniel Alexandre de Souza·Carlos Eduardo Madureira Trufen·Victor Wendel da Silva Gonçalves·Juliana de Paula Bernardes·Vinicius Miranda Baroni·Carlos DeOcesano-Pereira·Leonardo Fontoura Ormundo·Fabio Augusto Labre de Souza·Olga Celia Martinez Ibañez·Nancy Starobinas·Luciano Rodrigo Lopes·Aparecida Maria Fontes·Sonia Aparecida de Andrade·Ana Marisa Chudzinski-Tavassi
View original source ↗Read source document ↗Export metadata ↓

Abstract

A scientific study begins with a central question, and search engines like PubMed are the first tools for retrieving knowledge and understanding the current state of the art. Large Language Models (LLMs) have been used in research, promising acceleration and deeper results. However, besides caution, they demand rigorous validation. Assessing complex biological relationships remains challenging for SQL-based tools and LLM models. Here, we introduce the Digital Pathway Curation (DPC) pipeline to evaluate the reproducibility and accuracy of the Gemini models against PubMed search and human expert curation. Using two omics experiments, we created a large dataset (Ensemble) based on determining pathway-disease associations. With the Ensemble dataset, we demonstrate that Gemini achieves high run-to-run reproducibility of approximately 99% and inter-model reproducibility of around 75%. Next, we calculate the crowdsourced consensus using a smaller dataset. The CSC allows us to calculate accuracies, and the Gemini multi-model consensus reached a significant accuracy of about 87%. Our findings demonstrate that LLMs are reproducible, reliable, and valuable tools for navigating complex biomedical knowledge.

Explore related subjects

q-bio.QM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Flavio Lichtenstein, Daniel Alexandre de Souza, Carlos Eduardo Madureira Trufen, Victor Wendel da Silva Gonçalves, Juliana de Paula Bernardes, Vinicius Miranda Baroni, Carlos DeOcesano-Pereira, Leonardo Fontoura Ormundo, Fabio Augusto Labre de Souza, Olga Celia Martinez Ibañez, Nancy Starobinas, Luciano Rodrigo Lopes, Aparecida Maria Fontes, Sonia Aparecida de Andrade, Ana Marisa Chudzinski-Tavassi. 2025-05-07. Digital Pathway Curation (DPC): a pipeline able to assess the reproducibility, consensus and accuracy in biomedical search retrieval by comparing Gemini, PubMed, and Scientific Reviewers. https://arxiv.org/abs/2505.01259

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Multimodal AI predicts clinical outcomes of drug combinations from preclinical data

Predicting clinical outcomes from preclinical data is essential for selecting safe and effective drug combinations and for reducing late-stage failures. AI models use molecular structure and target annotations, and do not leverage the perturbation readouts that report how a compound acts in a cellular context. Here we introduce Madrigal, a multimodal AI model that learns from structural, pathway, cell-viability, and transcriptomic data. Madrigal aligns these modalities across 21,842 compounds into a shared latent space and predicts combination outcomes even for drugs observed in only a subset of the data modalities. Trained on 158 expert-curated and 795 patient-reported combination outcomes, Madrigal outperforms single-modality and state-of-the-art multimodal methods. Ablations show that modality alignment and multimodal input each improve predictive performance. Madrigal predicts elevated risk for combinations that share membrane transporters. In head-to-head trials that compare two combination arms,the arm with the higher observed incidence of neutropenia, anemia, alopecia, or hypoglycemia receives the higher predicted risk in 25 of 28 comparisons. In MASH, Madrigal ranks resmetirom among the candidates with favorable predicted safety when paired with type 2 diabetes drugs. Madrigal also improves adverse-event prediction in a longitudinal patient cohort and an independent oncology cohort and predicts efficacy in primary acute myeloid leukemia samples and patient-derived xenografts.

q-bio.QM↗
arXiv2026

ProteoEM: probabilistic protein abundance estimation from iterative affinity traces

Single-molecule affinity mapping enables molecular-level measurement of proteins and proteoforms, but imperfect and nonspecific probe binding makes individual affinity traces compatible with multiple molecular identities. Accurate abundance estimation therefore requires apportionment of ambiguous traces by weight rather than assignment to a single candidate. We developed ProteoEM, an expectation-maximization framework for weighted proteoform quantification, inspired by transcript abundance estimation methods for RNA sequencing and released as an open-source Python package. ProteoEM evaluates each molecule against every candidate using fixed, pre-calibrated probe-response rates held separate from the abundance estimate, while retaining the full likelihood of the observed affinity features. The framework estimates proteoform abundances, reports indistinguishable proteoforms as groups when measurements cannot separate them, and accounts for differential observation yields to distinguish the composition of observed molecules from that of the source sample. In simulations, ProteoEM accurately recovered the underlying molecular composition where approaches that reduce each trace to a hard yes/no call introduced substantial errors. ProteoEM's performance was insensitive to a moderate, uniform calibration error but was biased by informative missing data and by proteoforms absent from the reference. When observation yields were known, it also recovered source-sample composition from observed molecular counts. ProteoEM provides an open-source, reproducible framework for quantitative analysis of single-molecule affinity measurements, and these results motivate validation on experimental molecule-level data.

q-bio.QM↗
arXiv2026

PRAXIS-VirtualCell: A Programmable and Trustworthy Framework for Agentic Virtual Cell Experiments

Virtual cells are evolving from single-task predictive models toward programmable biological simulation systems, yet heterogeneous data, models, and validation evidence still lack a unified organizational framework. Here, we present PRAXIS-VirtualCell, a modular framework that organizes biological data, predictive models, perturbations, adapters, execution environments, and validation evidence to enable reproducible and auditable virtual experiments. The system supports cross-species tasks spanning Escherichia coli, Saccharomyces cerevisiae, and human K562 cells, while using biological contracts and evidence-aware execution to distinguish supported predictions from extrapolation and abstention. By further integrating agentic orchestration, PRAXIS-VirtualCell automatically translates natural-language questions into traceable virtual experiments, providing a unified runtime foundation for trustworthy and scalable Virtual Cell systems.

q-bio.QM↗