Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2407.10308

AI Detectors are Poor Western Blot Classifiers: A Study of Accuracy and Predictive Values

Romain-Daniel Gosselin
View original source ↗Read source document ↗Export metadata ↓

Abstract

The recent rise of generative artificial intelligence (GenAI) capable of creating scientific images presents a challenge in the fight against academic fraud. This study evaluates the efficacy of three free web-based AI detectors in identifying AI-generated images of Western blots, which is a very common technique in biology. We tested these detectors on a collection of artificial Western blot images (n=48) that were created using ChatGPT 4 DALLE 3 and on authentic Western blots (n=48) that were sampled from articles published within four biology journals in 2015; this was before the rise of generative AI based on large language models. The results reveal that the sensitivity (0.9583 for Is It AI, 0.1875 for Hive Moderation, and 0.7083 for Illuminarty) and specificity (0.5417 for Is It AI, 0.8750 for Hive Moderation, and 0.4167 for Illuminarty) are very different. Positive predictive values (PPV) across various AI prevalence were low, for example reaching 0.1885 for Is It AI, 0.1429 for Hive Moderation, and 0.1189 for Illuminarty at an AI prevalence of 0.1. This highlights the difficulty in confidently determining image authenticity based on the output of a single detector. Reducing the size of Western blots from four to two lanes reduced test sensitivities and increased test specificities but did not markedly affect overall detector accuracies and also only slightly improved the PPV of one detector (Is It AI). These findings strongly argue against the use of free AI detectors to detect fake scientific images, and they demonstrate the urgent need for more robust detection tools that are specifically trained on scientific content such as Western blot images.

Explore related subjects

q-bio.QM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Romain-Daniel Gosselin. 2024-07-14. AI Detectors are Poor Western Blot Classifiers: A Study of Accuracy and Predictive Values. https://arxiv.org/abs/2407.10308

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Lightning-fast adaptive immune receptor similarity search by symmetric deletion lookup

An individual's adaptive immune receptor (AIR) repertoire records immune history due to the receptors' antigen specificity. Reading this record requires computational approaches for inferring receptor function from sequence, as the diversity of possible receptor-antigen pairs vastly outstrips experimental knowledge. Identification of AIRs with similar sequence and thus putatively similar function is a common performance bottleneck in these approaches. Here, we benchmark the runtime scaling of five algorithmic approaches to radius-based search for Levenshtein neighbors. We show that a symmetric deletion lookup approach, originally proposed for spell-checking, is particularly scalable. We introduce SymScan and XTNeighbor, optimized CPU and GPU software implementing a parallelized variant of the algorithm. For one million input sequences, these tools identify all sequence pairs that differ by one or two edits in seconds, orders of magnitude faster than existing approaches. We demonstrate how symmetric deletion lookup can be integrated as a pre-filtering step in T cell receptor metaclone discovery and B cell receptor lineage identification. Our contribution is poised to greatly accelerate existing analysis pipelines and enable processing of immunosequencing data at scale.

q-bio.QM↗
arXiv2026

Implication of modelling choices on connectivity estimation: A comparative analysis

Landscape connectivity is an important field with important conservation implications. Connectivity modelling is a useful tool to inform and guide landscape planning. However, it involves assumptions and methodological decisions which ultimately impact connectivity outcomes. In order to understand the implications of modelling choices on final connectivity estimations, we compare two landscape characterisation approaches - expert knowledge and species distribution models - and three movements models - least-cost paths, circuit theory and an individual-based movement simulator. The implementation of the models and the construction of the analyses scope highlighted conceptual and methodological differences that made the comparison difficult. Landscape characterisation appears as the principal factor determining connectivity outcomes. Therefore, the confrontation between expert knowledge and species distribution models is critical to leverage points of convergence and complementarity between these two approaches. Conceptual differences between movement models are reported on connectivity map and habitat patch contribution estimations. In the want of data and protocol design specifically to validate connectivity models, approaches that integrate stochastic and behavioural processes, bring a more realistic perspective to connectivity estimation.

q-bio.QM↗
arXiv2026

Triplication: an important component of the modern scientific method

A scientific-study protocol (defined) is designed to deliver results from which inductive inference is allowed. In the nineteenth century, triplication was introduced into the plant sciences and Fisher's p<0.05 rule (1925) incorporated into triple-result protocols designed to counter random/systematic errors which contribute to real-world variability. The aims of the present study were to: (1) classify replication protocols; (2) assess their prevalence in plant-science studies (published during one twenty-first-century year; for defined variable construct); (3) explore triplication rationale. Methods: a plant-sciences protocol-prevalence report was produced; experimental/associational-study proportions analyzed; and real-world-data proxies used to show confidence-interval-width patterns with increasing replicate number. Results: 25% plant-science studies analyzed showed triplication, including 11% triple-result protocols (including greater replicate numbers: 48%;17%, respectively). Theoretical considerations indicated that even if systematic errors predominate, (previously-known) square-root rules sometimes apply, contributing to triplication importance (exemplified by real-world-data proxies). Conclusions: The defined protocols, with minor modifications, should provide the means for assessment of most sciences. Triplication was extensively applied in studies analysed and there are strong methodological reasons why triplication, rather than duplication/quadruplication, is the appropriate standard: triple-result protocols: (a) effectively reduce false positives to acceptable levels; (b) give qualitatively-different information (shape) from duplication; (c) have a large efficiency advantage (concerning confidence-interval widths) over quadruplication. The application of batch replication is not, primarily, a statistical problem and cannot effectively be replaced by simulation.

q-bio.QM↗