Retrieval and competition: how a protein foundation model starts a protein

Piotr Jedryszek; Oliver M. Crook

Home/Archive/arXiv

arXiv · 2605.16331

Retrieval and competition: how a protein foundation model starts a protein

Piotr Jedryszek·Oliver M. Crook

View original source ↗Read source document ↗Export metadata ↓

Abstract

Protein language models are increasingly used to guide experimental and clinical decisions, yet it is often unclear whether a confident prediction reflects recognition of biological evidence or retrieval of a statistical default. We examine this distinction for a near-universal biological rule, that proteins begin with methionine, by tracing the computational pathway through which ESM2-8M produces this prediction. The model does not detect methionine at the masked position. Instead, it retrieves a methionine-favouring signal from a reference representation at the beginning-of-sequence token via a position-specific query assembled across layers, with the final output emerging through competition with context-dependent circuits. To understand how positional information reaches the readout, we introduce a norm-direction decomposition of attention scores within rotary frequency bands. Positional encoding operates through coupled changes in query norm and angular alignment distributed across these bands. On sequences whose true N-terminus is not methionine, where the biological question matters, the model predicts methionine anyway. This is not a correct prediction produced by an unexpected mechanism, but the output of a positional-prior retrieval circuit that matches the statistical average and fails where biology diverges from it. Distinguishing the two requires resolution at the level of individual circuits, frequency bands, and query composition, suggesting that mechanistic verification will be necessary, and challenging, for predictions where the biological stakes are higher. Even for the simplest biological rule, the model's prediction is mediated by a distributed computational circuit rather than direct recognition, suggesting that increasing task complexity will further obscure the relationship between model confidence and underlying biological evidence.

Explore related subjects

q-bio.BM cs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Piotr Jedryszek, Oliver M. Crook. 2026-06-03. Retrieval and competition: how a protein foundation model starts a protein. https://arxiv.org/abs/2605.16331

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Decoding enzyme-substrate interaction topology reveals principles underlying catalytic efficiency and mutational outcomes

The enzyme turnover number (kcat) defines catalytic efficiency and constrains quantitative models of metabolism, yet the molecular determinants governing kcat and its response to mutation remain poorly understood. Measurements are sparse and labor-intensive, and most computational approaches provide numerical predictions without explaining how enzyme-substrate interactions shape catalytic outcomes. A central challenge is therefore to identify the topological principles that determine where mutations act and how their functional outcomes are encoded within the enzyme-substrate interaction network. Here, we show that catalytic efficiency and mutational effects can be interpreted through enzyme-substrate interaction topology. We developed Interkcat, an interpretable bidirectional cross-attention framework that captures reciprocal coordination between protein residues and substrate atoms. Optimized on a unified benchmark, Interkcat achieves state-of-the-art predictive performance (R2 = 0.701). From its learned representations, we derive an Interaction Topology Score (ITS) that identifies sequence regions statistically enriched for mutation-sensitive sites without explicit structural inputs. We further demonstrate that higher-order topological features distinguish opposing mutational outcomes: lethal mutations disrupt coordinated networks, whereas activity-preserving or enhancing mutations retain sparse, globally organized coupling. These findings establish interaction topology as a unifying principle linking enzyme sequence, catalytic efficiency, and evolutionary perturbation.

q-bio.BM↗

arXiv2026

Adapting Boltz-2 with limited experimental activity data improves early enrichment in virtual screening

Virtual screening aims to prioritize active compounds from large chemical libraries within a limited experimental budget. When applying Boltz-2 to virtual screening, a key challenge is how to use limited experimental data from the target assay to improve the prioritization of active compounds. We investigated whether fine-tuning the Boltz-2 affinity heads with a small number of binary activity labels could improve early enrichment of active compounds in hit discovery. We compared fine-tuning with 40-300 labels in a retrospective evaluation on eight MF-PCBA targets. With 300 activity measurements, fine-tuning increased the number of actives in the top 1% by a geometric mean of 1.77-fold across the eight targets and improved average precision (AP) by 2.14-fold relative to the control without fine-tuning. We also investigated whether rescoring a subset of candidates could retain the improvement in hit recovery by reranking only the top-ranked Boltz-2 candidates with the fine-tuned head. Restricting rescoring to approximately 10% of the evaluation set retained hit recovery comparable to full rescoring. These findings show that affinity-head fine-tuning with limited activity labels improves early enrichment with Boltz-2 and that this benefit can be retained when rescoring a restricted set of candidates.

q-bio.BM↗

arXiv2026

SaltyMeta: a curated benchmark and protein language model-informed web tool for salty peptide prediction

Excess sodium intake remains a major public health challenge, while salty and saltiness-enhancing peptides offer a potential route to preserve sensory saltiness in reduced-sodium foods. Machine-learning studies of salty peptides, however, are constrained by small datasets, heterogeneous evidence standards, uncertain negative labels, and sequence similarity leakage. Here we present SaltyMeta, a curated benchmark and web-accessible screening framework for salty or saltiness-enhancing short peptides. The benchmark contains 580 peptides, including 280 positive peptides and 300 negative peptides, all standardized to 2-15 residue one-letter amino-acid sequences. Quality control found no non-standard residues, exact duplicates, or positive-negative overlaps. A similarity-grouped split retained 456 peptides for training and 124 for held-out testing. We evaluated 548 interpretable peptide descriptors, frozen ESM2 embeddings at 8M, 35M, and 150M parameter scales, and descriptor-embedding fusion models under grouped cross-validation. The traditional ExtraTrees baseline selected by training-set grouped cross-validation achieved ROC-AUC=0.693 in cross-validation and ROC-AUC=0.704, PR-AUC=0.702, F1=0.626, and MCC=0.304 on the held-out test set. The best initial protein-language-model fusion was traditional descriptors plus ESM2-8M embeddings, with grouped CV ROC-AUC=0.696 and test ROC-AUC=0.700. Advanced optimization using PCA95 dimensionality reduction and ExtraTrees feature-importance filtering yielded a practical ESM2-8M PCA95 top-300 model with test ROC-AUC=0.715 and PR-AUC=0.703, although repeated-CV gains remained modest. SaltyMeta is a transparent prioritization tool, not a sensory validation substitute. We provide benchmark, models, scripts, GitHub, and Streamlit for reproducible screening of food-derived peptides.

q-bio.BM↗