Diffusion Sequence Models for Enhanced Protein Representation and Generation

Logan Hallee; Nikolaos Rafailidis; David B. Bichara; Jason P. Gleghorn

Home/Archive/arXiv

arXiv · 2506.08293

Diffusion Sequence Models for Enhanced Protein Representation and Generation

Logan Hallee·Nikolaos Rafailidis·David B. Bichara·Jason P. Gleghorn

View original source ↗Read source document ↗Export metadata ↓

Abstract

Proteins are fundamental to biology, executing diverse functions through complex physicochemical interactions, and they hold transformative potential across medicine, materials science, and environmental applications. Protein Language Models (pLMs) aim to unlock insights from the vast space of unlabeled protein sequences by learning rich, semantic representations from primary sequences via masked language modeling. However, these models typically exhibit limited generative capacity. In this work, we introduce the Diffusion Sequence Model (DSM), a novel pLM trained with masked diffusion to enable both high-quality representation learning and generative protein design. DSM builds upon the ESM2 architecture by incorporating a masked forward diffusion process inspired by the LLaDA framework. After training, DSM is capable of generating diverse, biomimetic sequences that align with expected amino acid compositions, secondary structures, and predicted functions, even with 90\% token corruption. Furthermore, DSM's learned representations match or exceed those of similarly sized pLMs on downstream tasks. We also introduce DSM(ppi), a variant fine-tuned to generate protein binders by attending to target sequences. We demonstrate DSM(ppi)'s effectiveness on the challenging Bench-tested Binder Benchmark (BenchBB), where both DSM and DSM(ppi) produce candidates with superior predicted binding affinity compared to known binders. Our results establish masked diffusion as a powerful paradigm for unifying protein representation and generation in a single framework.

Explore related subjects

q-bio.BM

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Logan Hallee, Nikolaos Rafailidis, David B. Bichara, Jason P. Gleghorn. 2025-06-09. Diffusion Sequence Models for Enhanced Protein Representation and Generation. https://arxiv.org/abs/2506.08293

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Decoding enzyme-substrate interaction topology reveals principles underlying catalytic efficiency and mutational outcomes

The enzyme turnover number (kcat) defines catalytic efficiency and constrains quantitative models of metabolism, yet the molecular determinants governing kcat and its response to mutation remain poorly understood. Measurements are sparse and labor-intensive, and most computational approaches provide numerical predictions without explaining how enzyme-substrate interactions shape catalytic outcomes. A central challenge is therefore to identify the topological principles that determine where mutations act and how their functional outcomes are encoded within the enzyme-substrate interaction network. Here, we show that catalytic efficiency and mutational effects can be interpreted through enzyme-substrate interaction topology. We developed Interkcat, an interpretable bidirectional cross-attention framework that captures reciprocal coordination between protein residues and substrate atoms. Optimized on a unified benchmark, Interkcat achieves state-of-the-art predictive performance (R2 = 0.701). From its learned representations, we derive an Interaction Topology Score (ITS) that identifies sequence regions statistically enriched for mutation-sensitive sites without explicit structural inputs. We further demonstrate that higher-order topological features distinguish opposing mutational outcomes: lethal mutations disrupt coordinated networks, whereas activity-preserving or enhancing mutations retain sparse, globally organized coupling. These findings establish interaction topology as a unifying principle linking enzyme sequence, catalytic efficiency, and evolutionary perturbation.

q-bio.BM↗

arXiv2026

Adapting Boltz-2 with limited experimental activity data improves early enrichment in virtual screening

Virtual screening aims to prioritize active compounds from large chemical libraries within a limited experimental budget. When applying Boltz-2 to virtual screening, a key challenge is how to use limited experimental data from the target assay to improve the prioritization of active compounds. We investigated whether fine-tuning the Boltz-2 affinity heads with a small number of binary activity labels could improve early enrichment of active compounds in hit discovery. We compared fine-tuning with 40-300 labels in a retrospective evaluation on eight MF-PCBA targets. With 300 activity measurements, fine-tuning increased the number of actives in the top 1% by a geometric mean of 1.77-fold across the eight targets and improved average precision (AP) by 2.14-fold relative to the control without fine-tuning. We also investigated whether rescoring a subset of candidates could retain the improvement in hit recovery by reranking only the top-ranked Boltz-2 candidates with the fine-tuned head. Restricting rescoring to approximately 10% of the evaluation set retained hit recovery comparable to full rescoring. These findings show that affinity-head fine-tuning with limited activity labels improves early enrichment with Boltz-2 and that this benefit can be retained when rescoring a restricted set of candidates.

q-bio.BM↗

arXiv2026

SaltyMeta: a curated benchmark and protein language model-informed web tool for salty peptide prediction

Excess sodium intake remains a major public health challenge, while salty and saltiness-enhancing peptides offer a potential route to preserve sensory saltiness in reduced-sodium foods. Machine-learning studies of salty peptides, however, are constrained by small datasets, heterogeneous evidence standards, uncertain negative labels, and sequence similarity leakage. Here we present SaltyMeta, a curated benchmark and web-accessible screening framework for salty or saltiness-enhancing short peptides. The benchmark contains 580 peptides, including 280 positive peptides and 300 negative peptides, all standardized to 2-15 residue one-letter amino-acid sequences. Quality control found no non-standard residues, exact duplicates, or positive-negative overlaps. A similarity-grouped split retained 456 peptides for training and 124 for held-out testing. We evaluated 548 interpretable peptide descriptors, frozen ESM2 embeddings at 8M, 35M, and 150M parameter scales, and descriptor-embedding fusion models under grouped cross-validation. The traditional ExtraTrees baseline selected by training-set grouped cross-validation achieved ROC-AUC=0.693 in cross-validation and ROC-AUC=0.704, PR-AUC=0.702, F1=0.626, and MCC=0.304 on the held-out test set. The best initial protein-language-model fusion was traditional descriptors plus ESM2-8M embeddings, with grouped CV ROC-AUC=0.696 and test ROC-AUC=0.700. Advanced optimization using PCA95 dimensionality reduction and ExtraTrees feature-importance filtering yielded a practical ESM2-8M PCA95 top-300 model with test ROC-AUC=0.715 and PR-AUC=0.703, although repeated-CV gains remained modest. SaltyMeta is a transparent prioritization tool, not a sensory validation substitute. We provide benchmark, models, scripts, GitHub, and Streamlit for reproducible screening of food-derived peptides.

q-bio.BM↗