Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2410.20667

PepDoRA: A Unified Peptide Language Model via Weight-Decomposed Low-Rank Adaptation

Leyao Wang·Rishab Pulugurta·Pranay Vure·Yinuo Zhang·Aastha Pal·Pranam Chatterjee
View original source ↗Read source document ↗Export metadata ↓

Abstract

Peptide therapeutics, including macrocycles, peptide inhibitors, and bioactive linear peptides, play a crucial role in therapeutic development due to their unique physicochemical properties. However, predicting these properties remains challenging. While structure-based models primarily focus on local interactions, language models are capable of capturing global therapeutic properties of both modified and linear peptides. Protein language models like ESM-2, though effective for natural peptides, cannot however encode chemical modifications. Conversely, pre-trained chemical language models excel in representing small molecule properties but are not optimized for peptides. To bridge this gap, we introduce PepDoRA, a unified peptide representation model. Leveraging Weight-Decomposed Low-Rank Adaptation (DoRA), PepDoRA efficiently fine-tunes the ChemBERTa-77M-MLM on a masked language model objective to generate optimized embeddings for downstream property prediction tasks involving both modified and unmodified peptides. By tuning on a diverse and experimentally valid set of 100,000 modified, bioactive, and binding peptides, we show that PepDoRA embeddings capture functional properties of input peptides, enabling the accurate prediction of membrane permeability, non-fouling and hemolysis propensity, and via contrastive learning, target protein-specific binding. Overall, by providing a unified representation for chemically and biologically diverse peptides, PepDoRA serves as a versatile tool for function and activity prediction, facilitating the development of peptide therapeutics across a broad spectrum of applications.

Explore related subjects

q-bio.BM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Leyao Wang, Rishab Pulugurta, Pranay Vure, Yinuo Zhang, Aastha Pal, Pranam Chatterjee. 2024-10-28. PepDoRA: A Unified Peptide Language Model via Weight-Decomposed Low-Rank Adaptation. https://arxiv.org/abs/2410.20667

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

SaltyMeta: a curated benchmark and protein language model-informed web tool for salty peptide prediction

Excess sodium intake remains a major public health challenge, while salty and saltiness-enhancing peptides offer a potential route to preserve sensory saltiness in reduced-sodium foods. Machine-learning studies of salty peptides, however, are constrained by small datasets, heterogeneous evidence standards, uncertain negative labels, and sequence similarity leakage. Here we present SaltyMeta, a curated benchmark and web-accessible screening framework for salty or saltiness-enhancing short peptides. The benchmark contains 580 peptides, including 280 positive peptides and 300 negative peptides, all standardized to 2-15 residue one-letter amino-acid sequences. Quality control found no non-standard residues, exact duplicates, or positive-negative overlaps. A similarity-grouped split retained 456 peptides for training and 124 for held-out testing. We evaluated 548 interpretable peptide descriptors, frozen ESM2 embeddings at 8M, 35M, and 150M parameter scales, and descriptor-embedding fusion models under grouped cross-validation. The traditional ExtraTrees baseline selected by training-set grouped cross-validation achieved ROC-AUC=0.693 in cross-validation and ROC-AUC=0.704, PR-AUC=0.702, F1=0.626, and MCC=0.304 on the held-out test set. The best initial protein-language-model fusion was traditional descriptors plus ESM2-8M embeddings, with grouped CV ROC-AUC=0.696 and test ROC-AUC=0.700. Advanced optimization using PCA95 dimensionality reduction and ExtraTrees feature-importance filtering yielded a practical ESM2-8M PCA95 top-300 model with test ROC-AUC=0.715 and PR-AUC=0.703, although repeated-CV gains remained modest. SaltyMeta is a transparent prioritization tool, not a sensory validation substitute. We provide benchmark, models, scripts, GitHub, and Streamlit for reproducible screening of food-derived peptides.

q-bio.BM↗
arXiv2026

Synthesizing State-of-the-Art Structure Predictions from Soup of Co-folding Models

Co-folding models have advanced rapidly, yet no single model consistently performs best across all biomolecular complexes. This raises the question of whether independently trained co-folding models encode complementary information that can be transferred across co-folding models. We introduce SoupFold, which improves co-folding predictions by learning simple mappings between the representation spaces of co-folding models. At inference time, SoupFold transfers and incorporates representations from other co-folding models to update the representation used for structure prediction. Importantly, this does not re-train the co-folding models. We evaluate SoupFold on protein-protein and protein-ligand prediction tasks of FoldBench using AlphaFold3, Protenix, ESMFold2, and OpenDDE. By combining their representations, SoupFold achieves state-of-the-art performance on both protein-protein and protein-ligand structure prediction, showing that independently trained co-folding models encode complementary information that can be effectively transferred across models.

q-bio.BM↗
arXiv2026

Exploring Optimal Parameters for Ligand-Based Virtual Screening in Early Drug Discovery

Ligand-based virtual screening depends on choices that are often treated as implementation details, including the similarity threshold, fingerprint setting and atom-invariant scheme. We examined how these choices altered the composition of ranked searches against the Enamine library for four aminergic reference ligands: atomoxetine, bupropion, mirtazapine and venlafaxine. Candidate sets were evaluated by compound-weighted scaffold novelty, normalized Shannon scaffold diversity and three computational estimates of synthetic accessibility. We first identified ligand-specific operating points along cumulative Tanimoto-ranked searches. We then compared five extended-connectivity fingerprint settings at matched retrieval depths and compared ECFP4 with the feature-class analogue FCSFP4. Finally, 240 records, corresponding to 229 unique structures, were scored independently by three chemists who were blinded to the computational scores. No single Tanimoto cutoff described all four searches. ECFP8 provided the most stable pooled setting, although venlafaxine favored ECFP2. ECFP4 was the stronger primary fingerprint in pooled comparisons, whereas FCSFP4 contributed nonredundant chemical space. Agreement among individual chemists was moderate, and the mean rating was more reliable than a single rating. SCScore showed the highest association with the blinded consensus, but performance varied by ligand. These results support a staged screening design in which a pooled default is followed by ligand-specific calibration and expert review.

q-bio.BM↗