Can Pre-trained Models Really Learn Better Molecular Representations for AI-aided Drug Discovery?

Ziqiao Zhang; Yatao Bian; Ailin Xie; Pengju Han; Long-Kai Huang; Shuigeng Zhou

Home/Archive/arXiv

arXiv · 2209.07423

Can Pre-trained Models Really Learn Better Molecular Representations for AI-aided Drug Discovery?

Ziqiao Zhang·Yatao Bian·Ailin Xie·Pengju Han·Long-Kai Huang·Shuigeng Zhou

View original source ↗Read source document ↗Export metadata ↓

Abstract

Self-supervised pre-training is gaining increasingly more popularity in AI-aided drug discovery, leading to more and more pre-trained models with the promise that they can extract better feature representations for molecules. Yet, the quality of learned representations have not been fully explored. In this work, inspired by the two phenomena of Activity Cliffs (ACs) and Scaffold Hopping (SH) in traditional Quantitative Structure-Activity Relationship (QSAR) analysis, we propose a method named Representation-Property Relationship Analysis (RePRA) to evaluate the quality of the representations extracted by the pre-trained model and visualize the relationship between the representations and properties. The concepts of ACs and SH are generalized from the structure-activity context to the representation-property context, and the underlying principles of RePRA are analyzed theoretically. Two scores are designed to measure the generalized ACs and SH detected by RePRA, and therefore the quality of representations can be evaluated. In experiments, representations of molecules from 10 target tasks generated by 7 pre-trained models are analyzed. The results indicate that the state-of-the-art pre-trained models can overcome some shortcomings of canonical Extended-Connectivity FingerPrints (ECFP), while the correlation between the basis of the representation space and specific molecular substructures are not explicit. Thus, some representations could be even worse than the canonical fingerprints. Our method enables researchers to evaluate the quality of molecular representations generated by their proposed self-supervised pre-trained models. And our findings can guide the community to develop better pre-training techniques to regularize the occurrence of ACs and SH.

Explore related subjects

q-bio.BM cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Ziqiao Zhang, Yatao Bian, Ailin Xie, Pengju Han, Long-Kai Huang, Shuigeng Zhou. 2022-08-21. Can Pre-trained Models Really Learn Better Molecular Representations for AI-aided Drug Discovery?. https://arxiv.org/abs/2209.07423

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

SaltyMeta: a curated benchmark and protein language model-informed web tool for salty peptide prediction

Excess sodium intake remains a major public health challenge, while salty and saltiness-enhancing peptides offer a potential route to preserve sensory saltiness in reduced-sodium foods. Machine-learning studies of salty peptides, however, are constrained by small datasets, heterogeneous evidence standards, uncertain negative labels, and sequence similarity leakage. Here we present SaltyMeta, a curated benchmark and web-accessible screening framework for salty or saltiness-enhancing short peptides. The benchmark contains 580 peptides, including 280 positive peptides and 300 negative peptides, all standardized to 2-15 residue one-letter amino-acid sequences. Quality control found no non-standard residues, exact duplicates, or positive-negative overlaps. A similarity-grouped split retained 456 peptides for training and 124 for held-out testing. We evaluated 548 interpretable peptide descriptors, frozen ESM2 embeddings at 8M, 35M, and 150M parameter scales, and descriptor-embedding fusion models under grouped cross-validation. The traditional ExtraTrees baseline selected by training-set grouped cross-validation achieved ROC-AUC=0.693 in cross-validation and ROC-AUC=0.704, PR-AUC=0.702, F1=0.626, and MCC=0.304 on the held-out test set. The best initial protein-language-model fusion was traditional descriptors plus ESM2-8M embeddings, with grouped CV ROC-AUC=0.696 and test ROC-AUC=0.700. Advanced optimization using PCA95 dimensionality reduction and ExtraTrees feature-importance filtering yielded a practical ESM2-8M PCA95 top-300 model with test ROC-AUC=0.715 and PR-AUC=0.703, although repeated-CV gains remained modest. SaltyMeta is a transparent prioritization tool, not a sensory validation substitute. We provide benchmark, models, scripts, GitHub, and Streamlit for reproducible screening of food-derived peptides.

q-bio.BM↗

arXiv2026

Synthesizing State-of-the-Art Structure Predictions from Soup of Co-folding Models

Co-folding models have advanced rapidly, yet no single model consistently performs best across all biomolecular complexes. This raises the question of whether independently trained co-folding models encode complementary information that can be transferred across co-folding models. We introduce SoupFold, which improves co-folding predictions by learning simple mappings between the representation spaces of co-folding models. At inference time, SoupFold transfers and incorporates representations from other co-folding models to update the representation used for structure prediction. Importantly, this does not re-train the co-folding models. We evaluate SoupFold on protein-protein and protein-ligand prediction tasks of FoldBench using AlphaFold3, Protenix, ESMFold2, and OpenDDE. By combining their representations, SoupFold achieves state-of-the-art performance on both protein-protein and protein-ligand structure prediction, showing that independently trained co-folding models encode complementary information that can be effectively transferred across models.

q-bio.BM↗

arXiv2026

Exploring Optimal Parameters for Ligand-Based Virtual Screening in Early Drug Discovery

Ligand-based virtual screening depends on choices that are often treated as implementation details, including the similarity threshold, fingerprint setting and atom-invariant scheme. We examined how these choices altered the composition of ranked searches against the Enamine library for four aminergic reference ligands: atomoxetine, bupropion, mirtazapine and venlafaxine. Candidate sets were evaluated by compound-weighted scaffold novelty, normalized Shannon scaffold diversity and three computational estimates of synthetic accessibility. We first identified ligand-specific operating points along cumulative Tanimoto-ranked searches. We then compared five extended-connectivity fingerprint settings at matched retrieval depths and compared ECFP4 with the feature-class analogue FCSFP4. Finally, 240 records, corresponding to 229 unique structures, were scored independently by three chemists who were blinded to the computational scores. No single Tanimoto cutoff described all four searches. ECFP8 provided the most stable pooled setting, although venlafaxine favored ECFP2. ECFP4 was the stronger primary fingerprint in pooled comparisons, whereas FCSFP4 contributed nonredundant chemical space. Agreement among individual chemists was moderate, and the mean rating was more reliable than a single rating. SCScore showed the highest association with the blinded consensus, but performance varied by ligand. These results support a staged screening design in which a pooled default is followed by ligand-specific calibration and expert review.

q-bio.BM↗