Search arXivSearch

arXiv · 2501.14248

Normalization and selecting non-differentially expressed genes improve machine learning modelling of cross-platform transcriptomic data

Abstract

Normalization is a critical step in quantitative analyses of biological processes. Recent works show that cross-platform integration and normalization enable machine learning (ML) training on RNA microarray and RNA-seq data, but no independent datasets were used in their studies. Therefore, it is unclear how to improve ML modelling performance on independent RNA array and RNA-seq based datasets. Inspired by the house-keeping genes that are commonly used in experimental biology, this study tests the hypothesis that non-differentially expressed genes (NDEG) may improve normalization of transcriptomic data and subsequently cross-platform modelling performance of ML models. Microarray and RNA-seq datasets of the TCGA breast cancer were used as independent training and test datasets, respectively, to classify the molecular subtypes of breast cancer. NDEG (p>0.85) and differentially expressed genes (DEG, p<0.05) were selected based on the p values of ANOVA analysis and used for subsequent data normalization and classification, respectively. Models trained based on data from one platform were used for testing on the other platform. Our data show that NDEG and DEG gene selection could effectively improve the model classification performance. Normalization methods based on parametric statistical analysis were inferior to those based on nonparametric statistics. In this study, the LOG_QN and LOG_QNZ normalization methods combined with the neural network classification model seem to achieve better performance. Therefore, NDEG-based normalization appears useful for cross-platform testing on completely independent datasets. However, more studies are required to examine whether NDEG-based normalization can improve ML classification performance in other datasets and other omic data types.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Fei Deng, Catherine H Feng, Nan Gao, Lanjing Zhang. 2025-01-24. Normalization and selecting non-differentially expressed genes improve machine learning modelling of cross-platform transcriptomic data. https://arxiv.org/abs/2501.14248

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

High Reconstruction Quality and Restart Repeatability Do Not Guarantee Recovery of Ground-Truth Muscle Synergies

High reconstruction quality and agreement across repeated fits do not necessarily establish recovery of muscle synergies. We tested whether a variance-accounted-for (VAF)/elbow rule recovers the generating synergy count and spatial vectors, whether high restart repeatability indicates recovery, and how five design factors affect recovery. Non-negative matrix factorisation was applied to 4,320 synthetic 16-muscle datasets varying generating rank, noise, trial count, spatial similarity and activation overlap. Combined recovery required the correct rank and cosine similarity of at least 0.80 for every matched spatial vector. Factor effects and two-factor interactions were assessed using exploratory heteroscedastic Wald tests with Benjamini-Hochberg adjustment. Rank selection was exact in 17.6% of datasets, too low in 54.9% and too high in 27.5%; combined recovery was 13.9%. Among fits with VAF at least 0.90, only 11.3% achieved combined recovery. Among 3,762 datasets with spatial repeatability at least 0.95, 19.6% had the correct rank and 15.7% achieved combined recovery. All five factors were associated with recovery (adjusted p < 0.001). Recovery declined from 26.2% to 1.7% with increasing spatial similarity and from 26.2% to 2.2% with increasing activation overlap. It was lower at ranks 7-9 than at 3-5, increased from 11.0% with 3 trials to 15.8% with 80 trials, and varied non-monotonically with noise. Five noiseless signals synthesised from measured-sEMG reference factors also showed under-selection despite VAF above 0.918. Under this selector, high reconstruction quality and restart agreement were insufficient indicators of correct rank and spatial recovery. Muscle-synergy interpretation should account for rank sensitivity and the separability of spatial and activation patterns.

q-bio.QM

A hierarchical memory architecture overcomes context limits in long-horizon multi-agent computational modeling

Large language models (LLMs) demonstrate remarkable reasoning capabilities, yet their stateless architecture fundamentally limits deployment in long-horizon research workflows requiring multi-session continuity and quantitative rigor. Here we present Ensemble QSP, a multi-agent framework featuring a three-layer hierarchical memory architecture that bounds injected context (median 301 tokens, max 4,050) by capping state categories and evicting completed work. This enables continuous autonomous operation without context degradation. The system orchestrates five specialist worker agents under domain-expert principal investigators (PIs), enforcing physical constraints through physics-based checklists and structured domain knowledge. Comprehensive benchmarking demonstrates autonomous pharmacokinetic-pharmacodynamic (PKPD) model selection, improved parameter recovery relative to single-agent baselines, and robust interpretation of linguistically diverse prompts. Replication with open-weight models (DeepSeek-V4-Flash/Pro, Llama 3.1 70B) confirmed these architectural conclusions across PKPD modeling, literature synthesis, and PBPK model implementation, proving the framework is independent of proprietary LLMs. Feature-level ablations show that memory, retrieval, and PI oversight address distinct scientific failure modes, though underlying LLM capability remains consequential for stringent physical-consistency checks. The architecture is structurally agnostic to computational biology; adding a new scientific domain requires only a new PI-agent configuration.

q-bio.QM

Hierarchical Maximum Likelihood Estimation for Time-Resolved NMR Data

Metabolic monitoring and reaction rate estimation using hyperpolarized NMR technology requires accurate quantitative analysis of multidimensional data scenarios. Currently, this analysis is often performed in a two-stage procedure, which is prone to errors in uncertainty propagation and estimation. We propose an approach derived from a Bayesian hierarchical model that intrinsically propagates uncertainties and operates on the full data to maximize the precision at minimal uncertainty. In an analytic treatment, we reduce the estimation procedure to a least-squares optimization problem which can be understood as an extension of the Variable Projection (VarPro) approach for data scenarios with two predictors. We investigate the method's efficacy in two experiments with hyperpolarized metabolites recorded with conventional high-field NMR devices and a micronscale NMR setup using Nitrogen-Vacancy centers in diamond for detection, respectively. In both examples, the new approach improves estimates compared to Fourier methods and proves operational advantages over a two-stage procedure employing VarPro. While the approach presented is motivated by NMR analysis, it is straightforwardly applicable to further estimation scenarios with similar data structure, such as time-resolved photospectroscopy.

q-bio.QM