Machine Learning Pipeline for Software Engineering: A Systematic Literature Review

Abstract

The rapid advancement of software development practices has introduced challenges in ensuring quality and efficiency across the software engineering (SE) lifecycle. As SE systems grow in complexity, traditional approaches often fail to scale, resulting in longer debugging times, inefficient defect detection, and resource-heavy development cycles. Machine Learning (ML) has emerged as a key solution, enabling automation in tasks such as defect prediction, code review, and release quality estimation. However, the effectiveness of ML in SE depends on the robustness of its pipeline, including data collection, preprocessing, feature engineering, algorithm selection, validation, and evaluation. This systematic literature review (SLR) examines state-of-the-art ML pipelines designed for SE, consolidating best practices, challenges, and gaps. Our findings show that robust preprocessing, such as SMOTE for data balancing and SZZ-based algorithms for feature selection, improves model reliability. Ensemble methods like Random Forest and Gradient Boosting dominate performance across tasks, while simpler models such as Naive Bayes remain valuable for efficiency and interpretability. Evaluation metrics including AUC, F1-score, and precision are most common, with new metrics like Best Arithmetic Mean (BAM) emerging in niche applications. Validation techniques such as bootstrapping are widely used to ensure model stability and generalizability. This SLR highlights the importance of well-designed ML pipelines for addressing SE challenges and provides actionable insights for researchers and practitioners seeking to optimize software quality and efficiency. By identifying gaps and trends, this study sets a foundation for advancing ML adoption and fostering innovation in increasingly complex development environments.