Tangling Pull Requests: Curating a Commit Untangling Dataset from Merged PRs

Abstract

Composite commits (CC), in which multiple unrelated changes are bundled into a single commit, are frequent in software development and significantly hinder code comprehension and maintenance. Although machine learning-based methods have been developed to ``untangle'' such commits into smaller, coherent change sets, these methods require large-scale training data with correct untangling labels. Preparing such datasets is costly and typically requires expert labelling. In this study, we propose a scalable and cost-effective method for dataset construction by leveraging commits extracted from open-source repositories' pull requests (PRs). We empirically validated our dataset and found that when applying our filtering rules, PRs that, when viewed as a single commit, are tangled, yet each individual commit on the feature branch is atomic (ideal PRs), increased from 9.5% to 55%. This composite commits dataset is more than 5.7 times larger than previous heuristic-based datasets. Using our new dataset, we find that the PR-based dataset differs statistically from previous datasets directly constructed using Herzig's proposed heuristics even after accounting for our proposed rules that may alter CC or STS sizes. When constructing datasets using the previous heuristics, they differ statistically along dimensions that impact the confidence voters and are likely to impact learning-based approaches. We validate the impact on the original Herzig \etal method, which used confidence voters across our dataset. To show that our approach extends to other languages, we also create a Python dataset which we empirically validate, finding comparable rates for ideal PRs (56.5%).