Moderately Mighty: To What Extent Can Internal Software Metrics Predict App Popularity at Launch?

Abstract

Predicting a mobile app's popularity before its first release can provide developers with a strategic advantage in a competitive marketplace, yet it remains a challenging problem. This study explores the extent to which internal software metrics, measurable from source code before deployment, can predict an app's popularity (i.e., ratings and downloads per year) at inception. For our analysis, we constructed a rigorously filtered dataset of 446 open-source Java-based Android apps that are available on both F-Droid and Google Play Store. Using app source code from F-Droid, we extracted a wide array of internal metrics, including system-, class-, and method-level code metrics, code smells, and app metadata. Popularity-related information, including reviews and download counts, was collected from the Play Store. We evaluate regression and classification models across three feature sets: a minimal Size-only baseline, a domain-informed Handpicked set, and a Voting set derived via feature selection algorithms. Our results show that, for both app ratings and number of downloads, regression models perform poorly due to skewed rating distributions and a highly scattered range of download counts in our dataset. However, when reframed as a binary classification (Popular vs. Unpopular), performance improves significantly-the best model, a Multilayer Perceptron, achieves an F1-score of 0.72. We conclude that, although internal code metrics alone are insufficient for accurately predicting an app's future popularity, they do exhibit meaningful correlations with it. Thus, our findings challenge prior studies that have entirely dismissed internal metrics as valid indicators of software quality. Instead, our results align with research suggesting that internal code metrics can be valuable when evaluated within the appropriate context-specifically, we found them useful for classification tasks.