An adaptive subsampling method for large-sample feature screening

Abstract

We consider the sure independence screening (SIS) method, a standard feature screening approach that aims to eliminate non-informative features in ultrahigh-dimensional datasets. Although effective, SIS incurs a computational cost of order $O(np)$ for a predictor matrix of size $n\times p$, which can be prohibitively expensive when both n and p are considerable. Motivated by the multi-armed bandit (MAB) problem, we propose a more computationally efficient feature screening algorithm that reduces the cost to $O(\sqrt{n}p)$. The core idea is to progressively increase the subsample size and eliminate variables with small empirical marginal Pearson correlations, thereby avoiding unnecessary computation on unpromising features. We develop a new interpretable statistical theoretical analysis that characterizes how the subsample size affects screening accuracy, thereby revealing the balance between computational efficiency and statistical reliability. Moreover, we show that the proposed method retains the sure screening property under mild regularity conditions. Extensive numerical experiments on synthetic and real-world datasets show that BanditSIS achieves screening and prediction performance comparable to SIS while substantially reducing computational time. Our method offers a scalable and adaptive alternative to SIS, particularly well-suited for large-sample, high-dimensional applications where computational efficiency is critical.