The power of A/B testing under interference

James D. Wilson; David T. Uminsky

Home/Archive/arXiv

arXiv · 1710.03855

The power of A/B testing under interference

James D. Wilson·David T. Uminsky

View original source ↗Read source document ↗Export metadata ↓

Abstract

In this paper, we address the fundamental statistical question: how can you assess the power of an A/B test when the units in the study are exposed to interference? This question is germane to many scientific and industrial practitioners that rely on A/B testing in environments where control over interference is limited. We begin by proving that interference has a measurable effect on its sensitivity, or power. We quantify the power of an A/B test of equality of means as a function of the number of exposed individuals under any interference mechanism. We further derive a central limit theorem for the number of exposed individuals under a simple Bernoulli switching interference mechanism. Based on these results, we develop a strategy to estimate the power of an A/B test when actors experience interference according to an observed network model. We demonstrate how to leverage this theory to estimate the power of an A/B test on units sharing any network relationship, and highlight the utility of our method on two applications - a Facebook friendship network as well as a large Twitter follower network. These results yield, for the first time, the capacity to understand how to design an A/B test to detect, with a specified confidence, a fixed measurable treatment effect when the A/B test is conducted under interference driven by networks.

Explore related subjects

cs.SI

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

James D. Wilson, David T. Uminsky. 2017-10-10. The power of A/B testing under interference. https://arxiv.org/abs/1710.03855

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

User Influence Analysis Based on Blogs

Rumor and word of mouth spread at the same speed as the highway of information diffusion in the age of the internet. Social networks play quite an important role in the huge internet. Nowadays, social networks have become indispensable in our lives, especially for the government and enterprises. A social network becomes a complex information diffusion network with users working as nodes and the relationships between users working as the vehicle. In this paper, we propose three kinds of algorithms for computing user influence based on the behavior of a user's forwarding microblogs and the symbol of @ in microblogs. We evaluate the effectiveness of the algorithms by comparing the results of our work with the training data in the dataset, and in the end, it proves that our algorithms work well.

cs.SI↗

arXiv2026

Location transparency reduces activity by accounts misrepresenting their location on X

Concerns about inauthentic accounts, including foreign actors posing as domestic voices, are central to debates about online discourse. Yet, little is known about accounts with inaccurate location claims and how they behave when discrepancies between their claimed and actual locations become publicly visible. In November 2025, X introduced an "About this account" feature that discloses each account's platform-inferred location of operation. We leverage this intervention in a large-scale quasi-experimental study of 8,200 politically engaged accounts claiming a U.S. location, comparing accounts whose disclosed locations matched versus contradicted their claims across 1.3 million posts and 3.6 million replies over 21 weeks. Before disclosure, location-mismatched accounts posted more misleading, scam-related, and cryptocurrency-related content, but showed no distinctive partisan leaning. Difference-in-differences estimates show that disclosure reduced the posting activity of location-mismatched accounts by 13.1% with the largest declines among accounts revealed to be in Africa (29.2%) and Asia (24.4%), and among accounts with VPN flags, username changes, or scam- and crypto-heavy content. Additionally, the decline in their replies was concentrated in interactions with U.S.-based recipients (10.3%), whereas replies to non-U.S.-based recipients showed no statistically significant change. Conversely, there was no significant change in average audience engagement with their posts. Location transparency thus works primarily by inducing restraint among the disclosed accounts rather than by shifting audience behaviour, and the accounts it constrains look at least as much like cross-border fraud as foreign political influence.

cs.SI↗

arXiv2026

Diffusion-Induced Spatial Attention Overlapping Community Detection

Detection of overlapping communities is essential for modelling networks in which nodes participate simultaneously in multiple structural or functional groups. Existing graph neural network approaches commonly rely on local message passing, which can obscure community boundaries through smoothing and limit the representation of structurally relevant long-range dependencies. We introduce Diffusion-Induced Spatial Attention Community Detection (DISCO), a deep-learning framework that combines a structural prior derived from influence spreading dynamics, sparse multi-head attention, and non-negative community-affiliation learning. The prior identifies candidate interactions beyond immediate graph neighbours and biases attention according to their structural proximity, while a Bernoulli-Poisson edge-reconstruction objective enables overlapping community inference from node attributes and structural profiles, or both. Benchmark experiments show that DISCO performs competitively against established graph convolutional and graph attention approaches across different input configurations. To demonstrate its practical applicability, we present a proof-of-concept cybersecurity use case in which changes between community assignments inferred from consecutive communication-network snapshots provide an interpretable anomaly signal. Temporal community similarity identifies structural deviations, while node-level contributions help locate the devices associated with them. DISCO therefore provides both a flexible method for overlapping community detection and a foundation for analysing structural change in dynamic networks.

cs.SI↗