Semantic burstiness across principal directions of textual variation

Abstract

Long-range correlations in written texts extend over scales far beyond local syntax, but their relation to semantic organisation remains incompletely understood. Previous work has linked these correlations to the bursty recurrence of topically relevant words. Here we generalise that construction from individual lexical items to distributed semantic observables. We map a text token by token onto a trajectory of pre-trained word embeddings, use principal component analysis (PCA) to identify text-specific directions of semantic variation, and threshold the projected series to define inter-event times; their coefficients of variation form a component-resolved burstiness spectrum $\{B_k\}$. Across 28 literary and scientific texts and the first 20 components, the originals remain systematically above the geometric inter-event-time baseline and above two complementary controls: a word-shuffled null, which preserves the embedding covariance, the PCA eigensystem and the marginal projected distributions while destroying temporal order, and a fractional-Gaussian-noise surrogate, which preserves the Zipfian frequency structure and matches the detrended fluctuation analysis (DFA) exponent of the original rank sequence. With events defined by the upper 5% tail of each projection, the original exceeds both controls in every text-component pair, while the two controls are statistically indistinguishable from each other. Autocorrelation and detrended fluctuation analysis of the semantic projections separate the originals from both controls in the same way. Global rank-level long-range correlations are therefore insufficient to generate the bursty recurrence observed along semantic directions. The results instead support an interpretation in terms of the ordered recurrence of semantically related material.