Input Representation, Not Model Size or Pretraining Scale, Shapes Speaker Verification Robustness to Temporal Degradation

Abstract

Speech remains intelligible to listeners under heavy temporal degradation, but whether speaker identity survives is largely unexplored. We evaluate six automatic speaker verification (ASV) models, trained on temporally intact speech, and tested under three degradations that remove different combinations of temporal cues. The models form three pairs, one per architecture family (convolutional, transformer, conformer), each pairing a waveform-input with a mel filterbank-input model. All models degrade, but robustness tracks input representation rather than pretraining scale or model size. In all nine family-by-condition comparisons the filterbank-input model degrades less, with eight of nine 95% confidence intervals excluding zero, even when it is smaller or has less pretraining than its waveform counterpart. In the controlled convolutional pair, where the models differ only in input representation, the filterbank-input model's equal error rate rises from 1.1% to 17.1% under the heaviest tested degradation, against 2.9% to29.7% for its waveform counterpart.