HistoFID- Calibrating Frechet-distance evaluation across pathology foundation models

Abstract

The Frechet Inception Distance (FID) compares two image sets by fitting a Gaussian to the features of a fixed network and measuring the distance between the two Gaussians. In digital pathology the Inception network is routinely replaced by a histology foundation model, on the assumption that a domain encoder gives a more meaningful score. We show that this choice changes the result. For one fixed pair of tile sets, the raw Frechet distance varies about thirty-fold across six common encoders, and the ordering does not follow embedding dimension, so a raw score cannot be read without naming the encoder. Using a held-out in-house cohort (about 500,000 H&E and immunohistochemistry tiles from 2,119 slides) and a public TCGA BRCA cohort (100 slides), we benchmark Inception-v3, Phikon-v2, CONCH, UNI2-h, Virchow2 and Prov-GigaPath across within-cohort baselines, cross-cohort drift, controlled perturbations, compression, stain normalization, and two generative models. Expressing each distance as a ratio to the encoder's own within-cohort floor restores comparability, cutting the across-encoder coefficient of variation by about 89% within cohort and 58% across cohorts. The encoders separate into a sensitive group (CONCH, Phikon-v2, Inception-v3) and an invariant group (UNI2-h, Virchow2, Prov-GigaPath), and this split decides which generative model is judged more realistic, so the encoder can change the conclusion of a generative evaluation. At the slide level, an attention-pooling encoder registers per-slide composition that a pooled patch distance cannot see, raising the distance about 320-fold on matched cohorts. Using the same protocol we evaluate TuroCompress, a proprietary pathology codec, which reaches the highest reconstruction fidelity at the smallest file size among codecs tested. We release the normalization protocol, the per-encoder perturbation panel, and the feature extracts.