Report-Derived Labels, an Image Teacher and the Gap to Expert Labels in Knee MRI: An Analysis of One Entry to the RSNA 2026 Knee Abnormality Detection Challenge

Abstract

In the RSNA 2026 Knee Abnormality Detection challenge, 58 of 4,407 training studies have expert labels for 12 findings, and the rest have only a radiology report. We compare training targets built from a language-model reading of the reports and from a public image model against two references: the expert labels and the 39,248 study-finding cells that the reports state as present or absent. The report labels and the out-of-fold predictions of the image model reach similar agreement with the expert labels (macro AUC 0.892 and 0.896) but fail on different findings. The report is better for ligament tears and patellofemoral osteoarthritis, which it never calls present when the experts do not; the image model is better for effusion, Baker's cyst, contusion, and synovitis, where the reports call the finding absent, leave it out, or call it present more often than the experts. Their equal mixture reaches 0.927 (gain over the report labels 0.035, 95% interval 0.019 to 0.052), also after removing the studies whose knee side was handled inconsistently. We did not observe a model-level gain from this better target: a 2.5D ResNet50 trained on the report labels alone scored 0.877 on the private leaderboard, after the deadline, against 0.872 for the same model trained on the mixture, and the out-of-fold advantage of the mixture is confounded by a change of code and by possible leakage through the image model's folds. A scanner prevalence prior raised agreement with the reports but not with the experts, so a report-derived reference can reward reporting habits. A rank correction for co-occurring findings gave small out-of-fold gains and no visible leaderboard effect. Our entry, a forked public ensemble with that correction, was displayed at rank 582 of 6,299 on the private leaderboard (0.943, a score shared by 287 teams). All results are exploratory; none of the models is meant for clinical use.