NGSE-Corr: A technique for objective clinical evaluation of quantitative-imaging methods without a gold standard

Abstract

Objective evaluation of quantitative-imaging (QI) methods based on how reliably they measure true values is important for clinical translation. Performing such evaluation with patient data is highly desirable but hindered by the lack of gold standards. To address this challenge, advancing on previous studies, we propose a no-gold-standard evaluation technique, NGSE-Corr, that objectively evaluates QI methods without true values. The technique assumes a linear stochastic relationship between true and measured values, characterized by a slope, bias, and multivariate Gaussian-distributed noise term that models correlated noise across QI methods. We derive a maximum-likelihood approach to estimate these parameters using only measured values. From the estimates, we compute noise-to-slope ratio (NSR) to rank QI methods based on precision. Numerical experiments showed that NGSE-Corr reliably estimated the NSR, accurately ranked methods, and maintained performance even when assumptions made by the technique were partially violated. We also validated NGSE-Corr in an in silico imaging trial to rank three quantitative SPECT methods for measuring regional activity uptake in patients with bone metastatic castrate-resistant prostate cancer treated with radium-223. NGSE-Corr correctly identified the most precise QI method and ranked the methods for 95% (95% CI, 89%-98%) and 91% (95% CI, 84%-95%) of trials, respectively, with data from 50 patients. Performance further improved with larger cohorts. With 200 patients, NGSE-Corr yielded same rankings as those obtained with true values across all trial instances. These findings demonstrate the ability of NGSE-Corr to accurately rank QI methods without gold standards and motivate clinical validation and broader applications.