Frank Samuelson | Search arXiv

arXiv2026

Evaluating the resolution of AI-based accelerated MR reconstruction using a deep learning-based model observer

Deep Learning-based Model Observers (DLMOs) were developed to evaluate a multi-coil sensitivity encoding parallel MRI at different acceleration factors on the Rayleigh discrimination task as a surrogate measure of resolution. Gaussian-convolved singlet and doublet signals with varying intensities and lengths were inserted into the white matter of synthetic brain images. K-space data were generated using a simulated MRI at acceleration factors of one (1x, fully sampled), 4.9x, and 16.4x, and reconstructed using a conventional root-sum-of-squares (rSOS) method and an AI-based U-Net method. DLMOs were first trained on fully sampled images and then fine-tuned for each acceleration factor using transfer learning. With a human-label alignment training strategy, the DLMOs achieved discrimination performance similar to that of trained human observers. Resolution was assessed using the area under the receiver operating characteristic curve (AUC), while PSNR and SSIM provided complementary task-agnostic comparisons. Although the U-Net method yielded significantly higher PSNR and SSIM than rSOS across different acceleration factors (p<0.05), task-based evaluation using the proposed DLMO showed inferior performance relative to fully sampled reconstruction. U-Net (4.9x) exhibited modest gains over rSOS (4.9x) for short signals (4-5 mm), but its AUC decreased by approximately 25% and 5% for 4 mm and 5 mm signals, respectively, compared with rSOS (1x). Similar declines were observed for U-Net (16.4x). These results demonstrate that AI-based accelerated MR reconstruction may improve visual appearance, but may not preserve task performance. The proposed DLMO approach may be employed to characterize the discriminative efficacy of AI-based undersampled MRI reconstruction.

physics.med-ph↗

arXiv2013

Properties of the Nonparametric Maximum Likelihood {ROC} Model with a Monotonic Likelihood Ratio

We expect that some observers in perceptual signal detection experiments, such as radiologists, will make rational decisions, and therefore ratings from those observers are expected to form a convex ROC curve. However, measured and published curves are often not convex. This article examines the convexity-constrained nonparametric maximum likelihood estimator of the ROC curve given by Lloyd (2002). Like Lloyd we use the Pool Adjacent Violator Algorithm (PAVA) to construct the estimate of the convex curve. We present a direct proof that this estimate is a convex hull of the empirical ROC curve. The estimate is simple to construct by hand, and follows the suggestions by Pesce, et~al.~(2010). We examine the properties of this constrained nonparametric maximum likelihood estimator (NPMLE) under a large number of experimental conditions. In particular we examine the behavior of the area under the curve which is often used as summary metric of diagnostic performance. This constrained ROC estimator gives an area under the curve (AUC) estimate that is biased high with respect to the usual empirical AUC estimate, but may be less biased with respect to the underlying continuous true AUC value. The constrained ROC estimator has lower variance than the usual empirical one. Unlike previous authors who used complex bootstrapping to estimate the variance of the constrained NPMLE we demonstrate that standard unbiased estimators of variance work well to estimate the variance of the NPMLE AUC.

stat.AP↗