Masoumeh Javanbakhat^{1}, Ludger Starke^{2}, Sonia Waiczies^{2}, and Christoph Lippert^{1}

^{1}Digital Health-Machine Learning Group, Hasso Plattner Institute, Potsdam, Germany, ^{2}Berlin Ultrahigh Field Facility, Max DelbrÃ¼ck Centre for Moleculare Medicine in the Helmholtz Association, Berlin, Germany

Deep learning (DL) has achieved state of the art results in semantic segmentation of numerous medical imaging applications. Despite promising results deep learning models tend to produce point estimates as outputs which leads to overconfident, miscalibrated predictions. These overconfident predictions are specifically problematic in medical applications. Hence, providing a measure of a system’s confidence to identify untrustworthy predictions is essential to guide clinical decisions. Here we propose a 3D Bayesian segmentation model to provide uncertainty estimation for the Fluorine-19 MRI dataset based on Stochastic Gradient Markov Chain Monte Carlo methods.

$$$P(y|x,D)= \int P(y|x,\theta)~ P(\theta|D)~d\theta\approx \frac{1}{s}\Sigma_{i=1}^{s} p(y|x, \theta_{i}).$$$

The uncertainty in predictions are then quantified by measuring either the variance or the entropy of the predictive distribution

$$$H(y|x,D) = -\Sigma_{c\in C} p(y=c|x,D) \log p(y=c|x,D)$$$

Where $$c$$ ranges over all classes. We evaluate the quality of estimated uncertainties with respect to their reliability and their benefit to correct the failure predictions in terms of 3 metrics: Calibration

1. Starke L, et al. in Preclinical MRI of the Kidney: Methods and Protocols, A Pohlmann, et al. Editors. 2021, Springer US: New York, NY. 711-722.

2. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI 2015.

3. Ma Y. A, et al. A complete recipe for stochastic gradient MCMC. NIPS 2015.

4. Nair T, Precup D, Douglas L. A, Arbel T. Exploring uncertainty measures in deep networks for Multiple sclerosis lesion detection and segmentation. Medical Image Analysis, Volume 59, 2020.

5. Jungo A, et al. Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentation. MICCAI 2019.

6. Starke L, et al.Magn Reson Med.2019. 1-17.

Figure 1: Predictions and uncertainties of 19F MRI segmentation using SG-MCMC methods. (A) Input image: a 2D slice of a 3D volume in test set, (B) Ground truth: mask of reference 19F MR images, (C) Prediction: predicted mask of 19F MR images, (D) Uncertainty map: voxel-wise uncertainties in order to determine the reliability of the predictions.

Figure 2: Evaluation of estimated uncertainties. (A) Uncertainty-error-overlap: Dice score between error map and uncertainty map, (B) Corrections: effect of removal uncertain voxels on improving Dice score, (C) ROC curve: effect of changing uncertainty thresholds on TPR and FDR.

Figure 3: Calibration of estimated uncertainties. (A) Reliability diagram on dataset level, (B) Reliability diagram of subject 2, (C) Reliability diagram of subject 4, (D) Reliability diagram of subject 6.

Table 1: Segmentation performance

DOI: https://doi.org/10.58530/2022/1793