1095

Deep learning-based uncertainty estimation: application in PET/MRI joint estimation of attenuation and activity for patients with metal implants
Andrew P. Leynes1,2, Sangtae Ahn3, Kristen Wangerin3, Florian Wiesinger4, Thomas A. Hope1, and Peder E.Z. Larson1

1University of California San Francisco, San Francisco, CA, United States, 2UC Berkeley - UC San Francisco Joint Graduate Program in Bioengineering, Berkeley and San Francisco, CA, United States, 3GE Global Research, Niskayuna, NY, United States, 4GE Global Research, Munich, Germany

Synopsis

PET image reconstruction requires accurate estimates of attenuation coefficients. Metal implants corrupt both the MRI and CT images and thus are not suitable for use in image reconstruction. In particular, the metal implant appears as a large signal void in the MRI and is incorrectly estimated as having the attenuation coefficients of air. We proposed to use Bayesian deep learning to identify the location of the metal implant and use it to guide PET joint estimation of attenuation and activity. We found that the metal implant is recovered and lesion uptake near the implant agree well with our reference data.

Introduction

Positron emission tomography (PET) image reconstruction requires estimating the attenuation of high energy photons as they travel through the body. This is typically done with CT scans, but these are not available on PET/MRI scanners.

Joint estimation of attenuation and activity is an algorithm to simultaneously estimate the attenuation map and PET image using only the PET emission data1. This produces an attenuation map however the solution is slow to converge, has low signal-to-noise, and suffers from cross-talk artifacts.

Deep learning has been successfully utilized to generate synthetic CT images for PET/MRI attenuation correction2. However, the synthetic CT images do not account for any bowel air in the body and are unable to account for metal implants. One potential way of identifying these regions is with the use of Bayesian deep learning. With Bayesian deep learning, data that were not present in the training data would have some uncertainty assigned to them: the larger the deviation the data from the training data, the larger the uncertainty.

In this work, we introduced a method to estimate regions of confidence in the synthetic CT image. The confidence map guided the joint estimation algorithm to estimate the attenuation and activity using the PET emission data and an initial attenuation map from MRI. To the best of our knowledge, this is the first work to integrate PET and MRI data in a fully automatic joint reconstruction framework.

Methodology

An overview of the network architecture and reconstruction pipeline is shown in Figure 1. A Monte-Carlo Dropout Bayesian convolutional neural network (BCNN)3 was trained to generate synthetic CT images using Dixon in-phase, Dixon fractional fat, and Dixon fractional water MRI from a population of 10 patients. For forward inference, the BCNN was used to probabilistically generate 283 synthetic CT images for each patient. The mean and variance images were calculated from the set; the mean image was used as an initial attenuation map and the variance image was used to generate the confidence map. PET reconstruction was then performed using the joint estimation of attenuation and activity algorithm1.

We reconstructed PET images for one patient with a metal implant using three different methods: (1) joint estimation with the BCNN synthetic CT and confidence map (JE), (2) joint estimation without an MR prior (JE w/o prior), and (3) without joint estimation using the BCNN synthetic CT (DL-based pCT). We classified lesions as near the metal implant and far from the metal implant and measured the maximum standardized uptake value in each category. We used JE w/o prior as a reference for lesions near the metal implant since the attenuation from the metal dominates, and we used DL-based pCT as a reference for lesions far from the metal implant since the metal implant would have less effect.

Results

Figure 2 shows the confidence map, initial attenuation map from the synthetic CT, reconstructed attenuation map, PET image, and MRI from a patient with a metal implant. Metal artifacts were apparent on the Dixon MRI. The initial attenuation map reasonably estimated the soft tissues and bones however the metal implant appeared like a large volume of bowel air due to the signal void in the MRI. Nevertheless, the metal artifact appeared as a region of low confidence and the joint reconstruction algorithm recovered the metal implant in the reconstructed attenuation map despite its absence in the initial attenuation map.

Figures 3 and 4 show the comparisons in PET uptake and SUVmax for lesions. We found that for lesions near the metal implant, JE w/o prior (i.e. a reconstruction using only PET data) and joint estimation with the BCNN synthetic CT and confidence map (JE) agree very well; using only DL-based pCT causes very large errors in lesions near the metal implant. For lesions far from the metal implant, JE agrees very well with DL-based pCT and JE w/o prior causes errors when the lesions are not in-plane with the implant.

Conclusion

A Bayesian deep learning framework was successfully used to generate confidence maps that can guide the PET joint estimation of attenuation and activity. The estimated attenuation map was able to recover the metal implant despite this region missing from the initial attenuation map. More accurately depicting actual imaging conditions would improve the quantitative and spatial accuracy of PET reconstructions: the JE approach performs well in lesions both near and far away from the metal implant unlike other approaches.

Acknowledgements

This work was supported by an NVIDIA GPU grant and NIH grant R01CA212148.

References

[1] S. Ahn et al., “Joint estimation of activity and attenuation for PET using pragmatic MR-based prior: application to clinical TOF PET/MR whole-body data for FDG and non-FDG tracers,” Phys. Med. Biol., vol. 63, no. 4, p. 045006, 2018.

[2] A. P. Leynes et al., “Direct PseudoCT Generation for Pelvis PET/MRI Attenuation Correction using Deep Convolutional Neural Networks with Multi-parametric MRI: Zero Echo-time and Dixon Deep pseudoCT (ZeDD-CT),” J Nucl Med, p. jnumed.117.198051, Oct. 2017.

[3] Y. Gal and Z. Ghahramani, “Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning,” arXiv:1506.02142 [cs, stat], Jun. 2015.

Figures

Figure 1. (top) Network architecture. Compared to the previous architecture1: only Dixon MRI was used, Dropout layers were added, and the number of channels were increased by 4 times to compensate for the Dropout layers. (bottom) Forward inference was done through Monte-Carlo Dropout. Several copies with varying structure and parameters were generated to produce a synthetic CT. The mean and variance images were collected from the stochastic outputs. The mean image was converted to an attenuation map and the variance image was used to generate a confidence map, which were then used in PET joint estimation of attenuation and activity.

Figure 2. (top row) Dixon MRI images at a slice with a metal artifact. (middle row, left) the confidence map generated from the variance image. (middle row, middle) the initial attenuation map generated from the mean image. (middle row, right) the final attenuation map generated from PET joint estimation of attenuation and activity. (bottom) the final reconstructed PET image.

Figure 3. (top row) the different attenuation coefficient maps generated with different PET reconstruction methods for a slice viewing a metal implant. Note that the metal implant appears as air in the DL-based pCT. (middle row, right) PET image. The yellow arrows point to high uptake lesions. (middle row, left and middle) PET difference images of JE and DL-based pCT compared to JE w/o prior. JE and JE w/o prior agree well in lesions located near the metal implant while DL-based pCT underestimates the uptake. This is also seen in the SUVmax correlation plots in the bottom row.

Figure 4. (top row) the different attenuation coefficient maps generated with different PET reconstruction methods for a slice without an implant. (middle row, right) PET image. The yellow arrows point to high uptake lesions. (middle row, left and middle) PET difference images of JE and JE w/o prior compared to DL-based pCT. JE and DL-based pCT agree very well for lesions far from the metal implant while JE w/o prior overestimates the uptake. This is also seen in the SUVmax correlation plots in the bottom row.

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)
1095