4936

PET Synthesis from Multi-Contrast MRI with Attention-Based 3D Encoder-Decoder Networks
Ramy Hussein1, David D. Shin2, Moss Zhao1, Jia Guo3, Michael Moseley1, and Greg Zaharchuk1
1Radiology, Stanford University, Stanford, CA, United States, 2Global MR Applications & Workflow, GE Healthcare, Menlo Park, CA, United States, 3Department of Bioengineering, University of California Riverside, Riverside, CA, United States

Synopsis

We present an attention-based 3D convolutional encoder-decoder network to synthetize PET Cerebral Blood Flow (CBF) maps from multi-parametric MRI images without using radioactive tracers. Inputs to the prediction model are structural MRI (T1 and T2 fluid-attenuated inversion recovery [FLAIR]), and arterial spin labeling (ASL) perfusion MRI images. Results show that encoder-decoder networks, with attention mechanisms and customized loss functions, can adequately combine multiple MRI image types and predict the gold-standard oxygen-15-water PET CBF maps. Adequate quantification of PET from MRI has a great potential for increasing the accessibility of cerebrovascular diseases assessment for underserved populations, underprivileged communities, and developing nations.

Introduction

Accurate quantification of cerebral blood flow (CBF) is essential for the diagnosis and assessment of cerebrovascular diseases such as stroke, carotid stenosis, and Moyamoya1. Yet, positron emission tomography (PET) with radiolabeled water (15O-water) remains the gold standard for measuring CBF in humans2. PET scans, however, are relatively expensive and not widely available. PET also can be done only if the radioactive substance is produced at a nearby location and can be injected into the bloodstream quickly. This study aims to improve the clinical utility of MRI-derived CBF measurements. We developed a 3D encoder-decoder network with attention mechanisms3 to predict the PET CBF maps from the combination of structural MRI (T1w and T2w FLAIR) and perfusion MRI (single- and multi-delay ASL) images.

Methods

Data were acquired from 114 subjects (64 healthy controls/45 Moyamoya patients/5 intracranial atherosclerotic steno-occlusive disease (ICSD) patients) on a 3T PET/MRI hybrid system (SIGNA, GE Healthcare) using an 8-channel head coil. The MRI scans included T1w and T2w-FLAIR, single-delay and multi-delay pseudo-continuous ASL, and quantified CBF/arterial transit time (ATT) maps derived from ASL. All MRI images were co-registered and normalized to the Montreal Neurological Institute (MNI) brain template and resized to 96×96×64 voxels. True quantitative PET CBF was determined using oxygen-15 water injection and the image-derived arterial input function kinetic model4.

A multidimensional attention network with encoder-decoder structure was developed to integrate spatial data across multiple MRI image types for synthetizing PET CBF maps, as shown in Figure 1. The encoder and decoder modules are specified as the 3D convolutional neural network (CNN), where the encoder compresses the input MRI images into a more condensed representation and the decoder uses this representation to output PET CBF maps. In addition, since different MRI sequences and spatial patterns impose different effects on the quality of synthetized PET images, an attention mechanism is embedded into the encoder-decoder to concurrently search the relevant aspects of the input at the channel and spatial levels for a fine-grained quality prediction. A set of 32 subjects was used for testing, and the rest of subjects were used for training and validation using four-fold cross-validation.

The prediction performance of the proposed encoder-decoder network was quantitatively evaluated in (i) image quality metrics including structural similarity index (SSIM), normalized root mean square error (NRMSE), and peak signal-to-noise ratio (PSNR), and by (ii) paired t-tests such as Box plot, Bland-Altman analysis, distribution plots, and Pearson’s correlation coefficient. We also evaluated agreement for detecting abnormally low true CBF in 10 ASPECTS5 vascular territories (anterior, 3 middle, and posterior in each hemisphere) in 17 of the patients with cerebrovascular disease, each of whom had two independent PET studies.

Results

Figure 2 shows the input MRI volumes, predicted PET CBF, true PET CBF, and magnified absolute error maps for healthy controls and patients with cerebrovascular diseases. Results indicate that attention-based encoder-decoder convolutional networks can adeptly predict PET CBF maps from structural and perfusion MRI images, achieving an SSIM of 0.94, NRMSE of 0.001, and PSNR of 38.8dB. Figure 3 also depicts the mean CBF of true and synthetic PET, showing no statistically significant difference (p > 0.05).

To assess the clinical significance of the proposed PET prediction algorithm, a set of paired comparison analyses were conducted. The Bland-Altman plot in Figure 4(a) delineates the agreement between the mean CBF of the true and predicted PET CBF maps. It shows a small bias, where the true gold-standard PET CBF maps are 4.6 ml/100g/min higher than the synthetic PET CBF maps produced by our encoder-decoder network, with 95% confidence intervals of -4.4 & 13.5 ml/100g/min. Figure 4(b) describes the histogram and density plots of the mean CBFs of true and synthetic PET’s vascular territories, showing strong positive correlation (Pearson correlation coefficient = 0.97). Figure 5 shows the diagnostic performance of the model to identify reduced CBF regions. An average sensitivity, specificity, classification accuracy, and AUC score of 100.0%, 96.52%, 96.76%, and 0.982 were achieved, respectively.

Discussion

Here, we demonstrate that a 3D convolutional neural network can synthetize gold-standard PET CBF maps from multi-contrast MRI without using radioactive tracers. We developed an encoder-decoder framework that uses attention mechanisms to identify the crucial aspects of the input MRI images, thus producing high-quality CBF maps. Compared with previous work that achieved an average SSIM of 0.846, the current model yields a notably higher prediction performance with an average SSIM of 0.94. Other quantitative analyses, such as Bland-Altman plot, show relatively tight 95% confidence intervals with minimal bias, implying that true and synthetic PET CBF maps are in good agreement. To evaluate the benefits of the proposed algorithm in clinical practice, we investigated how well the CBF values of the different vascular territories of synthetic and true PET compare. Box plots and joint plots show high levels of agreement and correlation.

Conclusion

Optimized encoder-decoder networks with attention mechanisms can efficiently synthetize PET CBF maps from multi-contrast MRI while eliminating the need for radioactive tracers. The synthetic PET CBF maps are strongly correlated with the true PET CBF measurements, with no statistically significant difference between them. Also, the vascular territories with low CBF are accurately identified in synthetic PET CBF maps with high accuracy.

Acknowledgements

Ramy Hussein has received grant funding from NIH/NIA (P30 AG066515).

References

  1. Harston, George WJ, et al. "Quantification of serial cerebral blood flow in acute stroke using arterial spin labeling." Stroke48.1 (2017): 123-130.
  2. Acker, Güliz, et al. "Brain perfusion imaging under acetazolamide challenge for detection of impaired cerebrovascular reserve capacity: positive findings with 15O-water PET in patients with negative 99mTc-HMPAO SPECT findings." Journal of Nuclear Medicine 59.2 (2018): 294-298.
  3. Ji, Zhong, et al. "Video summarization with attention-based encoder–decoder networks." IEEE Transactions on Circuits and Systems for Video Technology 30.6 (2019): 1709-1717.
  4. Khalighi, Mohammad Mehdi, et al. "Image-derived input function estimation on a TOF-enabled PET/MR for cerebral blood flow mapping." Journal of Cerebral Blood Flow & Metabolism 38.1 (2018): 126-135.
  5. Barber, Philip A., et al. "Validity and reliability of a quantitative computed tomography score in predicting outcome of hyperacute stroke before thrombolytic therapy." The Lancet355.9216 (2000): 1670-1674.
  6. Guo J, Gong E, Fan AP, Goubran M, Khalighi MM, Zaharchuk G. Predicting 15O-water PET cerebral blood flow maps from multi-contrast MRI using a deep convolutional neural network with evaluation of training cohort bias. J. Cereb. Blood Flow Metab 2019; 0271678X19888123.

Figures

Figure 1. Our attention-based encoder-decoder network for predicting PET CBF maps from multi-contrast MRI scans.

Figure 2. Examples of input MRI sequences, output synthetic PET, target PET, and corresponding Absolute Error maps.

Figure 3. Box plots of actual and synthetic PET CBF measurements.

Figure 4. Paired comparison analyses of actual and synthetic PET: (a) Bland-Altman plot of CBF Mean and Difference of actual and synthetic PET, (b) Joint plot of Mean CBF in actual and synthetic PET’s vascular territories.

Figure 5. Confusion matrix for identifying vascular territories with low CBF (n=17 cerebrovascular patients).

Proc. Intl. Soc. Mag. Reson. Med. 30 (2022)
4936
DOI: https://doi.org/10.58530/2022/4936