0811

High-Quality Brain MRI Reconstruction against Unknown Degradation: A Unified Framework with Prompt Learning

Ning Jiang^1,2,3 and Yao Sui^1,2
¹National Institute of Health Data Science, Peking University, Beijing, China, ²Institute of Medical Technology, Peking University, Beijing, China, ³School of Medical Technology, Beijing Institute of Technology, Beijing, China

Synopsis

Keywords: AI/ML Image Reconstruction, Machine Learning/Artificial Intelligence

Motivation: Spatial resolution, signal-to-noise ratio, and motion artifacts critically matter in any MRI practices. Current methods focus on a single source of known degradation of imaging. A unified framework is desired, which allows for high-quality reconstruction in the face of multiple unknown sources of degradation.

Goal(s): We reconstruct high-quality brain MRI against degradations by motion, noise, and low resolution, with an image-to-image translation-based deep neural framework.

Approach: We developed a prompt-based learning approach and assessed it on a public brain MRI dataset.

Results: Our method offered remarkably improved reconstructions (PSNR=30.96dB, SSIM=0.9133), as compared to two other state-of-the-art methods.

Impact: We developed a new methodology that enables high-quality MRI reconstruction from scans corrupted by a mixture of multiple unknown sources of degradations, which commonly happen in clinical and research MRI studies, with a unified reconstruction framework.

Introduction

Spatial resolution, signal-to-noise ratio (SNR), and motion artifacts critically matter in any MRI practices^1,2. High spatial resolution allows for delineating fine anatomical structure but unfortunately suffers from reduced SNR and prolongs scan time. Long scan time discomforts patients and increases the potential of motion artifacts. Subject motion adversely affects MRI quality and substantially increases image time due to the need for repeating scans³. The unnecessarily increased imaging time due to motion has been estimated to cost clinical and research studies over $115,000 per scanner, and $1.4B per year in the US alone⁴. End-to-end deep learning techniques have recently emerged to offer high-quality MRI reconstruction through motion correction⁵, denoising⁶, and super-resolution⁷. However, the majority of those methods are based on a single task-specific model that focuses on a single source of known degradation of imaging, and therefore are limited in MRI acquisitions with various degradations that commonly happen in a single scan. In the face of unknown sources of degradation, we developed a new methodology that enables high-quality MRI reconstruction from scans corrupted by a mixture of multiple unknown sources of degradations. Our approach offers a unified reconstruction framework equipped with a prompt-based joint learning strategy, so allows for reconstructing high-quality MRI with a single deep model. Experiments demonstrated the advantages of our approach in motion correction, SNR improvement, and resolution enhancement.

Methods

Our goal is to achieve a unified framework to reconstruct high-quality MRI from unknown sources of degradation. Based on the image-to-image translation technique^8,9, our framework incorporates a degradation encoder and a joint learning process. The former embeds degradation representations from low-quality images, whereas the latter enhances the representations for both high-quality image restoration and low-quality image re-degradation. Moreover, a dual-prompt module (DPM) is utilized to integrate the learned degradation representations into the generators in joint learning (Figure 1). The DPM consists of a spatial prompt branch for spatially adaptive modulation of the learned degradation representations and a soft prompt branch to encode degradation-specific information (Figure 2).
Specifically, in the joint learning process, the low-quality to high-quality image restoration combines $$$L_{1}$$$ loss, SSIM loss, and Edge loss¹⁰ as the loss function:$$L_{low-to-high}=\lambda_{1}L_{1}+\lambda_{2}L_{SSIM}+\lambda_{3}L_{Edge}$$In the image re-degradation process that maps high-quality images to low-quality correspondences, we employ a multi-scale discriminator to identify the fake degraded images, and perceptual loss¹¹, adversarial loss (hinge loss¹²), and feature matching loss¹³ are used in the loss function:$$L_{high-to-low}=\lambda_{4}L_{Perceptual}+\lambda_{5}L_{Adv}+\lambda_{6}L_{Feat}$$
We simulated the low-quality brain MRI scans with various degradations (Figure 3a). We degraded MPRAGE scans by adding head motion simulated from the translations and rotations of a random sampling of phase-encoding lines in the frequency domain¹⁴. We cropped out the low-frequency data in the center of k-space, and zeroed out the peripheral data, to generate low-resolution scans. We added white Gaussian noise to simulate noisy acquisitions.
Seventy MPRAGE images of healthy adults with ages ranging from 18 to 88 years from the Cam-CAN dataset¹⁵ were used in our experiments. We used 40 volumes for training, 15 volumes for validation, and 15 volumes for testing (Figure 3b). We extracted 90 axial slices from each volume, and each slice was normalized with the size of 192 × 224 pixels and with the pixel intensities in [-1, 1], and every three adjacent slices were inserted into a three-channel image as the network input.

Results

Figure 4 shows that our method offered superior performance, as compared to two other state-of-the-art methods: SRGAN and pix2pix, in terms of PSNR, SSIM, and RMSE. Figure 5 shows the reconstructions for representative subjects from the test set as qualitative assessments. The results show that SRGAN and pix2pix were unable to completely remove the motion artifacts or restore the images from the blurry and/or noisy acquisitions. In contrast, our method eliminated the motion artifacts, while in parallel, generated images with noise substantially removed and edges considerably sharpened. Particularly, our method provided stable reconstructions and generalized well across a wide range of ages (Subject 3: 76-year-old vs. Subject 6: 18-year-old) in different degradations/levels.

Discussion

We have developed a new methodology that enables high-quality brain MRI reconstruction from scans corrupted by a mixture of multiple unknown sources of degradations. We have designed a unified reconstruction framework without any expert knowledge or prior knowledge about degradations required. We have demonstrated the efficacy of our method on the dataset with various simulated degradations. Experiments have shown that our approach allowed for reconstructing high-quality MRI scans from a wide variety of unknown sources of degradations that commonly happen in clinical and research MRI studies.

Acknowledgements

This work was supported by the Faculty Development Award from Peking University under Award No. 71013Y2268.

References

1. Plenge E, Poot DH, Bernsen M, Kotek G, Houston G, Wielopolski P, van der Weerd L, Niessen WJ, Meijering E. Super-resolution methods in MRI: can they improve the trade-off between resolution, signal-to-noise ratio, and acquisition time? Magn Reson Med. 2012, 68(6):1983-93.

2. Afacan O, Erem B, Roby DP, Roth N, Roth A, Prabhu SP, Warfield SK. Evaluation of motion and its effect on brain magnetic resonance image quality in children. Pediatr Radiol. 2016, 46(12):1728-1735.

3. Lanka P, Deshpande G. Combining prospective acquisition correction (pace) with retrospective correction to reduce motion artifacts in resting state fMRI data. Brain Behav. 2019, 9(9):21-34.

4. Andre JB, Bresnahan BW, Mossa-Basha M, Hoff MN, Patrick Smith C, Anzai Y, Cohen WA. Toward quantifying the prevalence, severity, and cost associated with patient motion during clinical MR examinations. J. Am. Coll. Radiol. 2015, 12 (7):689-695.

5. Al-masni MA, Lee S, Yi J, Kim S, Gho S-M, Choi YH, Kim D-H. Stacked U-Nets with self-assisted priors towards robust correction of rigid motion artifact in brain MRI. NeuroImage 2022, 259:119411.

6. Yu M, Guo M, Zhang S, Zhan Y, Zhao M, Lukasiewicz T, Xu Z. RIRGAN: An end-to-end lightweight multi-task learning method for brain MRI super-resolution and denoising. Comput. Biol. Med. 2023, 167:107632.

7. Lyu Q, Shan H, Steber C, Helis C, Whitlow C, Chan M, Wang G. Multi-contrast super-resolution MRI through a progressive network. IEEE Trans. Med. Imaging. 2020, 39(9):2738-2749.

8. Isola P, Zhu J-Y, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition: 2017. 1125-1134.

9. Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision: 2017. 2223-2232.

10. Ma C, Rao Y, Cheng Y, Chen C, Lu J, Zhou J. Structure-preserving super resolution with gradient guidance. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition: 2020. 7769-7778.

11. Johnson J, Alahi A, Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14: 2016. Springer: 694-711.

12. Lim JH, Ye JC. Geometric gan. arXiv preprint arXiv:1705.02894. 2017.

13. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X. Improved techniques for training gans. Adv. Neural Inf. Process. Syst. 2016, 29.

14. Duffy BA, Zhao L, Sepehrband F, Min J, Wang DJ, Shi Y, Toga AW, Kim H. Retrospective motion artifact correction of structural MRI images using deep learning improves the quality of cortical surface reconstructions. NeuroImage 2021, 230:117756.

15. Taylor JR, Williams N, Cusack R, Auer T, Shafto MA, Dixon M, Tyler LK, Cam C, Henson RN. The Cambridge Centre for Ageing and Neuroscience (Cam-CAN) data repository: Structural and functional MRI, MEG, and cognitive data from a cross-sectional adult lifespan sample. NeuroImage 2017, 144:262-269.

Figures

Figure 1: Overall architecture of our joint-learning framework. Both the image restoration and re-degradation networks include a HINet backbone with Dual-Prompt Modules (DPM) serving as multi-scale degradation injectors. Only a single training stage is required: the image re-degradation assists the image restoration to encode latent spatially varying degradation representations during training and is disabled for testing. The red and black dotted lines denote the output features after the resblock (before the downsample block) and after the downsample block, respectively.

Figure 2: Architecture of our proposed Dual-Prompt Module. The soft prompt branch consists of two steps: prompt generation and injection. In the generation step, prompt components form a set of learnable parameters to embed degradation information. A set of attention-based weights is dynamically predicted from the degradation representation and applied to prompt components to yield degradation-conditioned prompts (soft prompt). The injection step enables the interaction between the input features and generated prompts.

Figure 3: Experimental design. (a) Degradation simulation strategies for five types of degradation at different levels. (b) Scan parameters and data splitting strategies on the Cam-CAN dataset.

Figure 4: Quantitative assessments. (a) Average PSNR, SSIM, and RMSE on the testing set (1380 slices in total). (b) Results of statistical analyses, where the error bars depict the standard deviations of all samples in the test set. *** indicates that p ≤ 1.0 × 10^-40 in a two-sample t-test between different methods.

Figure 5: Qualitative assessments. The corrupted images are shown in the first column, the ground truth in the second column, and the images reconstructed using SRGAN, pix2pix, and our proposed method in the third to five columns, respectively. The results show that our method offered superior results in removing artifacts (Subjects 1 and 2), reducing blur and noise (Subjects 3 and 4), and preserving anatomical details (Subjects 1-6). SRGAN and pix2pix generated fake anatomies that do not exist in the ground truth for Subject 3, while our approach provided correct reconstruction.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)

0811

DOI: https://doi.org/10.58530/2024/0811