1760
Denoising intrinsic MRI repetitions using self-supervised iterative residual learning1Department of Biomedical Engineering, Tsinghua University, Beijing, China, 2Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, United States, 3Harvard Medical School, Boston, MA, United States, 4Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom, 5Department of Engineering Physics, Tsinghua University, Beijing, China
Synopsis
Keywords: Analysis/Processing, Data Processing, magnetic resonance imaging, diffusion tensor imaging, self-supervised learning, transfer learning
Motivation: MRI with high resolution and/or acceleration factor suffers from intrinsic low signal-to-noise ratio. Supervised learning-based denoising significantly improves image quality, but requires high-SNR data as training targets.
Goal(s): To denoise images using noisy image repetitions without additional acquisition.
Approach: Noise2Average trains CNN to map each noisy image to its residual compared to the average of all noisy images at iteration 1 and all denoised images from iteration k-1 at iteration k. The images from opposite phase-encoding directions of EPI or different echo times of ME-MPRAGE are noisy repetitions.
Results: Noise2Average outperforms BM4D, AONLM and Noise2Noise in terms of image quality and DTI metrics.
Impact: By reducing the requirement for training data and time, Noise2Average substantially increases the feasibility and accessibility of deep learning based denoising methods for MRI and potentially benefits a wider range of clinical and neuroscientific studies.
Introduction
MRI is a non-invasive imaging tool widely adopted in research and clinical settings. Nonetheless, the quality of MR images is often hampered by noise, which not only confounds the qualitative interpretation for clinical diagnosis but also reduces the accuracy and precision of subsequent analytic tasks.Image denoising provides an alternative approach to effectively improve MRI image quality. Deep learning-based denoising using convolutional neural network (CNNs) has been demonstrated a superior technology1 and is widely adopted in biomedical imaging2,3.
Nonetheless, most deep learning-based denoising methods require high-SNR reference data for supervising the training, which significantly reduces their practical feasibility and accessibility. Another novel learning strategy entitled Noise2Noise addresses this problem by training CNN to map one noisy image to another repetition of the noisy image4. Unfortunately, Noise2Noise assumes that the two repetitions only differ in noise, which cannot be satisfied in practice due to motion and spatially and temporally varying image artifacts, leading to image blurring5.
We propose a self-supervised iterative denoising method Noise2Average to address this challenge. We previously demonstrated the efficacy of Noise2Average on two repetitions of T1w images6. Since many sequences intrinsically provide image repetitions, such as images acquired with reversed phase-encoding directions in EPI or with different echo times in multi-echo (ME) MPRAGE, in this study, we further simplify Noise2Average by leveraging such information and eliminating the need for collecting additional repetitions.
Methods
HCP in Aging (HCP-A) diffusion data. Diffusion data (14×b=0, 93×b=1500 s/mm2, 1.5 mm iso) were acquired with anterior–posterior (AP) and PA phase-encoding directions using 2D EPI on 30 subjects. Data were corrected for distortions and co-registered using the “topup” and “eddy” functions from FSL7-10. The corrected AP and PA images from the first b=0 and the first six b=1500 s/mm2 images served as noisy repetitions for denoising (15-fold acceleration). Ground truth tensor model was fitted using “dtifit” in FSL on all available data and were used to generate ground truth images.MGH T1-weighted ME-MPRAGE data. T1w data (0.6 mm iso, six repetitions, TR=2510 ms, TI=1200 ms, TE=2.88/5.6 ms) were acquired using a 3D ME-MPRAGE sequence11 on nine subjects. Two images from different echo time of the first repetition served as noisy repetitions for denoising (six-fold acceleration) whose contrast were different. Two image volumes of each repetition with different echo times were combined into one using root mean square. Ground truth image was the average of six repetitions for each subject.
Denoising. Noise2Average trains a Modified U-Net (MU-Net, Fig.1d, e) to map each noisy image to its residual compared to average of all noisy images at iteration 1, and all denoised images from iteration k-1 at iteration k (Fig1.c). Each iteration took ten epochs at learning rate=0.00001. To accelerate training, MU-Net's parameters were initialized with those from the MU-Net pre-trained on large datasets with different noise distributions (i.e., T1w data at 0.7×0.7×0.7 mm3 with simulated Gaussian noise of 20 subjects from WU-Minn-Ox HCP, and diffusion data at 1.5×1.5×1.5 mm3 of 35 subjects from MGH-USC HCP12) for 10 epochs.
Comparison. BM4D13,14 and AONLM15 were used to denoise the averaged noisy repetitions. Supervised denoising was trained on the data of four and 20 subjects from HCP-A data and ME-MPRAGE data respectively. Noise2Noise was trained to map each noisy images to its paired image (Fig.1a-b).
Evaluation. The mean absolute error (MAE), peak SNR (PSNR) and structural similarity index (SSIM) within the brain were computed to quantify the image similarity of raw and denoised images compared to the ground truth. The MAE within the brain tissue excluding cerebrospinal fluid of five DTI metrics including V1, FA, MD, AD, RD compared to the ground truth was used to quantify the quality of raw and denoised images for diffusion tensor modeling.
Results
Strong susceptibility induced geometric distortion and signal pile-up and dropout near the air-tissue interface existed for 2D EPI, even after correction, which did not satisfy Noise2Noise’s assumption (Fig.2). Noise2Average-denoised images from the 2nd iteration outperformed BM4D, AONLM and Noise2Noise for both diffusion and T1w data, in terms of both image quality (Fig.3, 5) and accuracy DTI metrics. (Fig.4). Images deried from Noise2Noise were blury. Noise2Average exhibited superior denoising performance at early iterations while led to image blurring due to error accumulation in target images.Discussion and Conclusion
Noise2Average does not require high-SNR reference training data or suffer from the generalization problem and can achieve self-supervision in combination with transfer learning. By constructing noisy pairs from regular imaging sequences, Noise2Average makes full use of the information present in the imaging data and further increases it practical feasibility and accessibility.Acknowledgements
T1w and diffusion MRI data were provided in part by the Human Connectome Project, WU-Minn-Ox Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; U54-MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research, by the McDonnell Center for Systems Neuroscience at Washington University, as well as the Human Connectome Project, MGH-USC Consortium (Principal Investigators: Bruce R. Rosen, Arthur W. Toga and Van Wedeen; U01MH093765) funded by the NIH Blueprint Initiative for Neuroscience Research grant; the National Institutes of Health grant P41EB015896; and the Instrumentation Grants S10RR023043, 1S10RR023401, 1S10RR019307.References
1. Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing. 2017;26(7):3142-3155.
2. Tian Q, Zaretskaya N, Fan Q, et al. Improved cortical surface reconstruction using sub-millimeter resolution MPRAGE by image denoising. NeuroImage. 2021;233:117946.
3. Li Z, Tian Q, Ngamsombat C, et al. High‐fidelity fast volumetric brain MRI using synergistic wave‐controlled aliasing in parallel imaging and a hybrid denoising generative adversarial network (HDnGAN). Medical physics. 2021;
4. Lehtinen J, Munkberg J, Hasselgren J, et al. Noise2Noise: Learning Image Restoration without Clean Data. The International Conference on Machine Learning. 2018:2965-2974.
5. Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017:4681-4690.
6. Li Z, Bilgic B, Li Z, Ying K, Polimeni J, Huang S, Tian Q. Noise2Average: deep learning based denoising without high-SNR training data using iterative residual learning. Proceedings of the 31st Annual Meeting of the International Society for Magnetic Resonance in Medicine (ISMRM) 2022. 2022;
7. Smith SM, Jenkinson M, Woolrich MW, et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage. 2004;23:S208-S219.
8. Jenkinson M, Beckmann CF, Behrens TEJ, Woolrich MW, Smith SM. FSL. NeuroImage. 8/15/ 2012;62(2):782-790.
9. Andersson JL, Skare S, Ashburner J. How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. Neuroimage. 2003;20(2):870-888.
10. Andersson JL, Sotiropoulos SN. An integrated approach to correction for off-resonance effects and subject movement in diffusion MR imaging. NeuroImage. 2016;125:1063-1078.
11. van der Kouwe AJ, Benner T, Salat DH, Fischl B. Brain morphometry with multiecho MPRAGE. NeuroImage. 2008;40(2):559-569.
12. Fan Q, Witzel T, Nummenmaa A, et al. MGH–USC Human Connectome Project datasets with ultra-high b-value diffusion MRI. NeuroImage. 2016;124:1108-1114.
13. Maggioni M, Katkovnik V, Egiazarian K, Foi A. Nonlocal transform-domain filter for volumetric data denoising and reconstruction. IEEE transactions on image processing. 2012;22(1):119-133.
14. Maggioni M, Foi A. Nonlocal transform-domain denoising of volumetric data with groupwise adaptive variance estimation. Computational Imaging X. 2012;8296:82960O
15. Manjón JV, Coupé P, Martí‐Bonmatí L, Collins DL, Robles M. Adaptive non‐local means denoising of MR images with spatially varying noise levels. Journal of Magnetic Resonance Imaging. 2010;31(1):192-203.
Figures
Figure 1. Learning strategies. Supervised denoising trains a CNN to map the average noisy images to its residual image compared to the ground-truth image (a). Noise2Noise trains a CNN to map one noisy image to other noisy images (b). Noise2Average trains a CNN to map one noisy image to image with increased SNR which is the average of the noisy images for iteration 1, and the average of all denoised images from iteration k-1 for iteration k (k>1, …) (c). 3D modified U-Nets with receptive view of 23 and 39 are used to adapt to accommodate different image resolutions (d: 1.5 mm, e: 0.6 mm).
Figure 2. DWI with opposite phase-encoding directions. Exemplary axial image slices acquired with opposite phase-encoding (PE) directions (i.e., anterior–posterior (AP) and posterior–anterior (PA)) (i, ii) before (a, c) and after (b, d) geometric distortion and signal intensity correction using the “topup” function of FSL through the middle brain (a, b) and brain regions near air-tissue boundaries (c, d, green and magenta arrowheads) of a subject from the HCP-A data are shown. Residual maps between corrected images with opposite PE directions are shown (b, iii and d, iii).
Figure 3. Image results of HCP-A diffusion data. Axial image slices of the DWI synthesized from ground-truth tensors (a, i), raw acquired anterior-posterior (AP) volume (a, ii), average volume of AP and PA volumes (a, iii), BM4D-denoised, AONLM-denoised, supervised learning-denoised averaged volume (a, iv-vi), Noise2Noise-denoised data (c, i), Noise2Average-denoised data from iteration 1 to 5 (c, ii-vi) of a subject of HCP-A diffusion data are displayed. The group means (± group standard deviations) of image metrics over 10 subjects are listed (e).
Figure 4. DTI metrics of HCP-A diffusion data. The group means (± group standard deviations) of the mean absolute errors between DTI metrics derived from one b = 0 and six DWI volumes acquired along anterior-posterior (AP) phase-encoding direction (a), AP and PA average volume (b), BM4D-denoised, AONLM-denoised, supervised learning-denoised and Noise2Noise-denoised data (c-f) and Noise2Average (N2A)-denoised data from iteration 1 to 5 (g-k) and the ground truth over 20 subjects. Red and green texts indicate the best and the second-best performance from different methods.
Figure 5. Image results of MGH T1w ME-MPRAGE data. Exemplary axial image slices from the 6-repetition averaged image volume (ground truth, a, i), single noisy image volume (the root mean square of the two images with different echo times of one repetition) (a, ii), BM4D-denoised, AONLM-denoised, supervised learning-denoised data (a, iii-v) , Noise2Noise-denoised data (c, i) and Noise2Average-denoised data from iteration 1 to 4 (d, ii-v) of a subject from MGH T1w ME-MPRAGE data are shown. The group means (± group standard deviations) of image metrics over five subjects are listed (e).