2785
Noise-induced Variability Quantification in Deep Learning-Based MRI Reconstructions1Electrical Engineering, Stanford University, Stanford, CA, United States, 2Radiology, Stanford University, Stanford, CA, United States, 3Bioengineering, Stanford University, Stanford, CA, United States
Synopsis
Keywords: Machine Learning/Artificial Intelligence, Data Analysis
Motivation: While noise propagation in linear imaging methods like SENSE is well-studied, similar analysis is not common in deep learning-based MRI reconstructions.
Goal(s): Evaluate, characterize, and compare noise propagation in deep learning-based and linear MRI reconstruction under varying conditions.
Approach: We uses Monte Carlo simulations to empirically analyze mean and variance of knee MRI images reconstructed by SENSE and deep learning methods.
Results: SENSE yields unstructured, relatively-uniform noise distribution, while deep learning methods produce anatomically structured noise with substantial variability across tissues, acceleration factors, and noise levels. Noise-aware deep learning reconstruction shows more uniform noise propagation and reduced tissue-specific variability.
Impact: Elucidating noise propagation in deep MRI reconstructions could direct algorithm refinement, optimizing image quality and reliability for clinical application. Simply reconstructing noise-propagation maps in routine protocols may help in image interpretation.
Introduction
Deep learning-based reconstruction for accelerated MRI offers considerable improvements in image quality, often surpassing traditional methods in their ability to reconstruct fine details and reduce artifacts4,5,6. Despite their prowess, the noise characteristics intrinsic to deep reconstructions remain largely unexplored, partly due to the nonlinearity in deep learning algorithms. In conventional methods like SENSE, noise propagation is modeled through linear processes, and is generally not correlated with the anatomical structures within the image1,7. In contrast, deep learning approaches process either k-space or image data through multiple non-linear layers, potentially leading to a structured noise behavior that is more intricate and less understood8,9. Prior work has shown that Monte Carlo-based approaches are effective at characterizing noise in standard reconstructions such as SENSE1 or GRAPPA2,3,9,11. In this study, we apply the same approach3 to systematically evaluate noise propagation across different reconstruction paradigms to illustrate substantial differences based on network structure.Theory
Accelerated MRI techniques aim to recover the image $$$x$$$ from undersampled k-space acquisitions $$$y$$$, which involves solving the inverse of the encoding:$$y = Ax$$where $$$A=\Omega F B$$$ is the imaging operator comprising undersampling mask $$$\Omega$$$, Fourier transform $$$F$$$, and coil sensitivity profiles $$$B$$$. Commonly, deep reconstruction algorithms first form an aliased image by applying the adjoint operator $$$A^*$$$, followed by a non-linear de-aliasing transformation characterized by a neural network:$$\hat{x}=f(A^*y)$$Acquisitions are assumed to be corrupted by zero-mean additive white Gaussian noise with coil covariance matrix $$$\Sigma_k$$$:$$y=y_0+n,n\sim\mathcal{N}(0,\Sigma_k)$$In deep reconstructions, spatial noise variance is influenced by the neural network Jacobian, manifesting non-linearity. Yet, calculating this exact Jacobian, necessary to determine the noise propagation precisely, is usually computationally intractable due to the high resolution of the images and the complexity of the network.Method
As a practical alternative, we employed a Monte Carlo-based approach to empirically calculate mean and variance maps associated with different reconstruction methods3. Inspired by the Pseudo Multiple Replica (PMR) method3, we assumed the repeated acquisition of k-space data only differs in noise content, with consistent signal component across trials. This approach emulates multiple acquisitions and simulates the observed variability with repeated measurements due to noise fluctuations. We assumed noise to be uncorrelated between each channel of the phased-array, a condition expected after standard preprocessing involving channel whitening12. 100 Monte Carlo trials were performed for each reconstruction method, and mean and standard deviation maps were calculated across these trials. We investigated three MRI reconstruction techniques: SENSE1, U-net13, and Noise2Recon10, a state-of-the-art noise-aware unrolled architecture that enables semi-supervised training. All methods were implemented as described in original manuscripts. Experiments were performed on a 3D fast-spin-echo multi-coil knee scan14 with matrix size $$$320×320×256$$$. Four noise levels ($$$\Sigma_k=\sigma_k I$$$, $$$\sigma_k=\{0.001,0.01,0.02,0.05,0.10\})$$$ and two acceleration rates $$$R=\{3,6\}$$$ were considered.Results
The variance image across SENSE reconstructions displays a uniform pattern, aligning with the analytical noise propagation in linear reconstructions (Fig. 1). On the other hand, variance maps associated with deep methods reveal anatomically structured noise patterns, with certain tissues exhibiting heightened noise variability, notably around tissue boundaries. U-net suppresses noise in the background and high-SNR muscles, albeit selectively amplifies noise in diagnostically relevant tissues, especially fluid and cartilage. While still reflecting structured noise, Noise2Recon has a more consistent variability across the image, with modest amplifications around cartilage. Elevated noise levels at a constant acceleration spread variability more uniformly across tissues, leading to a more homogeneous noise pattern without distinguishing different tissue types (Fig. 1).Increased acceleration factor preserves noise amplification in certain tissues albeit reducing overall variability, thereby increasing discrepancies in variation between different tissues (Fig. 2). displays the average variability inside fluid and cartilage tissue reconstructions.
Although SENSE exhibits lower variability than U-net and Noise2Recon around fluid and cartilage, signaling noise amplification of deep methods, Noise2Recon demonstrates less variability due to noise propagation over U-net (Fig 3).
Discussion
Here we observed the U-net's intrinsic bias towards tissue homogeneity, suppressing noise in smooth and high-SNR areas like muscle tissue, possibly reflecting its learned local interpolation strategy as a deep network. In contrast, Noise2Recon achieves more uniform noise distribution across various tissues. This might be attributed to method's consistency training, which aims to reduce variance and promote stability across noisy conditions, aiming for consistent reconstructions that maintain diagnostic quality uniformly across tissues without bias toward specific tissue characteristics. Future work should systematically evaluate how deep learning architectures and training paradigms influence noise variability, especially in clinically significant regions. This behavior could potentially be harnessed to develop more advanced and targeted reconstruction techniques that prioritize clinically relevant features according to the diagnostic context. Furthermore, reconstructing the noise-propagation maps may provide useful insight for interpretation.Acknowledgements
This study was supported in part by the National Institutes of Health under grant R01-AR077604.References
1. Pruessmann, K. P., Weiger, M., Scheidegger, M. B., & Boesiger, P. (1999). SENSE: sensitivity encoding for fast MRI. Magnetic resonance in medicine, 42(5), 952–962.
2. Kellman, P., & McVeigh, E. R. (2005). Image reconstruction in SNR units: a general method for SNR measurement. Magnetic resonance in medicine, 54(6), 1439–1447. https://doi.org/10.1002/mrm.20713
3. Robson, P. M., Grant, A. K., Madhuranthakam, A. J., Lattanzi, R., Sodickson, D. K., & McKenzie, C. A. (2008). Comprehensive quantification of signal-to-noise ratio and g-factor for image-based and k-space-based parallel imaging reconstructions. Magnetic resonance in medicine, 60(4), 895–907. https://doi.org/10.1002/mrm.21728
4. Jun, Y., Eo, T., Kim, T., Jang, J., & Hwang, D. (2017). Deep convolutional neural network for acceleration of magnetic resonance angiography (MRA). Proceedings of the 25th Annual Meeting of ISMRM; 2017 April 22-27; Honolulu, Hawaii.
5. Dar, S. U. H., Özbey, M., Çatlı, A. B., & Çukur, T. (2020). A Transfer‐Learning Approach for Accelerated MRI Using Deep Neural Networks. In Magnetic Resonance in Medicine (Vol. 84, Issue 2, pp. 663–685). Wiley. https://doi.org/10.1002/mrm.28148
6. Knoll, F., Hammernik, K., Zhang, C., Moeller, S., Pock, T., Sodickson, D. K., & Akcakaya, M. (2020). Deep-Learning Methods for Parallel Magnetic Resonance Imaging Reconstruction: A Survey of the Current Approaches, Trends, and Issues. In IEEE Signal Processing Magazine (Vol. 37, Issue 1, pp. 128–140). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/msp.2019.2950640
7. Robson, P. M., Grant, A. K., Madhuranthakam, A. J., Lattanzi, R., Sodickson, D. K., & McKenzie, C. A. (2008). Quantification of SNR and g-Factor for parallel MRI: Universal application to image-based and k-space-based image reconstructions. Proceedings of the 16th Annual Meeting of ISMRM; 2008 May 3-9; Toronto, Canada.
8. Lebel, R.M. (2020). Performance characterization of a novel deep learning-based MR image reconstruction pipeline. ArXiv, abs/2008.06559.
9. Fischer, P., Thomas, K., Baumgartner, C.F. (2023). Uncertainty Estimation and Propagation in Accelerated MRI Reconstruction. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. UNSURE 2023. vol 14291. Springer, Cham. https://doi.org/10.1007/978-3-031-44336-7_9
10. Desai, A. D., Ozturkler, B. M., Sandino, C. M., Boutin, R., Willis, M., Vasanawala, S., Hargreaves, B. A., Ré, C., Pauly, J. M., & Chaudhari, A. S. (2023). Noise2Recon: Enabling SNR-robust MRI reconstruction with semi-supervised and self-supervised learning. Magnetic resonance in medicine, 90(5), 2052–2070. https://doi.org/10.1002/mrm.29759
11. Koonjoo, N., Zhu, B., Bagnall, G. C., Bhutto, D., & Rosen, M. S. (2021). Boosting the signal-to-noise of low-field MRI with deep learning image reconstruction. Scientific reports, 11(1), 8248. https://doi.org/10.1038/s41598-021-87482-7
12. Rodgers, C. T., & Robson, M. D. (2010). Receive array magnetic resonance spectroscopy: Whitened singular value decomposition (WSVD) gives optimal Bayesian solution. Magnetic resonance in medicine, 63(4), 881–891. https://doi.org/10.1002/mrm.22230
13. Ghodrati, V., Shao, J., Bydder, M., Zhou, Z., Yin, W., Nguyen, K. L., Yang, Y., & Hu, P. (2019). MR image reconstruction using deep learning: evaluation of network structure and loss functions. Quantitative imaging in medicine and surgery, 9(9), 1516–1527. https://doi.org/10.21037/qims.2019.08.10
14. Ong, F., Amin, S., Vasanawala, S., & Lustig, M. (2018). Mridata.org: An open archive for sharing MRI raw data. Proceedings of the 26th Annual Meeting of ISMRM; 2018 June 16-21; Paris, France.
Figures
Results for a representative 2D sagittal knee slice at $$$R=3$$$, and a) $$$\sigma_k=0.001$$$, b) $$$\sigma_k=0.01$$$, c) $$$\sigma_k=0.1$$$. Left cell: Fully-sampled non-noisy ground truth image, and mean reconstructed images with PSNR and SSIM measurements between mean and ground truth images. Right cell: Variance maps calculated across 100 Monte Carlo trials. With increased noise levels, a more uniform variability across different tissues is observed. Although the overall variability increases, variability differences between tissues are reduced.
