4773
Accuracy of Deep Learning-based Signal-to-noise Measurements using Air Recon DL
Evan McNabb1, Véronique Fortier1,2,3,4, and Ives R. Levesque3,4
1Medical Imaging, McGill University Health Centre, Montreal, QC, Canada, 2Diagnostic Radiology, McGill University, Montreal, QC, Canada, 3Gerald Bronfman Department of Oncology, McGill University, Montreal, QC, Canada, 4Medical Physics Unit, McGill University, Montreal, QC, Canada
1Medical Imaging, McGill University Health Centre, Montreal, QC, Canada, 2Diagnostic Radiology, McGill University, Montreal, QC, Canada, 3Gerald Bronfman Department of Oncology, McGill University, Montreal, QC, Canada, 4Medical Physics Unit, McGill University, Montreal, QC, Canada
Synopsis
Keywords: Data Acquisition, Data Analysis
Noise estimates in Deep Learning-based image reconstruction (DLR) from background regions lead to artificially low noise estimates and thus inaccurate SNR. Noise estimate accuracy was improved using the difference between two identical acquisitions. SNR increased in DLR compared to standard reconstruction in clinical sequences over a range of acquired signal averages and voxel sizes. With DLR, varying signal averages had little effect on the measured SNR in fast spin echo acquisitions, while SNR increased by more than three-fold in single-shot fast spin echo. However, with DLR, SNR no longer follows predicable behavior, therefore sequence optimizations need to be performed experimentally.Introduction
Deep Learning (DL)-based image reconstruction provided by vendors give clinicians the option to use data-driven reconstructions to improve image quality and lower scan time with higher signal-to-noise ratios (SNR) and resolution1–9. This facilitates access to DLR in facilities that do not otherwise have the necessary resources in place to manually perform DL-based reconstruction (DLR).SNR is an important measurement that has been used to quantify the impact of DLR compared to standard Fourier reconstruction1–10. SNR measurements take the mean signal intensity within a region of interest (ROI) divided by a noise estimate, typically the standard deviation (S.D) of an ROI placed in the background11. However, the use of parallel imaging, intensity filters, Partial Fourier acquisitions, and the lack of signal free background in the frequency-encoding direction often violates the assumed noise distribution11. Noise accuracy was shown to improve by measuring the voxel-wise S.D. acquired over many measurements of the same image, and to a good approximation, the S.D. in an ROI using the difference between two images11,12. The difference method is faster in terms of scan time but assumes that the difference image is zero-mean and normally distributed.
Previous SNR evaluations in DLR have relied upon background-based ROI methods3,4,7–9,13. The purpose of this work was to investigate the accuracy of these SNR methods in DLR. We also investigated whether the SNR in DLR follows predictable SNR behavior14 when the number of signal averages and resolution are modified in four commonly used pulse sequences.
Methods
All measurements were performed on a 1.5 T system (Artist, GE Healthcare) equipped with a 20-channel anterior and 40-channel posterior array coil provided by the manufacturer. All acquisitions were reconstructed twice, first using the standard reconstruction pipeline, and second using the commercial DLR algorithm (Air Recon DL).Images were acquired using a large ACR quality control phantom15. SNR measurements were initially performed on a T1-weighted Fast Spin Echo (FSE) acquisition (Table 1). The SNR was measured in three ways: SNRtemp which used 12 measurements and calculated the voxel-wise S.D. in a 190 cm2 ROI centered on a uniform region at the midpoint of the phantom in the superior-inferior direction, SNRdiff which measured the spatial S.D.in the same ROI from the difference of two identical images, and SNRbackground which used the spatial S.D. in a smaller 12 cm2 ROI placed outside the phantom, while avoiding the zero-pixel intensity regions (Figure 1).
SNR was then modified, first by increasing the number of signal averages, both on the console (online averaging) from 1 to 6, and in post-processing of the temporal images (image space averaging). Second, the acquisition matrices were incremented from 2562 to 5122. SNRtemp and SNRdiff were measured in four clinical sequences: T1w FSE, T1w Spoiled Gradient Echo (SPGR), T2w Single-shot FSE (SSFSE), and a Diffusion Weighted-EPI (DWI) with b=1000 s/mm2. Full acquisition parameters are listed in Table 1. For each sequence, SNR measurements were normalized to the standard reconstruction method with one signal average and a 2562 matrix.
Results
Noise estimates measured by SNRtemp and SNRdiff were equivalent for each of the standard reconstruction (Figure 1) and DLR (Figure 2) algorithms. SNRbackground in both cases underestimated noise estimates due the intensity correction of multichannel coil combinations. Similarly, in both standard and DLR, the distributions over the large spatial ROI demonstrated that the difference image was approximately zero-mean and normally distributed, satisfying the assumptions required by this method11. Noise estimates were significantly lower in DLR by a factor of 1.89 using SNRtemp and by a factor of 1.94 using SNRdiff.SNR measurements were higher in DLR when varying the number of averages compared to standard reconstruction (Figure 3). Using standard reconstruction, online signal averaging and image-based averaging followed the predicted SNR curve in all sequences. However, the SNR in DLR underperformed the predicted SNR increase in FSE and SPGR acquisitions. DLR in FSE acquisitions did not benefit from increased signal averaging as the normalized SNR varied from 2.0 to 2.9, below the predicted result. DWI demonstrated predicable increases in SNR with online signal averaging in both algorithms.
SNR did not decrease at the same rate in DLR compared to standard reconstruction when acquiring at higher image resolutions for FSE and SPGR acquisitions (Figure 4). Normalized SNR measurements in FSE and SPGR pulse sequences using DLR did not decrease as much as predicted, though trended downward towards as the acquisition matrix increased. Finally, in SSFSE acquisitions, a greater than three-fold increase in SNR was observed in DLR compared to standard reconstruction at all acquisition matrices.
Discussion and Conclusion
Images reconstructed with DL-based algorithms resulted in significantly lower noise estimates. Noise estimates from background-based ROIs lead to artificially low noise estimates and thus inaccurate SNR regardless of the reconstruction algorithm. Noise estimates and SNR measurements for both reconstruction algorithms can be reliably estimated with the difference method, though it doubles acquisition time. Furthermore, the results demonstrated that common image parameters that modulate SNR in well-established ways with standard reconstruction no longer hold in DLR, and that sequence optimizations need to be performed experimentally. SNR in the context of body imaging, where background ROIs are difficult to obtain, are currently being investigated.Acknowledgements
No acknowledgement found.References
1. Kim M, Kim HS, Kim HJ, et al. Thin-Slice Pituitary MRI with Deep Learning–based Reconstruction: Diagnostic Performance in a Postoperative Setting. Radiology. 2021;298(1):114-122. doi:10.1148/radiol.20202007232. Koch KM, Sherafati M, Arpinar VE, et al. Analysis and Evaluation of a Deep Learning Reconstruction Approach with Denoising for Orthopedic MRI. Radiol Artif Intell. 2021;3(6). doi:10.1148/ryai.2021200278
3. Mulé S, Kharrat R, Zerbib P, et al. Fast T2-weighted liver MRI: Image quality and solid focal lesions conspicuity using a deep learning accelerated single breath-hold HASTE fat-suppressed sequence. Diagn Interv Imaging. 2022;103(10):479-485. doi:10.1016/j.diii.2022.05.001
4. Zerunian M, Pucciarelli F, Caruso D, et al. Artificial intelligence based image quality enhancement in liver MRI: a quantitative and qualitative evaluation. Radiol Med. Published online September 7, 2022. doi:10.1007/s11547-022-01539-9
5. Ogawa R, Kido T, Nakamura M, et al. Reconstruction of cardiovascular black-blood T2-weighted image by deep learning algorithm: A comparison with intensity filter. Acta Radiol Open. 2021;10(9). doi:10.1177/20584601211044779
6. Park JC, Park KJ, Park MY, Kim M, Kim JK. Fast T2‐Weighted Imaging With Deep Learning‐Based Reconstruction: Evaluation of Image Quality and Diagnostic Performance in Patients Undergoing Radical Prostatectomy. Journal of Magnetic Resonance Imaging. 2022;55(6). doi:10.1002/jmri.27992
7. Kim M, Kim HS, Kim HJ, et al. Thin-Slice Pituitary MRI with Deep Learning–based Reconstruction: Diagnostic Performance in a Postoperative Setting. Radiology. 2021;298(1). doi:10.1148/radiol.2020200723
8. Kaniewska M, Deininger-Czermak E, Getzmann JM, Wang X, Lohezic M, Guggenberger R. Application of deep learning–based image reconstruction in MR imaging of the shoulder joint to improve image quality and reduce scan time. Eur Radiol. Published online September 27, 2022. doi:10.1007/s00330-022-09151-1
9. Gkotsis DE, Vlachopoulou A, Dimos K, Seimenis I, Despotopoulos E, Kapsalaki EZ. 2D-MRI of the Central Nervous System: The effect of a deep learning-based reconstruction pipeline on the overall image quality. Published online 2022. doi:10.48550/ARXIV.2206.01082
10. Lebel RM. Performance characterization of a novel deep learning-based MR image reconstruction pipeline. Published online August 14, 2020.
11. Dietrich O, Raya JG, Reeder SB, Reiser MF, Schoenberg SO. Measurement of signal-to-noise ratios in MR images: Influence of multichannel coils, parallel imaging, and reconstruction filters. Journal of Magnetic Resonance Imaging. 2007;26(2):375-385. doi:10.1002/jmri.20969
12. Constantinides CD, Atalar E, McVeigh ER. Signal-to-noise measurements in magnitude images from NMR phased arrays. Magn Reson Med. 1997;38(5):852-857. doi:10.1002/mrm.1910380524
13. Lee DH, Park JE, Nam YK, et al. Deep learning-based thin-section MRI reconstruction improves tumour detection and delineation in pre- and post-treatment pituitary adenoma. Sci Rep. 2021;11(1):21302. doi:10.1038/s41598-021-00558-2
14. BERNSTEIN MA, KING KF, ZHOU XJOE. CHAPTER 11 - SIGNAL ACQUISITION AND K-SPACE SAMPLING. In: BERNSTEIN MA, KING KF, ZHOU XJOE, eds. Handbook of MRI Pulse Sequences. Academic Press; 2004:367-442. doi:https://doi.org/10.1016/B978-012092861-3/50017-0
15. Price R, Allison J, Clarke G, et al. 2015 ACR MR QCManual Table of Contents. Published online November 2015.
Figures
Figure 1: Baseline noise estimates measured by the SNRbackground, SNRtemp, and SNRdiff in a large ACR phantom for standard reconstruction. Horizontal bars represent measured standard deviations over spatial ROIs. SNRbackground was Rayleigh distributed, however it underestimates the image noise due to intensity correction. The spatial distribution of the difference image was zero-mean normally distributed, and noise estimates from SNRdiff agreed with the expected value of SNRtemp (vertical bar).
Figure 2: SNRbackground, SNRtemp, and SNRdiff in DLR resulted in lower noise estimates compared to standard reconstruction by a factor of 1.9. Horizontal bars represent measured standard deviations over spatial ROIs. SNRbackground was Rayleigh distributed, however it underestimates the image noise due to intensity correction. The spatial distribution of the difference image was zero-mean normally distributed, and noise estimates from SNRdiff agreed with the expected value of SNRtemp (vertical bar).
Figure 3: SNR versus signal averages measured by the difference method in a uniform slice of the ACR phantom for (a) FSE, (b) SPGR, and (c) b-1000 DW images. Measurements were performed in standard (gray) and DL-based (blue) reconstruction algorithms. The modeled SNR (solid lines) was derived from the temporal method and extrapolated to increased averages. Image- and online based averaging followed the modeled SNR curve in standard reconstructions, but is sequence dependent in DL-based reconstructions.
Figure 4: SNR versus acquisition matrix measured using the difference method in a uniform slice of the ACR phantom for (a) FSE, (b) SPGR, and (c) SSFSE images. Measurements were performed in standard (gray) and DL-based (blue) reconstruction algorithms. The modeled SNR (solid lines) was derived from the temporal method and extrapolated to increased matrices.
Table
1: Sequence parameters for
both Standard and DL-based image reconstructions.
DOI: https://doi.org/10.58530/2023/4773