3642

Plug-and-Play Deep Learning Module for Faster Parallel MR Imaging
Kamlesh Pawar1, Gary Egan1, and Zhaolin Chen1
1Monash Biomedical Imaging, Monash University, Melbourne, Australia

Synopsis

Deep learning (DL) methods are superior to the conventional method of accelerated imaging such as parallel imaging and compressed sensing but the integration of DL methods into the MR scanners is still in its infancy. The integration of the DL methods into the MR scanner requires the design of new pulse sequences with a modified sampling patterns/trajectories and development of DL reconstruction framework within the MR scanner. In this work, we present an effective plug-and-play approach of integrating the DL reconstruction into the MR scanner that eliminates the need to modify the existing image acquisition and reconstruction pipeline.

Introduction

Current methods of applying DL to MR image reconstruction include - artifact image to clean image translation1,2 undersampled k-space to image translation3 and undersampled k-space to fully sampled k-space translation4. These DL based methods optimize both data acquisition and image reconstruction incorporating modification to pulse sequences for altered sampling patterns/trajectories. Recently, efforts have been made towards integrating parallel imaging (PI) with DL5,6 by incorporating SENSE7/GRAPPA8 as an additional layer. With these modifications, the DL reconstruction network can be trained in an end-to-end manner using k-space as input. Although DL methods improve the image reconstruction performance compared with the conventional methods of accelerated imaging such as PI and compressed sensing9, the integration of DL reconstruction methods into clinics is still in its infancy due to the technical challenge of implementing DL inside scanner environment. In this work, we propose a plug-and-play DL module to be used in combination with PI for post-reconstruction parallel imaging artifact correction. With high acceleration factors, parallel imaging results in noise amplification and residual aliasing artifacts. The post-reconstruction DL module is applied to remove these artifacts, that circumvents the need for pulse sequence modification and full integration of reconstruction software. The DL module assists existing image reconstruction pipeline to achieve high acceleration factors and reduce the scan time.

Methods

The design flow of the plug-and-play DL module is depicted in Figure 1(a), in which the parallel imaging acceleration factor for the vendor-provided sequences is increased. At high acceleration factors, the coil sensitivity information is not sufficient to reconstruct a clean image, and hence the reconstructed image includes noise amplification and aliasing artifacts in the image. A trained deep learning network is then plugged in to reduce the noise and remove the artifacts from these images.
To train a network that removes residual parallel imaging artifact, we simulated the GRAPPA artifact with an undersampling factor of 4 as depicted in Figure 1(b) and trained a Unet architecture. The knee k-space dataset used to train the network was obtained from the NYU fastMRI10 initiative database. The dataset consisted of 973 subjects for training and 199 subjects for validation. There were two contrasts one ‘with fat saturation’ and another ‘without fat saturation’. We trained a Unet on ‘without fat saturation’ dataset which consisted of 483 subjects for training and 100 subjects for validation. The number of slices for each subject varied from 30 to 38.
For experimental validation, a knee scan was performed on Siemens Skyra 3T using a turbo spin-echo sequence. The acquisition parameters were: TE/TR=31/2500 ms, echo train length = 4, phase encodes = 320, slices = 30, image matrix=320x320. Two scans were performed one without any parallel imaging and another with GRAPPA factor of 4.

Results

The images from the GRAPPA reconstruction and with the DL module were quantitatively compared using structural similarity (SSIM), peak signal to noise ratio (PSNR) and normalized mean squared error (NMSE) as shown in Figure 2. The mean SSIM index was improved from 0.7177 to 0.8910; the mean PSNR increased from 29.27 dB to 34.44 dB; the mean NMSE decreased from 0.1719 to 0.0935. All comparisons were statistically significant (p-value < 0.0001, paired t-test). Figure 3 shows a comparison of the image quality for different methods, where the first column shows the reconstruction from fully sampled k-space data, the second column shows GRAPPA reconstruction with an acceleration factor of 4 and the third column shows the GRAPPA images enhanced with DL. The GRAPPA reconstruction resulted in noise amplification and residual aliasing artifact as observed from the difference images and enlarged images. It can be seen in the enlarged images that in the GRAPPA reconstructed image the cartilage was not visible clearly while after processing through the DL network in the proposed manner the cartilage can be observed clearly. Moreover, the qualitative metric PSNR and SSIM were significantly improved while the NMSE was significantly reduced.
The proposed method was also validated in prospective under-sampling experiments whereby two scans were acquired firstly with no acceleration and secondly with an acceleration factor of 4. The result shown in Figure 4 demonstrates that the reconstruction from the vendor-provided GRAPPA reconstruction (second column) was improved with the proposed method (third column).

Discussion

Our results demonstrate that the plug-and-play DL module is effective in further reducing the reconstruction errors in parallel imaging when high acceleration factors are used. The proposed method is minimally disruptive to the existing imaging workflow since it works as an additional processing module outside reconstruction software. Furthermore, compared with the fully integrated reconstruction methods, the proposed method still relies on a PI method to reconstruct images and DL is only employed to remove residual artifacts and noise.

Conclusion

We have proposed a novel plug-and-play module for integration of DL reconstruction as a post-reconstruction technique which reduces the scan time, improves the image quality of the vendor-provided parallel imaging methods and facilitates the deployment of DL based image reconstruction in clinical applications.

Acknowledgements

Training data used in the preparation of this article were obtained from the NYU fastMRI Initiative database7. As such, NYU fastMRI investigators provided data but did not participate in analysis or writing of this report. A listing of NYU fastMRI investigators, subject to updates, can be found at: fastmri.med.nyu.edu.

References

  1. Hammernik K, Klatzer T, Kobler E, Recht MP, Sodickson DK, Pock T, Knoll F. Learning a variational network for reconstruction of accelerated MRI data. Magnetic resonance in medicine. 2018 Jun;79(6):3055-71.
  2. Pawar K, Chen Z, Shah NJ, Egan GF. ReconNet: A Deep Learning Framework for Transforming Image Reconstruction into Pixel Classification. Proc. Intl. Soc. Mag. Reson. Med. 2018: 574
  3. Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature. 2018 Mar;555(7697):487.
  4. Han Y, Sunwoo L, Ye JC. k-space deep learning for accelerated MRI. IEEE transactions on medical imaging. 2019 Jul 5.
  5. Zhou Z, Han F, Ghodrati V, Gao Y, Yin W, Yang Y, Hu P. Parallel imaging and convolutional neural network combined fast MR image reconstruction: Applications in low‐latency accelerated real‐time imaging. Medical physics. 2019 Aug 1.
  6. Sriram A, Zbontar J, Murrell T, Zitnick CL, Defazio A, Sodickson DK. GrappaNet: Combining Parallel Imaging with Deep Learning for Multi-Coil MRI Reconstruction. arXiv preprint arXiv:1910.12325. 2019 Oct 27.
  7. Pruessmann KP, Weiger M, Scheidegger MB, Boesiger P. SENSE: sensitivity encoding for fast MRI. Magnetic resonance in medicine. 1999 Nov 1;42(5):952-62.
  8. Griswold MA, Jakob PM, Heidemann RM, Nittka M, Jellus V, Wang J, Kiefer B, Haase A. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine. 2002 Jun;47(6):1202-10.
  9. Lustig M, Donoho D, Pauly JM. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine. 2007 Dec;58(6):1182-95.
  10. Zbontar J, Knoll F, Sriram A, Muckley MJ, Bruno M, Defazio A, Parente M, Geras KJ, Katsnelson J, Chandarana H, Zhang Z. fastmri: An open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.08839. 2018 Nov 21.

Figures

Figure 1: (a): The vendor-provided protocol is set to a high acceleration factor for GRAPPA/SENSE which results in residual aliasing and noise amplification (panel-a top image); this degraded quality image is feed to trained DL model for image enhancement that results in high SNR, artifact-free image (panel-a bottom image); (b): The process of generating trained model that will be used in the inference phase for image enhancement; a pair clean image and image with simulated GRAPPA artifact is generated and a DL model is trained using such pairs of images.

Figure 2: Boxplots comparing the quantitative scores between GRAPPA reconstructed image and image reconstructed using the proposed approach on validation dataset of 100 subjects for acceleration factor of 4; the mean SSIM index was improved from 0.7177 to 0.8910; the mean PSNR increased from 29.27 dB to 34.44 dB; the normalized mean squared error decreased from 0.1719 to 0.0935.

Figure 3: One representative image from the validation dataset (100 subjects). First column: Reference image without any artefact; Second column: Image reconstructed using GRAPPA with acceleration factor of 4 and 16 ACS lines; Third column: image reconstructed using the proposed system. The SSIM index, PSNR and NMSE is shown below the images. It is evident from the visual inspection of enlarged images and the quantitative scores that the proposed system is able to provide higher SNR and better structural information compared to the using only GRAPPA.

Figure 4: Experimental result from the Siemens 3T Skyra Scanner using a 15 channel Tx-Rx Knee coil. First column: Reference image obtained from fully sampled k-space data and using default reconstruction pipeline of the Siemens scanner; Second column: Image obtained from the scanner with GRAPPA factor 4 and ACS lines 24 using default reconstruction pipeline of the Siemens scanner; Third column: image enhanced using the trained DL model. The SSIM index, PSNR, and NMSE is shown below the images

Proc. Intl. Soc. Mag. Reson. Med. 28 (2020)
3642