4659

Parallel Imaging in Time-of-Flight Magnetic Resonance Angiography Using Deep Multi-Stream Convolutional Neural Networks

Yohan Jun¹, Taejoon Eo¹, Hyungseob Shin¹, Taeseong Kim¹, Hojoon Lee^2,3, and Dosik Hwang¹

¹Electrical Electronic Engineering, Yonsei University, Seoul, Korea, Republic of, ²Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea, Republic of, ³Department of Radiology, Inje University College of Medicine, Busan, Korea, Republic of

Synopsis

A deep parallel imaging network (“DPI-net”) was developed to reconstruct 3D multi-channel MRA from undersampled data. It comprises two deep-learning networks: a network of multi-stream CNNs for extracting feature maps of multi-channel images and a network of reconstruction CNNs for reconstructing images from the multi-stream network output feature maps. DPI-net was effective in reconstructing 3D time-of-flight MRA from highly undersampled multi-channel MR data, achieving superior performance, both quantitatively and qualitatively, over conventional parallel imaging and other deep-learning methods.

Introduction

A commonly used non-contrast-enhanced technique for MRA is 3D time-of-flight (TOF) MRA, which is used to diagnose vascular-related diseases such as intracranial vascular aneurysms, stenosis, and occlusion^1-3. However, 3D TOF sequences inevitably require an increased scan time to cover the large field of view and to provide detailed vascular anatomy information with high spatial resolution, and this potentially increases the risk of motion-related artifacts⁴. In recent years, several studies have applied deep-learning to processing medical images, especially for MR image reconstruction from undersampled k-space data, demonstrating better performance than with the conventional compressed sensing or PI-based methods even when the reduction factor was high^5-9. In this study, we have developed a 3D PI method (called “DPI-net”, Deep Parallel Imaging Network) based on 3D convolutional neural networks (CNNs) to reconstruct undersampled 3D TOF MRA data acquired from multiple channels.

Methods

Our purpose was to restore the undersampled multi-channel 3D MR images to a fully sampled 3D MR image using deep-learning networks. Thus, the objective function can be formulated as the following minimization equation:

$$\arg\min_{{\bf{\theta}}}\begin{Vmatrix}{{\bf{y}}-D_{H}({\bf{x}};{\bf{\theta}})}\end{Vmatrix}_2^2=\arg\min_{\theta}{\begin{Vmatrix}{{\sqrt{\sum_{c=1}^{N_{c}}{\mid{{{\bf I}_f^c}}\mid}^2}}-D_{H}(\begin{bmatrix}{\mid{{\bf{I}}_u^1}\mid},{\mid{{\bf I}_u^2}\mid},...,{\mid{{\bf{I}}_u^{N_{c}}}\mid}\end{bmatrix};{\theta})}\end{Vmatrix}}_2^2$$

where $$${\bf I}_f^c\in{C}^{{N_{k_{x}}{\times}N_{k_{y}}{\times}N_{k_{z}}}}$$$ is the fully sampled image of the $$$c$$$-th coil ($$$c=1,...,N_{c}$$$), $$${\bf I}_u^c\in{C}^{{N_{k_{x}}{\times}N_{k_{y}}{\times}N_{k_{z}}}}$$$ is the undersampled image of the $$$c$$$-th coil, $$${\bf{y}}={\sqrt{\sum_{c=1}^{N_{c}}{\mid{{{\bf I}_f^c}}\mid}^2}}\in{R}^{{N_{k_{x}}{\times}N_{k_{y}}{\times}N_{k_{z}}}}$$$ is the desired output (the square root of sum-of-squares (SSOS) of the fully sampled multi-channel 3D MR images), $$${\bf{x}}=\begin{bmatrix}{\mid{{\bf{I}}_u^1}\mid},{\mid{{\bf I}_u^2}\mid},...,{\mid{{\bf{I}}_u^{N_{c}}}\mid}\end{bmatrix}\in{R}^{{N_{k_{x}}{\times}N_{k_{y}}{\times}N_{k_{z}}}}$$$ is the magnitude of the undersampled multi-channel 3D MR images, and $$$D_{H}$$$ is the hypothesis function of the deep-learning network with parameters $$${\theta}$$$. The objective was to find $$${\theta}$$$ that minimized the $$${l_{2}}$$$ difference between $$${\bf{y}}$$$ and $$$D_{H}({\bf{x}};{\bf{\theta}})$$$ for the given training data set. DPI-net’s deep-learning architecture is based on fully CNNs and consists of two main networks: MS-net, comprising multi-stream CNNs for extracting feature maps of multi-channel images, and RC-net, comprising deep reconstruction CNNs for reconstructing the images. In this study, multi-stream CNNs architecture was introduced to address MR images acquired from multiple channels. Each channel’s undersampled MR image was fed in parallel as input into the multi-stream CNNs and output feature maps were obtained. Then, reconstruction CNNs processed the feature maps and the final MR image was reconstructed from the output of the reconstruction CNNs. The overall architecture is presented in Fig. 1.

MRI was performed using a 3.0T scanner with a 32-channel sensitivity-encoding head coil. Data for seven subjects were acquired using 3D TOF sequences, with three subjects’ data sets used for the training set, three for the test set, and one for the validation set. The parameters of 3D TOF sequence were as follows: TR, 20 ms; TE, 3 ms; flip angle, 18 degrees; matrix size, 432 $$${\times}$$$ 432; pixel resolution, 0.49 $$${\times}$$$ 0.49 mm²; slice thickness, 0.5 mm; and acquisition time, 11 min 51 s. Three slabs were acquired using 3D TOF sequences, each with 56 slices. A total of 120 slices were reconstructed from the three slabs, with the matrix size of the 3D volume image being 432 $$${\times}$$$ 432 $$${\times}$$$ 120. We retrospectively undersampled 3D TOF MRA k-space data for each slab with a variable-density Poisson-disk sampling pattern on $$$(k_{y},k_{z})$$$ domain such that the k-space data was fully sampled in the read-out direction $$$(k_{x})$$$. The reduction factors was R = 5.7.

Results

We compared DPI-net with the conventional methods SAKE¹⁰, ESPIRiT¹¹, and U-net¹². Fig. 2 shows fully sampled and reconstructed 3D TOF axial slices with magnified images and difference images produced by zero-filled, the three conventional methods, and DPI-net, with a reduction factor R = 5.7. DPI-net showed better performance than the conventional methods in recovering the vessel signals and removing aliasing artifacts. Sagittal, and coronal MIP images, fully sampled and reconstructed by the various methods with the reduction factor R = 5.7, are shown in Fig. 3, and Fig. 4, respectively. In the sagittal MIP images, DPI-net restored vessel signals lost in the zero-filled image, unlike the other methods (Fig. 3). In addition, DPI-net showed peripheral arteries as clearly as those in fully sampled. Similar results were seen with the coronal MIP images (Fig. 4). Fig. 5 shows the comparison of vessel sharpness for four vessels (right MCA, left MCA, right PCA, and left PCA) between the different reconstruction methods with the reduction factor R = 5.7. DPI-net showed significantly higher vessel sharpness than the other reconstruction methods in right MCA, left MCA, and left PCA.

Conclusion

DPI-net was effective in reconstructing 3D time-of-flight MRA from highly undersampled multi-channel MR data, achieving superior performance, both quantitatively and qualitatively, over conventional parallel imaging and other deep-learning methods.

Acknowledgements

This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP) (2016R1A2B4015016) and National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (2018M3C7A1024734).

References

[1] White PM, Teasdale EM, Wardlaw JM, Easton V. Intracranial aneurysms: CT angiography and MR angiography for detection—prospective blinded comparison in a large patient cohort. Radiology. 2001;219(3):739–749.

[2] Gibbs GF, Huston J, Bernstein MA, Riederer SJ, Brown RD. Improved image quality of intracranial aneurysms: 3.0-T versus 1.5-T time-of-flight MR angiography. Am J Neuroradiol. 2004;25(1):84–87.

[3] Weber J, Veith P, Jung B, et al. MR angiography at 3 Tesla to assess proximal internal carotid artery stenoses: contrast-enhanced or 3D time-of-flight MR angiography? Clin Neuroradiol. 2015;25(1):41–48.

[4] Wilson GJ, Hoogeveen RM, Willinek WA, Muthupillai R, Maki JH. Parallel imaging in MR angiography. Top Magn Reson Imaging. 2004;15(3):169–185.

[5] Yang G, Yu S, Dong H, et al. Dagan: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging. 2018;37(6):1310–1321.

[6] Kwon K, Kim D, Park H. A parallel MR imaging method using multilayer perceptron. Med Phys. 2017;44(12):6209–6224.

[7] Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med. 2018;79(6):3055–3071.

[8] Eo T, Jun Y, Kim T, Jang J, Lee HJ, Hwang D. KIKI‐net: cross‐domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn Reson Med. 2018. doi: 10.1002/mrm.27201.

[9] Hyun CM, Kim HP, Lee SM, Lee S, Seo JK. Deep learning for undersampled MRI reconstruction. Phys Med Biol. 2018;63:135007.

[10] Shin PJ, Larson PE, Ohliger MA, et al. Calibrationless parallel imaging reconstruction based on structured low‐rank matrix completion. Magn Reson Med. 2014;72(4):959–970.

[11] Uecker M, Lai P, Murphy MJ, et al. ESPIRiT—an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magn Reson Med. 2014;71(3):990–1001.

[12] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015. p. 234–241.

Figures

Figure 1. DPI-net’s deep-learning architecture. This comprises two networks: MS-net, a network of multi-stream convolutional neural networks (CNNs); and RC-net, a network of reconstruction CNNs. MS-net extracts the feature maps of multi-channel images and RC-net reconstructs images from the MS-net output feature maps.

Figure 2. Reconstruction results for 3D time-of-flight images reconstructed by fully sampled, zero-filled, SAKE, ESPIRiT, U-net, and DPI-net, with the reduction factor R = 5.7. (a) Axial slices. (b) Magnified images. (c) Difference images. The black and grey arrows indicate vessel signals. As indicated by the arrows, DPI-net shows better performance than the conventional methods in recovering the vessel signals.

Figure 3. Reconstruction results for 3D time-of-flight images reconstructed by fully sampled, zero-filled, SAKE, ESPIRiT, U-net, and DPI-net, with the reduction factor R = 5.7. (a) Sagittal maximum intensity projection images. (b) Magnified images. The black and grey arrows indicate blood vessels. As indicated by the arrows, DPI-net shows better performance than the conventional methods in recovering the vessel signals.

Figure 4. Reconstruction results for 3D time-of-flight images reconstructed by fully sampled, zero-filled, SAKE, ESPIRiT, U-net, and DPI-net, with the reduction factor R = 5.7. (a) Coronal maximum intensity projection images. (b) Magnified images. The black and grey arrows indicated blood vessels. The arrows show that SAKE, ESPIRiT, and U-net hardly restore blood vessels, whereas DPI-net recovers them.

Figure 5. Vessel sharpness comparison on four different vessels (right MCA, left MCA, right PCA, and left PCA) with different reconstruction methods including zero-filled, SAKE, ESPIRiT, U-net, and DPI-net with the reduction factor R = 5.7. Paired t-tests were used to calculate P-values. *P < 0.05, **P < 0.01, ***P < 0.001.

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)

4659