3626
A Recurrent Neural Network (RNN) based reconstruction of extremely undersampled neuro-interventional MRI1Institute for Medical Imaging Technology, Shanghai Jiao Tong University, Shanghai, China, 2Functional Neurosurgery,Ruijin Hospital affiliated to Shanghai Jiao Tong University, Shanghai, China, 3Shanghai Jiao Tong University Medical School Affiliated Ruijin Hospital, Shanghai, CHINA, Shanghai, China, 4Beckman Institute for Advanced Science & Technology, Department of Electrical & Computer Engineering，University of Illinois at Urbana-Champaign, Urbana-Champaign, IL, United States
Synopsis
Real –time MR image-guided neurosurgery could greatly improve the surgery accuracy and outcome. However, real-time guidance requires highly accelerated imaging. In this study, we proposed a Convolutional Long Short-term Memory (Conv-LSTM) based U-net to reconstruct consecutive image frames with golden-angle sampling. The Conv-LSTM based architecture was developed to explore time coherence information. Training and test datasets were generated from MR images of patients treated with Deep Brain Stimulation (DBS). Results showed that our model could achieve an acceleration rate ~80x, which provided great potentials for application in MR-guided interventional therapy.
Introduction
Interventional MRI (I-MRI) could provide exceptional advantages to other imaging modalities in neurosurgery applications such as Deep Brain Stimulation (DBS)1. For real-time visualization of the interventional process, I-MRI requires rapid image acquisition and reconstruction. Many methods have been developed to improve both temporal and spatial resolutions2,3. Compressive sensing (CS)4,5 as well as deep learning techniques are among the most used in fast MR imaging6,7. However, most of the methods were not tailored for I-MRI thus may not satisfy the temporal and spatial requirements of I-MRI. In this study, using radial sampling scheme, we proposed an end-to-end recurrent network to reconstruct I-MRI by utilizing the time coherence information between consecutive frames. Results were compared with that from NUFFT and Golden-Angle Radial Sparse Parallel MRI (GRASP).Methods
A total of 29 patients were scanned before and after DBS surgery at Ruijin Hospital in Shanghai, China. Both T1-weighted and T2-weighted, whole brain, pre- and post-operative images were acquired for each patient. Among all the patient data, 23 were for training and 6 for testing. To prepare the training data set, pre-operative reference images and post-operative images with interventional features were registered (Figure 1a). After registration, the interventional feature was extracted for simulation (Figure 1b). The golden-angle sampling pattern was tested. A total of 400 spokes were collected with 256 readout points for each spoke. Only 5 spokes were used to reconstruct each interventional image frame.Our proposed model is based on a U-net architecture with inserted LSTM blockfor utilizing the time coherence between different frames. The reference scan xref was used to initialize the Conv-LSTM and the interventional image xtunI were reconstructed by the RNN-based algorithm (Figure 2). For each Conv-LSTM Block at BtunI time t, the internal states ctk and htk automatically updated given a new observation xtk. The inference process could be formulated as follows: $$\text{x}_{\text{k}}^{\text{t}}=\text{Deconv}(\text{h}_{\text{k-1}}^{\text{t}})$$ $$\text{c}_{\text{k}}^{\text{t}},\text{h}_{\text{k}}^{\text{t}}=\text{B}_{\text{k}}^{\text{t}}(\text{x}_{\text{k}}^{\text{t}},\text{c}_{\text{k}}^{\text{t-1}},\text{h}_{\text{k}}^{\text{t-1}})$$ $$\text{Initialized with }\text{x}_{\text{1}}^{\text{t}}=\text{Encoder}(\text{x}_{\text{unl}}^{\text{t}})\text{ and }\text{c}_{\text{k}}^{\text{1}}\text{,}\text{h}_{\text{k}}^{\text{1}}=\text{Initializer}$$
And the loss is adapted from DAGAN6, which contains a pixel-wise image domain mean square error MSE loss, a frequency domain MSE loss, a perceptual VGG loss and the adversarial loss of the generator.$$L_{total}=αL_{iMSE}+βL_{fMSE}+γL_{VGG}+L_{GEN}$$We tested our models on 188 images generated from 6 patients. To prove the feasibility of proposed LSTM blocks, we compared the results of LSTM blocks masked and unmasked. In the case where the LSTM blocks were masked, a DAGAN architecture was retained. The reconstruction results were also compared with NUFFT and GRASP in terms of PSNR and computational time.
Results
Reconstruction results showed that both NUFFT and GRASP could not reconstruct the interventional image using 5 sampled spokes. Although the interventional feature could be reconstructed with LSTM blocks masked, the position of the feature was not clear and the PSNR was inferior to the proposed method (Figure 3). The reconstruction time was about 23.5ms per frame (Tesla P100-PCIE-16GB).Discussion and conclusions
In this study, a RNN-based reconstruction algorithm was proposed for reconstruction of brain interventional images. We demonstrated the algorithm could reconstruct an image with only 5 spokes of golden-angle sampled k-space data. This is especially helpful for application in neuro interventions such as biopsy or DBS. Compared with the work by Carles Ventura and Miriam Bellver et al. (2019), who extended the recurrent network to find both the spatial and temporal coherence for video object segmentation8, we utilized the reference information before intervention. Future work will explore the possibility of applying the proposed algorithm to consecutive interventional images in 3D cases.Acknowledgements
Authors would like to thank Dr. Qun Chen from UIH for facility support. Funding support from grant 31870941 from National Natural Science Foundation of China (NSFC), grant 1944190700 from Shanghai Science and Technology Committee (STCSM) are acknowledged.References
1.Peters, Terry, and Kevin Cleary, eds. Image-guided interventions: technology and applications. Springer Science & Business Media, 2008.
2.Campbell-Washburn, Adrienne E., et al. "MR sequences and rapid acquisition for MR guided interventions." Magnetic resonance imaging clinics of North America 23.4 (2015): 669.
3.Feng, Li, et al. "Golden‐angle radial sparse parallel MRI: combination of compressed sensing, parallel imaging, and golden‐angle radial sampling for fast and flexible dynamic volumetric MRI." Magnetic resonance in medicine 72.3 (2014): 707-717.
4.Christodoulou, Anthony G., et al. "High-resolution cardiac MRI using partially separable functions and weighted spatial smoothness regularization." 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. IEEE, 2010.
5.Song, Hee Kwon, and Lawrence Dougherty. "Dynamic MRI with projection reconstruction and KWIC processing for simultaneous high spatial and temporal resolution." Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 52.4 (2004): 815-824.
6.Yang, Guang, et al. "DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction." IEEE transactions on medical imaging 37.6 (2017): 1310-1321.
7.Schlemper, Jo, et al. "A deep cascade of convolutional neural networks for dynamic MR image reconstruction." IEEE transactions on Medical Imaging 37.2 (2017): 491-503.
8.Ventura, Carles, et al. "Rvos: End-to-end recurrent network for video object segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.
Figures
