3630
Assessment of the Generalization of Learned Unsupervised Deep Learning Method1Research Center for Medical AI, Shenzhen Institutes of Advanced Technology，Chinese Academy of Sciences, Shenzhen, China, 2Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China, 3Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology，Chinese Academy of Sciences, Shenzhen, China, 4Department of Biomedical Engineering and Department of Electrical Engineering, The State University of New York, Buffalo, NY, United States
Synopsis
In our previous work, we proposed an unsupervised deep learning method for parallel MR cardiac imaging via time interleaved sampling. The comparisons with classical methods on in vivo data have shown that this method can achieve improved reconstruction results. However, the proposed unsupervised framework is based on the time interleaved sampling scheme. Does the model trained with time interleaved undersampling pattern have good generalization to other sampling patterns? In this paper, we will explore the generalization performance of the learned unsupervised deep learning method under different sampling patterns.
Introduction
Cardiac MR imaging is one of the most challenging MRI techniques because of the high clinical requirements for temporal and spatial resolution. In the past decade, the traditional optimization methods [1-3] based on sparse or low rank have achieved good success. However, some issues still remain such as difficult to choose the hyper-parameters, long iteration time and losing details. Recently, deep learning has been introduced to speed up dynamic MR reconstruction [4-6]. In our another work, we propose an unsupervised deep learning method for parallel MR cardiac imaging via time interleaved sampling. This unsupervised framework has several advantages such as no need for full-sampled dynamic MR data set, coil correlation can be explored, no need for coil sensitivity maps, elimination of time redundancy. However, the proposed unsupervised framework is based on the time interleaved sampling scheme. Does the network trained with time interleaved undersampling pattern have good generalization to other sampling patterns? In this work, we will explore the generalization performance of the learned unsupervised deep learning method under different sampling patterns.Method
The proposed unsupervised deep learning method for parallel MR cardiac imaging is shown in Figure 1. In particular, we designed a time interleaved acquisition scheme, which is shown in Figure 2, to build a set of full-encoded reference data by merging the adjacent time frames. Then these full-encoded data can be used to train a coil-by-coil parallel network to reconstruct image of each coil separately. The ADMM-III [7] was selected as the reconstruction network. Finally, the reconstructed images were combined together via another convolutional neural network. Although the training data sets are synthetic, they can effectively represent true sampled data and eliminate temporal redundancy.The comparisons with classical methods on in vivo data have shown that this method can achieve improved reconstruction results. However, the proposed unsupervised framework is based on the time interleaved sampling scheme. In this work, we will explore the generalization performance of the learned unsupervised deep learning method under different sampling patterns. Specifically, the network is trained under the time interleaved sampling scheme, and then tested in the 1D uniform sampling pattern and 1D random gaussian sampling pattern respectively. If good reconstruction performance can be obtained under both sampling patterns, we can agree that the proposed unsupervised model has good generalization capability.
Experiment and results
We collected 386 2DT (refer to 2D dynamic) full-sampled cardiac MR data from 30 healthy volunteers using 3T scanner (SIEMENS MAGNETOM Trio) with T1-weghted bSSFP sequence. After normalization and extraction, we got 2149 cardiac data for network training. For each of these, we retrospectively under-sampled it with 16 ACS lines according to the time-interleaved scheme. The models were implemented on an Ubuntu 16.04 LTS (64-bit) operating system equipped with an Intel Xeon E5-2640 Central Processing Unit (CPU) and a Tesla TITAN Xp Graphics Processing Unit (GPU, 12GB memory). The open framework Tensorflow was used.To demonstrate the generalization of the proposed unsupervised learning method, we compared it with k-t FOCUSS, k-t SLR, L+S under different sampling mask (1D Uniform sampling pattern, 1D random gaussian sampling pattern, R=4). We adjusted the parameters of the CS-MRI methods to their best performance. The proposed unsupervised learning method were trained on the time interleaved sampling pattern and tested on different sampling patterns. The reconstruction results are shown in Figure 3. The three CS-based reconstruction results contain less structural details and more artifacts, especially in the heart region, than the reconstructions of CNN-based methods. The y-t images, which are extracted from 124-th slice along the spatial and temporal dimensions, can also clearly illustrate the superior performance of the proposed method. As indicated by the red arrow, only the proposed method accurately exhibits the detail.
Conclusion
In this paper, we explore the generalization performance of the learned unsupervised deep learning method under different sampling patterns.The comparisons with classical k‐t FOCUSS, k‐t SLR, L+S on in vivo data under different sampling patterns show our method has good generalization performance. The good generalization performance is of great significance for the promotion of deep learning method to clinical practice.Acknowledgements
This research was partly supported by the National Natural Science Foundation of China (61601450, 61871371, 81830056, U1805261,61671441), Science and Technology Planning Project of Guangdong Province (2017B020227012, 2018B010109009), the Basic ResearchProgram of Shenzhen (JCYJ20180507182400762, JCYJ20150831154213680), Youth Innovation Promotion Association Program of ChineseAcademy of Sciences (2019351), and the Innovation and Technology Commission of the government of Hong Kong SAR (MRP/001/18X).References
[1]. Jung, Hong, Jong Chul Ye, and Eung Yeop Kim. "Improved k–t BLAST and k–t SENSE using FOCUSS." Physics in Medicine & Biology 52, no. 11 (2007): 3201.
[2]. Lingala, Sajan Goud, Yue Hu, Edward DiBella, and Mathews Jacob. "Accelerated dynamic MRI exploiting sparsity and low-rank structure: kt SLR." IEEE transactions on medical imaging30, no. 5 (2011): 1042-1054.
[3]. Otazo, Ricardo, Emmanuel Candès, and Daniel K. Sodickson. "Low‐rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components." Magnetic Resonance in Medicine 73, no. 3 (2015): 1125-1136.
[4]. J. Schlemper, J. Caballero, J.V. Hajnal, A. Price, D. Rueckert, “A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction”, IEEE TMI, DOI: 10. 1109/TMI.2017.2760978 (2017)
[5]. Qin, Chen, Joseph V. Hajnal, Daniel Rueckert, Jo Schlemper, Jose Caballero, and Anthony N. Price. "Convolutional recurrent neural networks for dynamic MR image reconstruction." IEEE transactions on medical imaging (2018).
[6]. Wang S, Ke Z, Cheng H, et al. DIMENSION: Dynamic MR imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training. NMR in Biomedicine. 2019;e4131. https://doi.org/10.1002/nbm.4131.
[7]. Cheng J, Wang H, Zhu Y, Liu Q, Ying L, Liang D. Model-based Deep MR Imaging: the roadmap of generalizing compressed sensing model using deep learning. arXiv preprint arXiv:1906.08143. 2019 Jun 19.
Figures
