4703
Multi-supervised Learning in Cross-domain Networks for Cardiac Imaging1Research center for Medical AI, Shenzhen Institutes of Advanced Technology, Shenzhen, China, 2Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Shenzhen, China, 3Department of Biomedical Engineering and Department of Electrical Engineering, The State University of New York, Buffalo, NY, United States
Synopsis
Dynamic MR image reconstruction from incomplete k-space data is an important technique for reducing its scan time. Deep learning has shown great potential in assisting this task. Nevertheless, most frameworks only adopt a final loss for network training and the intermediate results generated during the network forward pass haven't been considered for the network training. This work proposes a multi-supervised learning strategy, which constrains the frequency domain information and reconstruction results at different levels. Improved reconstruction results have been achieved with the proposed strategy.
Introduction
Deep learning has shown great potential in accelerating MR imaging. Compared to the classical compressed sensing or low rank based approaches, convolutional neural networks (CNNs) can reconstruct MR images with improved quality [1-7]. Despite all the successes, there are only two works that specifically apply to dynamic MR imaging [4][5]. Both works made great contributions to dynamic MR imaging. Nevertheless, the reconstruction results can still be improved since they just learn in image domain without combining frequency domain information sufficiently. Eo et al [7] proposed cross-domain convolutional neural networks (CNNs) for reconstructing 2D undersampled MR brain images termed as KIKI-Net. They demonstrate that the combination of K-Net and I-Net is superior to single-domain CNNs. This work proposes a novel multi-supervised learning strategy based on a cascaded cross-domain network for dynamic MR imaging. The multi-supervised learning strategy can guarantee that the frequency domain learning gets better completed k-space, and can also make the reconstruction results of different levels in the spatial domain learning closer to the full sampling MR images. The comparisons with k-t FOCUSS [8], k-t SLR [9], the state-of-the-art CNN method D5C5 [4] and the cross-domain network without multi-supervised learning on in vivo datasets show our method can achieve improved reconstruction results.Theory and method
In this work, we propose a novel multi-supervised learning based on a cascaded cross-domain neural network. The network architecture and the multi-supervised learning strategy are shown in Figure 1. The cross-domain network consists of two parts: the first part is frequency domain network for k-space completion termed as FDN; the second part is spatial domain network term as SDN, which is used to extract high-level features of images. The two are connected by Fourier inversion (see Inverse Fast Fourier Transform (IFFT) in Figure 1). The multi-supervised learning consists of two parts: progressive loss and primary loss. In the ordinary deep learning methods, there is only primary loss, while the proposed multi-supervised learning introduces progressive loss. The primary loss is the mean squared error (MSE) between the reconstructions and corresponding fully sampled images. The progressive loss is auxiliary loss include k-space loss and spatial loss. Let $$$k_u, k_c, k_f, x$$$ be the undersampled k-space, completed k-space, fully sampled k-space and fully sampled image respectively. Let $$$b_1,b_2,b_3,b_4$$$ be the output of block I1, block I2, block I3 and block I4 respectively. The total loss of the network training can be expressed as the formula: $$total\ loss=||x-b_4||^2_2+\sum_{i=1}^3\lambda_i||x-b_i||^2_2+\lambda_k||k_f-k_c||_2^2$$
(1) The first term is primary loss, the second term is spatial loss and the third term is k-space, where $$$\lambda_i$$$ and $$$\lambda_k$$$ are the weights of spatial loss and k-space loss respectively.
Experiment
We collected 101 fully sampled cardiac MR data using a 3T scanner (SIMENS MAGNETOM Trio) with T1-weighted FLASH sequence. Multi-coil data were combined to a single channel and then retrospectively undersampled using 1D random Cartesian masks [3]. After normalization and extraction, we got 17502 cardiac data, where 15000, 2000, and 502 were used for training, validating, and testing, respectively. The models were implemented on an Ubuntu 16.04 LTS (64-bit) operating system equipped with an Intel Xeon E5-2640 Central Processing Unit (CPU) and a Tesla TITAN Xp Graphics Processing Unit (GPU, 12GB memory). The open framework Tensorflow was used.Results and discussion
To demonstrate the efficacy of the multi-supervised learning, we compare it with k-t FOCUSS, k-t SLR, the state-of-the-art method D5C5, the cross-domain network without multi-supervised learning termed as CDN. The three CNN-based methods have the same amount of network parameters. And for fair comparison, the networks’ hyperparameters are also set to the same. We also adjust the parameters of the CS-MRI methods to their best performance. The reconstructions of these methods are shown in Figure 2. The k-t SLR removes artifacts better than the k-t FOCUSS. However, these two CS-based methods lose more structural details than the CNN-based methods. Compared with the D5C5 method and CDN, the proposed method can not only retain the details, but also remove the artifacts better. We show the evaluation indexes in Table 1. We observe the proposed cross-domain neural network achieves the optimal performance in MSE, PSNR and SSIM indexer among the compared methods.Conclusion
This paper proposed a multi-supervised learning strategy based on a cross-domain network for dynamic MR imaging. The multi-supervised learning strategy can guarantee that the frequency domain learning gets better completed k-space, and can also make the reconstruction results of different levels in the spatial domain learning closer to the full sampling MR images. The comparisons with k-t FOCUSS, k-t SLR, the state-of-the-art CNN method D5C5 and the cross-domain network without multi-supervised learning on in vivo datasets show our method can achieve improved reconstruction results.Acknowledgements
Grant support: China NSFC 81830056, 61771463, 61471350, Science and Technology Planning Project of Guangdong Province(2017B020227012), the Basic Research Program of Shenzhen JCYJ20150831154213680, and Key Laboratory for Magnetic Resonance and Multimodality Imaging of Guangdong Province 2017YFC0108802, and and US NIH R21EB020861 for Ying.References
[1]. S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng, D. Liang, “Accelerating magnetic resonance imaging via deep learning”, ISBI 514-517 (2016)
[2]. Kwon, Kinam, Dongchan Kim, and HyunWook Park. "A parallel MR imaging method using multilayer perceptron." Medical physics 44, no. 12 (2017): 6209-6224.
[3]. Han, Yoseob, Jaejun Yoo, Hak Hee Kim, Hee Jung Shin, Kyunghyun Sung, and Jong Chul Ye. "Deep learning with domain adaptation for accelerated projection‐reconstruction MR." Magnetic resonance in medicine 80, no. 3 (2018): 1189-1205.
[4]. J. Schlemper, J. Caballero, J.V. Hajnal, A. Price, D. Rueckert, “A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction”, IEEE TMI, DOI: 10. 1109/TMI.2017.2760978 (2017)
[5]. Qin, Chen, Joseph V. Hajnal, Daniel Rueckert, Jo Schlemper, Jose Caballero, and Anthony N. Price. "Convolutional recurrent neural networks for dynamic MR image reconstruction." IEEE transactions on medical imaging (2018).
[6]. Zhu, Bo, Jeremiah Z. Liu, Stephen F. Cauley, Bruce R. Rosen, and Matthew S. Rosen. "Image reconstruction by domain-transform manifold learning." Nature 555, no. 7697 (2018): 487.
[7]. Eo, Taejoon, Yohan Jun, Taeseong Kim, Jinseong Jang, Ho‐Joon Lee, and Dosik Hwang. "KIKI‐net: cross‐domain convolutional neural networks for reconstructing undersampled magnetic resonance images." Magnetic resonance in medicine (2018).
[8]. Jung, Hong, Jong Chul Ye, and Eung Yeop Kim. "Improved k–t BLAST and k–t SENSE using FOCUSS." Physics in Medicine & Biology 52, no. 11 (2007): 3201.
[9]. Lingala, Sajan Goud, Yue Hu, Edward DiBella, and Mathews Jacob. "Accelerated dynamic MRI exploiting sparsity and low-rank structure: kt SLR." IEEE transactions on medical imaging30, no. 5 (2011): 1042-1054.
Figures
