Exploiting deep convolutional neural network for fast magnetic resonance imaging
Shanshan Wang1, Zhenghang Su1,2, Leslie Ying3, Xi Peng1, and Dong Liang1

1Shenzhen Institutes of Advanced Technologies, Shenzhen, China, People's Republic of, 2School of Information Technologies, Guangdong University of Technology, Guangzhou, China, People's Republic of, 3Department of Biomedical Engineering and Department of Electrical Engineering, The State University of New York, Buffalo, NY, United States

Synopsis

This paper proposes a deep learning based approach for accelerating MR imaging. With the utilization of a large number of existing high-quality MR images, we train an off-line convolutional neural network (CNN) to identify the mapping relationship between MR images obtained from zero-filled and fully-sampled k-space data. Then the trained CNN is employed to predict an image from undersampled data, which is used as the reference in solving an online constrained imaging problem. Results on in vivo datasets show that the proposed approach is capable of restoring fine details and presents great potential for efficient and effective MR imaging.

INTRODUCTION

Signal processing based MR image reconstruction from reduced samples have played an essential role in accelerating MR scans in recent years [1,2]. A foundation for the success of these methods is the utilization of prior information on MR images, such as sparsity [3], low-rank [4], manifold fitting [5] and generalized series [6]. Nevertheless, despite all the achievements obtained by the existing methods, the exploited priors are still quite limited to the knowledge about the target image or very few reference images. Based on the observation that the anatomic structure of the same organ/tissue between different people are quite similar, we try to learn an off-line prior model to aid online fast imaging by taking advantage of the enormous images acquired each day. Specifically an off-line convolutional neural network is trained to describe the end-to-end mapping between zero-filled and fully-sampled MR images. The network is not only capable of restoring the details and fine structures of the MR images, but is also compatible with online constrained reconstruction model for efficient and effective MR imaging.

THEORY AND METHOD

The proposed method consists of two main parts: the off-line training and online imaging. In the off-line settings, consider T pre-acquired MR images $$$u_t$$$ reconstructed from fully-sampled data, we design and train an L-layer convolutional neural network $$$\left\{ \begin{array}{l} C_0=x \\ C_l=\sigma(W_l*C_{l-1}+b_l), l\in{1,2,...,L-1} \\ C_L=\sigma(W_L*C_{L-1}+b_L) \end{array} \right.$$$ by minimizing $$$\mathop{\rm argmin}_{{ \Theta}} \left\{\frac{1}{{\rm 2T}}\sum^T_{t=1}\|C(z_t;\Theta)-u_t\|_2^2\right\} $$$ where $$$z_t$$$ is the zero-filled MR image generated as the direct inverse of the undersampled data and $$$C$$$ means the end-to-end mapping function with hidden parameters $$$\Theta=\{(W_1,b_1),...,(W_l,b_l),...,(W_L,b_L)\}$$$. Once we learned the hidden parameters $$$\hat{\Theta}$$$ from the pre-acquired datasets, we can reconstruct MR images by considering the online constrained optimization problem $$$\mathop{\rm argmin}_{{u}} \left\{\|C(F^Hf;\hat{\Theta})-u\|_2^2+\lambda \|f-PFu\|_2^2\right\}$$$, where $$$f$$$ means the undersampled data, $$$P$$$ is the undersampling diagonal mask, $$$F$$$ denotes the full Fourier encoding matrix normalized as $$$F^HF=I$$$ and $$$H$$$ represents the Hermitian transpose operation. As we can see, the first term in the cost function promotes the similarity between the network prediction and the target image, and the second term enforces the data fidelity in k-space. As a simple least squares problem, we can give an analytical solution $$$Fu(k_x,k_y)=\left\{ \begin{array}{ll} S(k_x,k_y) &, ~{\rm if}~ (k_x,k_y)\notin \Omega \\ \frac{S(k_x,k_y)+\lambda S_0(k_x,k_y)}{1+\lambda} &, ~{\rm if}~ (k_x,k_y)\in \Omega\\\end{array} \right.$$$, where $$$S_0(k_x,k_y)= FF_M^Hf$$$, $$$S(k_x,k_y)=FC(F^Hf; \hat{\Theta})$$$; and $$$\Omega$$$ means the sampled locations.

EXPERIMENT

The training data consists of over 500 fully sampled MR brain images we collected from a 3T scanner (SIEMENS MAGNETOM Trio). Informed consent was obtained from the imaging subject in compliance with the Institutional Review Board policy. Undersampled measurements were retrospectively obtained using the 2D Poisson disk sampling mask. To increase the robustness of the proposed approach, we further generate more samples by separating the image pairs into 33×33 sub-image pairs, among which 90% are used for updating the network dataset and the rest 10% for validating the training process. We use three layers of convolution for the network. The first layer consists of 64 filters with the size of 9×9, while the second layer has 32 filters of size 5×5 and the last layer is one filter with size 5×5 . The filter weights of each layers are initialized by random values from a Gaussian distribution with zero mean and standard deviation 0.001. The bias is all initialized as 0. The training takes about three days, on a workstation equipped with 128G memory and a processor of 16 cores (Intel Xeon (R) CPU E5-2680 V3 @2.5GHz).We evaluated the proposed approach on a fully sampled transversal brain dataset which was acquired on a 3T scanner (SIEMENS MAGNETOM Trio) with a 12-channel head coil by T2-weighted turbo spin-echo (TSE) sequence (TE=91.0ms, TR=5000ms, FOV=20×20 cm, matrix=256×270, slice thickness=3mm) and a sagittal brain image which was acquired on a GE 3T scanner (GE Healthcare, Waukesha, WI) with a 32-channel head coil by 3D T1-weighted spoiled gradient echo sequence (TE=minimum full, TR=7.5ms, FOV=24 24cm, matrix=256×256, slice thickness=1.7mm). Undersampled measurements were retrospectively obtained using the 2D Poisson disk sampling mask.

RESULTS AND DISCUSSION

The first row of Fig. 1 presents the test results of the proposed method on the transversal brain at the acceleration of 5. It can be observed that the zero-filled images are very blurry with some details lost. After being put through the network, some fine structures and textures are restored and the noise is reduced. Further combining with the online constrained imaging model, we can reconstruct an image quite close to the reference one. For a close-up look, the zoom-in results have also been presented for the transversal brain. We also have presented the sagittal brain reconstruction at the acceleration of 3. According to Fig. 1k, we can see the difference image is noise-like and consists of only the contour information. There are no obvious details and structures lost. It demonstrates that the proposed network is capable of restoring the details and fine structures which are discarded in the zero-filled MR image. Furthermore, although the off-line training takes roughly three days, under the same GPU configurations, it takes far less than 1 second for each online MR reconstruction case.

CONCLUSION

We propose to design and train an off-line convolutional neural network to aid online fast MR imaging. Experimental results on two in vivo datasets have shown that the network is capable of restoring fine structures and details while removing noise, and have demonstrated great potential for efficient and effective MR imaging.

Acknowledgements

Grant support : China NSFC 61471350, the Natural Science Foundation of Guangdong 2015A020214019, 2015A030310314, the Basic Research Program of Shenzhen JCYJ20140610152828678, JCYJ20140610151856736 and the youth innovation project of SIAT under Y4G0071001 and US NIH R21EB020861 for Ying.

References

[1] Caballero J et al. ISMRM, p1560, 2014. [2] Ravishankar S et al., IEEE TMI 30(5):1028-1041, 2011. [3] Liu Q et al. IEEE TIP, 22(12):4652-4663, 2013. [4] Haldar J et al. ISBI, 857-860, 2011. [5] Shi C et al. ISBI, 901–904, 2015. [6] Liang Z et al. IEEE TMI, 22(8): 1026–1030, 2003.

Figures

Fig. 1 Visual quality comparison: (a) and (e) Reference images, (b) and (f) Zero-filled MR images, (c) and (g) Image prediction from the network, (d) and (h) Reconstructed results, (i) the 5x-acceleration mask for the transversal brain, (j) the 3x-acceleration mask for the sagittal brain, and k) reconstruction error of the sagittal brain



Proc. Intl. Soc. Mag. Reson. Med. 24 (2016)
1778