3977
Super resolution reconstruction in MRI by deep convolutional neural networks11. Columbia University, New York, NY, United States 2. New York State Psychiatric institute, New York, NY, United States, NY, United States, 21. Columbia University, New York, NY, United States 2. School of Science,Tianjin University, Tianjin, People's Republic of China
Synopsis
The sparse-representation-based super resolution is an efficient learning-based method. This method involves two key steps. One is to learn two dictionaries for low/high-resolution image patches, and the other is to learn a mapping between low resolution example patches and their corresponding high resolution patches from massive external images. We presented a super resolution method for MRI from reduced k-space acquisition sequences via deep convolutional neural networks. The proposed method directly learns an end-to-end mapping between the low/high-resolution images.Our proposed method is tested on the OpenfMRI database. It significantly outperforms the zero-filled reconstruction and an existing learning-based MRI SR method.
PURPOSE
Scan time and spatial resolution have been competing factors in the applications of MRI techniques. Many fast MRI techniques, such as parallel imaging, compressed sensing, and multi band imaging, have been developed to reduce scan time while keeping resolution or to increase resolution. Super-resolution (SR) MRI is another kind of technique that increases the spatial resolution after data acquisition and a variety of algorithms have been applied to MRI1,2. In the present work, we designed a neural networks model to directly learn the mapping between the low/high resolution images from the train image database. The proposed method significantly outperforms the zero-filled reconstruction and an existing learning-based MRI SR method. We call the algorithm as deep convolutional neural networks super resolution (DCNNMRI).METHODS
Algorithm The sparse coding based SR is one of the representative methods for learning-based image SR. Several deep neural networks models, motivated by the success achieved by DCNN for image restoration 3, were designed for SR of natural image that extended the sparse coding based SR method4, 5. According to the feature of undersampled MRI that the available partial data is in k-space rather than in image domain, we designed a model for deep convolutional neural networks to learn the mapping between the HR images and the zero filled LR images from the central k-space data which lacks the high frequency information. The network includes three convolutional layers to implement the following three operations: (1) patch extraction and feature representation for zero filled LR input image, (2) non-linear mapping between the features of low/high resolution images, and (3) reconstruction from the features of high resolution image. This network includes four layers (Figure 1). The first layer inputs the zero-filled image of MRI which has the same size as the unknown high resolution image. The next three layers are all convolutional layer with filter kernel weights and biases: one extracts features for LR image by filter kernel weights, one maps the feature vector of LR image onto another feature vector which is called as the HR feature image, and last one reconstructs the HR image by the HR feature vectors. Rectified linear unit (ReLU) is used as the activation function for the first two convolutional layers. ReLU function is defined as$$$ReLU(x)=max(0,x)$$$6.
Experiments We tested and evaluated the proposed method (DCNNSR)using high resolution T1-weighted imaging data in the OpenfMRI database 7. We firstly selected 200 images as training images taken from 61 to 70 frames of the first twenty subjects. The low resolution images were generated by acquiring the central k-space data with downsampling factor 2. Then, ten images from outside of the training set were selected for test images.For comparison, we implemented three representative MRI SR methods: Bicubic interpolation (BI), the zero-filled interpolation (ZF) and learning-based MRI (DLMRI) 8. We used peak signal to noise ratio (PSNR) to evaluate the quality of reconstructed SR image quantitatively. PSNR is defined as $$$10\times\log_{10}{\frac{(ab)^{2}}{MSE}}$$$，where$$$MSE$$$ is the mean square error between the original image and the generated super resolution image,$$$ a$$$ and$$$ b$$$ are the size of HR MRI.
Results
The results are showed quantitatively in Fig.2. We can observe the average PSNR values of BI, ZF, DLMRI and our method are 30.08 dB, 32.04 dB, 32.71 dB, and 33.56 dB, respectively. The improvements of average PSNR values are 3.48 dB, 1.52 dB and 0.85 dB compared with BI, ZF and DLMRI. An example is shown in Fig. 3 for visual inspection.Our method can recover more detailed structures than BI, DLMRI and ZF can.DISCUSSION &CONCLUSION
Compared with BI, ZF and DLMRI, the proposed method can achieve superior performance over the other two MRI SR methods. The reason lies in that DCNNMRI can learn the mapping between the input LR image and output HR one from end to end, and optimize all the parameters simultaneously. This mapping are represented a complex regression function trained by a DCNN. However, in the existing sparse coding based SR, the H/L resolution dictionaries and the mapping between the features of H/L patches are trained separately. The future research will focus on better using the characteristics of MRI and exploiting a more efficient structure of neural network to achieve the high frequency information to reconstruct SR-MRI from undersampled k-space data.Acknowledgements
No acknowledgement found.References
1. Reeth EV, Tham IWK, Tan CH, etal. Super-resolution in magnetic resonance imaging: A review. Concepts Magn Reson A. 2012, 40A(6): 306-325.
2. Plenge E, Poot DHJ, Bemsen M, etal. Super-resolution methods in MRI: can they improve the trade-off between resolution, signal-to-noise ratio, and acquisition time? Magn Reson Med. 2012, 68(6):1983-1993.
3. Jain V, Seung HS. Natural image denoising with convolutional networks. In: NIPS, 2008:769-776. 4. 4. Dong C, Loy C, He K and Tang X. Learning a deep convolutional network for image super resolution. In: ECCV, 2014:184-199.
5. Liu D, Wang Z, Wen B, Yang J, Han W and Huang T. Robust single image super-resolution via deep networks with sparse prior. IEEE Transactions on Image Processing, 2016, 25(7): 3194-3207.
6. Krizhevsky A, Sutskever B and Hilton GE. ImageNet classification with deep convolutional neural networks. In: NIPS 2012:1097-1105.
7. https://openfmri.org/dataset/ds000201
8. Ravishankar S and Bresler Y. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Transactions on Medical Image. 2011, 30(5): 1028–1041.
Figures
