1516
7T-like MR Images Synthesis from 3T MRI using Auto-Context Convolutional Neural Network1University of North Carolina at Chapel Hill, Chapel Hill, NC, United States, 2Department of Brain and Cognitive Engineering, Korea University, Seoul, Korea, Republic of
Synopsis
We propose a novel multi-step Convolutional Neural Network (CNN) architecture to cascade multiple CNNs, along with an Auto-Context Model (ACM), called Auto-Context CNN, to reconstruct 7T-like MR images from 3T MR images. Basically, we non-linearly map the input 3T MR images to their corresponding 7T MR images. To do so, in the training stage, we first partition the training 3T and 7T MR images into overlapping 3D patches, then we train the Auto-Context CNN to map each 3T patch to the center voxel in the corresponding 7T patch. In the testing step, we apply the trained Auto-Context CNN to generate the 7T-like MRI patch from each input 3T patch.
Purpose
7T MRI scanners provide images with much higher spatial resolution and tissue contrast in comparison to the 3T MRI scanners, which facilitates image-based disease diagnosis. However, there are still a very limited number of 7T MRI scanners worldwide. This motivates us to propose a method for reconstruction of 7T-like MR images from 3T MRI.Methods
We learn a mapping, from 3T MR images to 7T MR images, using both 3D CNN and Auto-Context Model (ACM), a shown in Fig.1, to generate from an input 3T MR image a high resolution 7T-like MR image, with quality similar to the ground-truth 7T MRI. Specifically, we propose an Auto-Context CNN, each CNN with four layers, as shown in Fig.2, which takes both the original 3T MRI and the reconstructed 7T-like image from the previous step as inputs to gradually refine the 7T-like MRI reconstruction. Note that this process represents the basic concept of Auto-Context Model, where both image appearance features from the source domain (i.e., the input 3T MRI) and context features from the target domain (i.e., reconstructed 7T-like MRI from the previous step) are used to boost up the performance of conventional CNN.Results
We compare our reconstructed 7T-like MR image numerically and visually with several reconstruction methods, including histogram-based method, LIS [1], MIMECS [2], M-CCA [3], RF [4] as well as the ground-truth 7T MRI. For evaluation, we use average peak-signal-to-noise ratio (PSNR) over 15 subjects. Fig.2 (a) compares the averaged PSNR using different methods. Also, we evaluate the influence of the reconstructed 7T-like image on post-processing steps, such as brain tissue segmentation, where brain tissues are separated into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) using FAST [5]. Fig.2 (b) shows the segmentation results for the 3T MRI, the reconstructed 7T-like MR images, and the ground-truth 7T MRI. The visual result reveals that, compared to 3T MRI, the reconstruction and segmentation of our reconstructed 7T-like image is much closer to the ground-truth 7T MRI.
Discussion
In this paper, we evaluated our method for reconstructing 7T-like images from 3T MR images and improving brain tissue segmentation. Of note, this work has several applications such as accurate segmentation of small brain regions, e.g. hippocampal subfields, which is very challenging in 3T MR images due to low resolution. Our method also has some weaknesses. For instance, it may fail in generating 7T-like MR images with any abnormality, if all the training 3T MR images are normal.Conclusion
We presented a novel Auto-Context CNN method for reconstructing 7T-like MR images from 3T MR images. We evaluated our proposed method for improving brain tissue segmentation of 3T MRI. Our visual and numerical results showed that our proposed method has superior result in 7T-like MR image reconstruction compared to other reconstruction methods. In addition, we achieved significantly higher accuracy of WM, GM, and CSF brain tissue segmentations by reconstructing 7T-like MR images. It is worth-noting that our reconstructed 7T-like MR images can be combined with any segmentation method for improving the accuracy of anatomical structures labeling.Acknowledgements
No acknowledgement found.References
1. Burgos N, Cardoso M J, Thielemans K, Modat M, Pedemonte S, Dickson J, Barnes A, Ahmed R, Mahoney C J, Schott J M, Duncan J S, Atkinson D, Arridge S R, Hutton B F, and Ourselin S. Attenuation Correction Synthesis for Hybrid PET-MR Scanners: Application to Brain Studies. IEEE Transactions on Medical Imaging, vol. 33, no. 12, 2014.
2. Roy S, Carass A, Prince J L, Magnetic Resonance Image Example Based Contrast Synthesis, IEEE Transactions on Medical Imaging, vol. 32, no. 12, pp. 2348-2363, 2013.
3. Bahrami K, Shi F, Zong X, Shin H W, An H, Shen D. Hierarchical Reconstruction of 7T-like Images from 3T MRI using Multi-Level CCA and Group Sparsity. MICCAI, 1-8, 2015.
4. Bahrami K, Rekik I, Shi F, Shen D. 7T-Guided Learning Framework for Improving the Segmentation of 3T MR Images. MICCAI, pp. 225-232, 2016.
5. Zhang Y, Brady M, Smith S. Segmentation of brain MR images through Markov random field model and the expectation-maximization algorithm. IEEE Transactions on Medical Imaging, vol. 20, no.1, pp. 45-57, 2001.
Figures
