Introduction

Model-based and Data-centric deep learning (DL) based models have shown promising results in accelerating Magnetic Resonance Imaging (MRI)^{[1,2, 4,8]}. However, DL architectures demand sufficient training data to learn the mapping function from undersampled aliased images to desired high-resolution images. This makes DL frameworks infeasible in cases where such high-resolution training data are not abundantly available. In this abstract, we present a new framework that exploits the low-level image features to substantially decrease the training data size for accelerated MRI.

Methods

Our framework is motivated by two fundamental principles: i) In the image domain, several images have common coarse-level image features such as smoothness, contrast, structures, and ii) most of the energy of MR images are often concentrated around low-frequency components of the k-space (around center k-space).
Deep Learning Model: To exploit these two principles and learn image features at two different scales coarse-scale and fine-scale corresponding to low frequency and high-frequency components of k-space we developed a multi-scale CNN-based unrolled DL framework as shown in Fig.1. Multi-scale DL architecture consists of two deep learning blocks Coarse-scale and Fine-scale blocks for which the inputs are correspondingly low-pass filtered and high-pass filtered images. We use a Gaussian kernel filter to minimize Gibb’s ringing. Each DL block consists of two ResNet ^[5] blocks which are made using two convolution layers and a relu activation unit as shown in Fig. 2. Outputs of each DL block are combined through a single CNN layer as shown in Fig. 1. Data consistency and coil sensitivity information are incorporated using unrolled architecture as described in ^[6,7,8,9]. Weighed $$$\ell_2$$$ loss is computed in the k-space such that $$$\cal{L}(k,\hat{k})=\omega_l* ||k_l-\hat{k}_l|| +(1-\omega_l)* ||k_h-\hat{k}_h||$$$, where $$$ k_l,\;k_h,\;\hat{k}_l\;and\;\hat{k}_h $$$, are low frequency and high-frequency k-space components of estimated and reference k-space corresponding to the low pass and high pass filters outputs, $$$\omega_l >0.5$$$ is a weighting factor which weights the low-frequency components heavily than the high-frequency components.
Implementation, Training Dataset and Transfer Learning: We implemented our framework in TensorFlow 1.10.1. Adam optimizer with β1 = 0.99, β2 = 0.99, learning rate α = 0.001 is used. Weight factor of $$$\omega_l=0.6$$$ was chosen during the training phase. Coarse scale DLN consists of F = 64 convolution kernel filters of size k1 = 8 × 8 at each convolution layer whereas, fine-scale DLN consists of F = 64 convolution kernel filters of size k2 = 2 × 2 at each convolution layer. For the unrolled network, Q = 2 numbers of proximal iterations were carried out. Training Data: The model was trained on the knee MRI dataset. The raw data were 32 coils, 3D volume data. The data was converted into 2 D image data by taking an inverse Fourier Transformation along the z-direction and coil compressed using [3]. As a result, 6400, 8-coil axial view knee MR images were obtained. The training data images were divided into 60-15-25% train, validation, and test sets, respectively. For training purposes, a total of 35, 2D variable density undersampling masks with different undersampling rates of 2, 3, 4, 6, and 9 were used. Investigating Transferability of the Model: To test the transferability of the model, we use two datasets. i) Coronal view knee images and ii) body MR images. In the case of coronal view reconstruction, we use the model trained in the axial view knee images directly and reconstruct (make inference directly) without any further training and new data. In the case of body MR images, we carry out two experiments. First out of ~5200 MR images from various anatomical regions, the data is divided into 60-15-25% train, validation, and test sets, respectively and the DL model is trained from scratch (Figure 4,5, case-a). In the second experiment, we infer reconstructions of undersampled test images using the model trained from scratch using only 1/3^rd of training data (Figure 4,5, case-c), and from the model initialized with weights learned using axial knee image training dataset and further trained on 1/3^rd of body image training data set (Figure 4,5,case-b) for 20,000 further iterations.

Results and Discussions

Figure 3 shows reconstruction results of coronal view knee images at undersampling rates of 6 and 9 using the single-scale unrolled network^[8,7] and multi-scale unroll network^[8] trained on axial view knee images. We can see that the reconstruction results from the multi-scale network are much cleaner and have lower RNMSE values confirming that the multi-scale network has better transferability for multi-view reconstruction. This is possible because the multi-scale network is able to capture smoothness of images more efficiently due to higher priority given to the coarse-scale image features. Figure 4 shows the reconstruction results for body MR test images at a reduction factor of 4. Figure 5 shows RNMSE values for two different test cases at different reduction factors. In both cases, we can see that the multi-scale network has a better transferability. In the case of body MR images, we can see that by transferring weights, the multi-scale network is able to achieve similar performance using only 1/3^rd of training data.

Conclusion

We investigated the transferability of multi-scale unrolled CNN architecture which is capable of learning corse scale image features.

Acknowledgements

This work is supported in part by the NIH R01EB009690, NIH R01 EB026136 grants, and GE Healthcare.