Synopsis

Keywords: Image Reconstruction, Artifacts, Deep Learning, Compressed Sensing , Parallel MRI

Most deep learning methods apply U-Net either in image or k-space domain. Nevertheless, these methods have limitations: (1) Directly applying U-Net in k-space domain is not optimal for extracting features; (2) conventional image-domain oriented U-Net does not fully utilize the information of encoder part of the network for extracting features in the decoder part. In this paper, a dual-domain deep learning-based approach is presented, incorporating multi-coil data consistency layers for the reconstruction of cardiac MR images from 1-D Variable Density (VD) under-sampled data. Experiments show superior reconstruction results of the proposed method than conventional Compressed Sensing (CS) method.

Introduction

Deep Learning-based techniques have recently gathered a lot of attention for their accuracy and efficiency in MR image reconstruction¹. Most deep learning reconstruction methods apply U-Net in image domain or in k-space domain². Conventional U-Net was originally designed for data in the image domain; directly applying the image-oriented U-Net in k-space data is not optimal for extracting features in the k-space domain. Further, classical image-domain U-Net fails to utilize information of the encoder part of the network for successful feature extraction in the decoder part of the network. To overcome limitations of conventional deep learning methods, dual-domain methods have been proposed recently³. These methods exploit the correlated information between dual-domains (image and k-space domains) to improve reconstruction performance. The dual-domain methods perform superior compared to single-domain methods owing to highly coupled information between the image domain and k-space domain^1-3. In this paper, a hybrid dual domain deep learning framework is proposed that learns the image reconstruction problem in both the image and frequency domains for a robust reconstruction of multi-coil under-sampled cardiac MRI data.

Methods

The proposed deep learning framework consists of an architecture which has been created by cascading two customized U-Nets (termed as IK-Net in this paper) as shown in Fig.1. It consists of 10 convolution layers (followed by Rectified Linear Unit (ReLU) and Batch Normalization (BN)) with a filter size of 3×3 and the last convolution layer (followed by BN) with a filter size of 1×1 besides max pooling and up sampling layers. The proposed framework consists of two subnetworks; named as ‘I-Net’ and ‘K-Net’. The first subnetwork (i.e. I-Net) is trained in image domain followed by Multi-coil Data Consistency (MCDC) operation; and then the output is fed into the second subnetwork (i.e. K-Net) after applying Fourier transform followed by the second layer of MCDC operation.
Block diagram of the proposed method (IK-Net) is shown in Fig. 2. In this paper, training dataset is extracted from the fully sampled, multi-slice, eight receiver coils (Cartesian) human cardiac data of fifteen patients⁶. The fully sampled human cardiac k-space data is VD under-sampled by an acceleration factor (AF) of 2 and 4 retrospectively; followed by an adaptive coil combination to get the composite under-sampled k-space data.
The IFFT of the composite under-sampled k-space data gives the aliased human cardiac images which are given as an input whereas the corresponding coil combined fully sampled human cardiac images are used as the ground truth for training the I-Net (sub network-1). The output of the trained I-Net is the reconstructed image which is given as an input to the MCDC⁷.
In MCDC operation, sensitivity maps obtained by the Walsh method⁹ are used to apply data consistency on the multi-coil interpolated images. These are later ‘coil combined’ to get the corrected composite images and then the fast Fourier transform is applied to achieve the corrected k-space data. The MCDC operation has been described in detail in Fig. 4.
The K-Net (subnetwork-2) is trained by using the corrected k-space data (obtained as an output of I-Net followed by MCDC operation) as an input and the corresponding fully sampled k-space data as the ground truth label. The MCDC operation is applied again on the output of K-Net, to get the corrected k-space data whose IFFT gives the final reconstructed image.
In our experiments, training of both the subnetworks (I-Net and K-Net) is performed separately on Python 3.7.1 by Keras using TensorFlow as a backend on Intel(R) core (TM) i7-4790 CPU, clock frequency 3.6GHz, 16 GB RAM and GPU NVIDIA GeForce GTX 780 for approximately 16 hours. We use RMSprop optimizer to minimize the loss function of mean square error.
The proposed method is tested on human cardiac data acquired from a 3T Siemens scanner⁸. The reconstruction results obtained from the proposed method are compared with the conventional Compressed Sensing (CS) reconstruction¹⁰.

Results

Fig.3 shows reconstruction results of the cardiac data obtained from the proposed method and CS along with the root mean square error (RMSE) and structure similarity index (SSI) values at AF= 2 and 4. SSI values of the reconstructed images obtained from the proposed method and CS are 0.9412 and 0.9105, respectively at AF=2. Similarly, RMSE values of the reconstructed images obtained from the proposed method and CS are 0.0102 and 0.0145, respectively; showing that the proposed method outperforms CS in reconstructing the under-sampled k-space data. Moreover, visual inspection of the results shows that the reconstruction results obtained from the proposed method are sharper as compared to the CS reconstruction.

Discussion and Conclusions

We propose a hybrid dual domain cascaded deep learning framework to reconstruct the human cardiac images from zero filled VD under-sampled k-space data. The proposed method surpasses the conventional CS method in reconstructing the cardiac images as indicated by their RMSE and SSI values as well as visual quality of the reconstructed images.

Acknowledgements

We are thankful to NVIDIA USA for providing Academic Hardware Grant for this research which includes 400 compute hours on V100 GPU instances via NVIDIA’s partner, Saturn Cloud.