Introduction

Deep Learning (DL) algorithms have been demonstrated to be effective in image reconstruction^1–3 from undersampled data and motion correction^4–6 from motion degraded MR images. Motion artifacts in MRI are one of the frequently occurring artifacts⁷ due to patient movements during scanning. The image reconstruction task involves removing undersampling artifacts such as noise, aliasing, and incoherent aliasing artifacts; while motion correction involves removing artifacts including blurring, ghosting, and ringing. Even though motion is present in approximately 20% of clinical cases⁷, it is not explicitly modeled in DL accelerated image reconstruction models. In this work, we propose a novel method of incorporating motion information during DL training for the task of accelerated MRI. We model motion as a tightly integrated auxiliary layer in the DL model during training that makes the DL model ‘motion aware’. During inference, the motion layer is removed and image reconstruction is performed from the raw k-space data.

Methods and Materials

Deep learning model:
Our model is built upon the end-to-end variational network¹ consisting of a cascade of convolutional neural network (CNN) driven reconstruction steps described as
$$ k^{t+1}=k^{t} - \eta^{t}M(k^{t}-\hat{k})+R^{t}(k^{t}) \quad [1]$$
where, $$$k^{t}$$$ is current k-space, $$$k^{t+1}$$$ is updated k-space, $$$\hat{k}$$$ is acquired k-space, $$$\eta^{t}$$$ is a learnable data consistency parameter, $$$M$$$ is sampling mask with ones at the sampling locations and $$$R^{t}$$$ is the reconstruction CNN (Figure.1e). Eq.1 is equivalent to one step of gradient descent. The reconstruction CNN proceeds as follows: (i) uses intermediate k-space, (ii) performs the inverse Fourier transform, (iii) combines the multi-channel images to a complex-valued image using the sensitivity maps estimated from the sensitivity maps estimation (SME) network (Figure 1f), (iv) the combined complex-valued image is processed through a Unet, (v) the processed image is converted back to multi-channel k-space and (vi) data consistency (DC) is enforced (Figure 1c). To make the variational network ‘motion aware’ we propose to incorporate a motion layer (MS) Figure 1d) and motion informed data consistency parameter estimator (MIDCP) (Figure 1b)

Motion simulation layer:
The motion simulation layer generates random motion parameters in three degrees of freedom, two translations parameters with a maximum of +/- 10 pixels, and one rotation parameter with a maximum of +/- 10⁰. The number of motion events for each image varied from 0 to 16 i.e a set of three motion parameters were generated randomly up to a maximum of 16 times and the k-space was distorted using these motion parameters. The motion layer was incorporated in the motion aware variational network (VarnetMi) as shown in Figure 1.

Motion informed data consistency parameter estimator (MIDCP):
As shown in Figure 1(a-b), the introduced MIDCP layer takes the intermediate k-space and learns a data consistency parameter using a CNN, thus Eq.1 is modified for VarnetMi as follows:
$$k^{t+1}=k^{t} - H^{t}(k^{t})\;M(k^{t}-\hat{k})+R^{t}(k^{t}) \quad [2]$$
where, $$$H^{t}$$$ is a CNN (Figure 1b) that takes k-space and predicts a single parameter for data consistency.

Training:
We used the fastmri dataset⁸ for training and validation of Varnet (not motion aware) and VarnetMi (motion aware). After the last iteration, the k-space was converted to root sum of squares (rss) image (Figure 1a) which was used to calculate the loss for supervised training and the loss function was difference structural similarity⁹.

Results

Experiments were performed using both with simulated motion and without motion images (T1, T2, T1-contrast, Flair) to compare the performance of Varnet and VarnetMi. When the motion was present in the input images with the undersampling factor of four, we observe a marked difference in the performance of the Varnet and VarnetMi in terms of quantitative scores with VarnetMi being superior. The reconstructed images using Varnet consisted of artifacts as shown in Figure 2, while VarnetMi being motion aware was able to recover images without any residual artifacts in the images. On the other hand, when there was no motion present (Figure 3) in the input images the performance difference between the Varnet and VarnetMi was negligible, with VarnetMi being slightly inferior. However, there was no visible artifact in both the reconstructions in this case (Figure 3). We also performed a quantitative group assessment on 1300 slices (different subjects) consisting of multiple contrasts as shown in Figure 4 which demonstrated that there was a consistent improvement for VarmetMi in terms of SSIM, PSNR, and NMSE when motion was present (Figure 4, top-row) while there was a negligible degradation in the quantitative scores when motion was not present (Figure 4, bottom-row).

Discussion

A novel DL based motion aware accelerated imaging method was presented which can reconstruct images in the presence of frequently occurring patient movement. The experiments suggest that the proposed method makes the DL reconstruction more robust for practical clinical purposes. A limitation of making the DL model motion aware includes minor degradation in the image quality for the proposed method when motion is not present. Future work will involve minimizing this limitation.

Conclusion

A robust motion aware variational network based deep learning reconstruction method for accelerated imaging was developed. The proposed method works both on motion degraded and non-motion degraded undersampled k-space, thus making it highly practical for clinical applications.

Acknowledgements

No acknowledgement found.