Introduction

Cardiac CINE MR imaging allows for accurate and reproducible measurement of cardiac function¹. Conventionally, multi-slice 2D CINE images are acquired under multiple breath-holds leading to patient discomfort and imprecise assessment. Parallel imaging^2,3 and compressed sensing (CS)^4-8 have enabled the reduction of scan time, but the possible acceleration remained limited.

Recently, deep learning-based MR image reconstruction methods showed potentials to further increase the acceleration rates while maintaining or improving spatial and temporal resolution. Image enhancement^9-11, physics-based unrolled learning^12-14, k-space learning^15,16 and hybrid learning^17,18 networks have been proposed. These networks either operate in image^9-14 or k-space^15,16 domain. Moreover, the spatio-temporal cardiac dynamics can be exploited using recurrent¹⁹, low-rank²⁰ or spatial-temporal filters^21,22. However, most approaches so far focused on either spatial reconstruction in one or both domains¹⁸, or dynamic reconstruction in a single domain^23-25.

In this work, we propose a novel complex-valued physics-based unrolled reconstruction network that a) operates in image and k-space domain and b) utilizes spatio-temporal filters for multi-coil dynamic data. An interleaved architecture with 2D+t filters is proposed to maximize the sharing of spatio-temporal k-space and image information. The proposed approach is investigated on in-house acquired 2D cardiac CINE MRI.

Methods

Our hybrid network adopts a mutual learning strategy to process the MR data in the k-space and image domain in parallel. The overall network architecture is depicted in Fig.1. The network consists of blocks with interleaved k-space and image subnetworks, and intermittent data consistency (DC) steps. The final reconstructed image is the output of the last image DC layer, but both outputs including k-space branch are evaluated in loss calculation, and thus impact back-propagation.

The image subnetwork (Fig.1b) is expressed as a residual 2D+t UNet. It has three encoding and decoding stages with two 2D+t complex-valued convolutional layers ($$$t \times x \times y=9\times5\times5$$$) in each stage and ModReLU activations between two convolutions as well as after the temporal convolution to enhance the network's representation power²¹. Downsampling is achieved by striding while upsampling is performed via transposed convolutions.
The k-space subnetwork (Fig.1c) is inspired by Robust artificial-neural-network for k-space interpolation (RAKI)¹⁶. It consists of three complex-valued convolutional layers with ModReLU activations that operate on the multi-coil k-t-coil data.

Within one iteration, the outputs of the individual image and k-space subnetworks are exchanged and combined with each other, i.e. for the image branch the k-space subnetwork output will be converted to the image domain through the multi-coil adjoint operator and added to the output of the image subnetwork.
A data consistency (DC) layer is applied to both k-space and image domain. The DC layer of the image branch is implemented by a gradient descent algorithm. The k-space branch performs a null-space projection such that sampled k-space locations will be replaced by acquired values. The image and k-space after DC layers are used as inputs for the next block. For six iterations, the overall network results in 2,512,350 trainable parameters.

2D cardiac CINE datasets were acquired in 104 subjects (66 patients and 38 healthy subjects) in-house with a bSSFP sequence (TE/TR=1.06/2.12ms, flip angle 52°, bandwidth=915Hz/px, spatial resolution 1.9$$$\times$$$1.9mm², slice thickness 8mm, temporal resolution $$$\approx$$$40ms, 25 cardiac phases) on a 1.5T MRI in eight breath-holds (14s each). Data were split into 83 training and 21 test subjects and retrospectively undersampled using VISTA²⁶ with acceleration factor R=2 to 15. The mean absolute errors (MAE) between reconstructed image/k-space and reference image/k-space are optimized by ADAM²⁷ (learning rate 10^-4 in the first 30 epochs and 10^-5 in another 30 epochs, batch size of 1). Impact of hybrid processing (hybrid vs. unrolled network) and spatio-temporal filters is investigated and the proposed approach is compared to a CS ($$$\ell_1$$$ wavelet regularized), UNet-based image enhancement⁹ and modified RAKI (our k-space subnetwork) reconstruction.

Results and Discussion

Animated Fig. 2 shows the cardiac time-resolved reconstructions by our proposed hybrid network for R=5, 10 and 15 retrospectively undersampled 2D CINE in one patient with cardiac lymphoma and potential myocarditis. A comparison to other reconstruction methods is shown in Fig. 3 (R=5) and Fig. 4 (R=15). The hybrid k-space and image network outperforms other reconstruction methods for the presented acceleration factors. Even for high accelerations that would enable single breath-hold imaging, the proposed hybrid network can successfully reconstruct the images with good quality and sharp details. Fig. 5 depicts the quantitative assessment (PSNR, SSIM, MSE) over all test subjects, slices and cardiac phases. The hybrid network significantly outperforms all other networks regarding PSNR, SSIM and MSE metrics.

We acknowledge limitations of this study. This work focused on a retrospective undersampling. In the future, we plan to acquire prospectively undersampled data within a single breath-hold. Furthermore, we will try to adapt the proposed approach for 3D cardiac CINE imaging.

Conclusion

Our novel hybrid multi-coil 2D+t deep learning-based reconstruction network can efficiently reconstruct highly undersampled 2D cardiac CINE with high image quality. The proposed hybrid network of interleaved and unrolled architecture with spatio-temporal kernels exploits the spatio-temporal redundancies in the k-space and image domain for dynamic image reconstruction which enables high acceleration rates.

Acknowledgements

The authors would like to thank Jens Kübler and Stanislau Chekan for study coordination. This project was supported by the Germany’s Excellence Strategy – EXC-Number 2064/1 – Project number 390727645 and EXC-Number 2180 – Project number 390900677.

References

1. American College of Cardiology Foundation Task Force on Expert Consensus Documents et al. Circulation 2010;121(22).
2. Kellman et al. Magn Reson Med 2008;59(4).
3. Kellman et al. Magn Reson Med 2009;62(6).
4. Chaâri et al. Med Image Anal 2011;15(2).
5. Block et al. Magn Reson Med 2007;57(6).
6. Osher et al. Multiscale Modeling and Simulation 2005;4(2).
7. Weller. IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI) 2016. 8. Liu et al. IEEE Trans Image Process 2013;22(12).
9. Hauptmann et al. Magn Reson Med 2019;81.
10. Kofler et al. IEEE Trans Med Imaging 2020;39(3).
11. Lee et al. IEEE Trans on Biomedical Engineering 2018;65(9).
12. Hammernik et al. Magn Reson Med 2018;79(6).
13. Schlemper et al. IEEE Trans Med Imaging 2018;37(2).
14. Aggarwal et al. IEEE Trans Med Imaging 2019;38(2).
15. Lee et al. Magn Reson Med 2016;76(6).
16. Akçakaya et al. Magn Reson Med 2019;81.
17. Eo et al. Magn Reson Med 2018;80(5).
18. El-Rewaidy et al. Magn Reson Med 2021;85(3).
19. Qin et al. IEEE Trans Med Imaging 2019;38(1).
20. Sandino et al. Proc Intl Soc Mag Reson Med 2021;29.
21. Sandino et al. Magn Reson Med 2021;85.
22. Küstner et al. Sci Rep 2020;10(1).
23. Biswas et al. Magn Reson Med 2019;82(1).
24. Hammernik et al. Proc ISMRM 27th Annu Meeting Exhibit 2019.
25. Qin et al. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 2019.
26. Ahmad et al. Magn Reson Med 2015;74(5).
27. Kingma et al. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, United States 2015.