Physics-Based Deep Learning for CMR Image Reconstruction
Mehmet Akcakaya1
1University of Minnesota, United States

Synopsis

Deep learning (DL) techniques have emerged as a powerful reconstruction approach for high-quality accelerated MRI. Among these, physics-based DL reconstruction approaches, which incorporate the MRI encoding operator to solve a regularized least squares problem, have gained interest due to its improved generalization abilities. Our purpose is to look at physics-based DL methods in the context of CMR reconstruction.

Target Audience

Engineers and scientist interested in developing physics-based deep learning reconstruction techniques for accelerated cardiac MRI (CMR), and in understanding the nuances and implications of performing such acceleration.

Methods and Results

The canonical inverse problem in accelerated MRI is based on solving a regularized least squares objective function, consisting of a data consistency/fidelity term with acquired sub-sampled measurement and a regularizer. In conventional accelerated imaging, such regularizers include Tikhonov regularization in parallel imaging or l1-norm of transform domain coefficient in compressed sensing (1,2).

Recently, deep learning (DL) has gained interest for high-quality accelerated MRI. DL based MRI reconstruction algorithms can be roughly divided into two categories, purely data-driven and physics-based (3,4). In purely data-driven approaches, a mapping between the undersampled k-space/aliased image to full k-space/artifact-free image is learned (5-7). In the physics-based methods, the forward encoding operator, containing the undersampling pattern and the coil sensitivities, is incorporated to solve a regularized least squares problem (8-10). These techniques typically rely on algorithm unrolling (4), where an iterative algorithm for solving regularized least squares, which typically alternates between data consistency and a regularizer, is unrolled for a fixed number of iterations. In the unrolled networks, the proximal operation for the regularizer is implicitly solved using a neural network. Subsequently, these unrolled networks are trained end-to-end, either in a supervised manner using reference images (4) or more recently in unsupervised manners (11).

While physics-driven DL reconstruction has become popular in several anatomies, and outperforming data-driven methods in the fastMRI challenge (12), their application to CMR has only recently been explored. There are several reasons for this: 1) The data-driven methods have inherently very short reconstruction times, as they do not require data consistency operations, which is especially pronounced for non-Cartesian trajectories (13,14). 2) In most CMR applications, fully-sampled reference data is difficult to acquire due to physiological and scan time constraints. Thus, surrogate reconstructions, such as compressed sensing are used to generate reference data (13,14), which limits the appeal of physics-driven DL reconstructions as their performance will be inherently limited by the surrogate method, which is already implemented in a physics-driven manner. 3) Training and implementing physics-driven DL reconstructions for large-scale datasets and high-resolution non-Cartesian acquisitions have generally been difficult due to GPU memory constraints (15). However, recent advances in MRI reconstruction aim to tackle these challenges, offering new venues for the use of physics-driven DL reconstruction in CMR.

In this talk, we will first consider physics-driven DL methods for CMR applications in a supervised manner, when fully-sampled references are available, including cine imaging and coronary MRI (9,16,17,18), both with Cartesian sampling. We will then overview the use of physics-driven DL methods for non-Cartesian CMR (19), including recent advances in memory-efficient learning (15), enabling physics-driven mapping for 3D non-Cartesian coronary MRI (20). Then, we will overview unsupervised physics-driven DL reconstruction methods used in CMR (11, 21), and discuss their applications, such as late gadolinium enhancement and perfusion imaging (22). Finally, we will overview recent advances that may find applications in CMR, as well as challenges.

Acknowledgements

The author’s work is supported by NIH R01HL153146, NIH R21EB028369, NIH P41EB027061, NIH U01EB025144, and NSF CAREER CCF-1651825.

References

1. Lustig M, Donoho DL, Pauly JM. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn Reson Med 2007;58(6):1182-1195.

2. Block KT, Uecker M, Frahm J. Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint. Magn Reson Med 2007;57(6):1086-1098.

3. Liang D, Cheng J, Ke Z, Ying L. Deep MRI Reconstruction: Unrolled Optimization Algorithms Meet Neural Networks. arXiv preprint arXiv:1907.11711; 2019.

4. Knoll F, Hammernik K, Zhang C, Moeller S, Pock T, Sodickson DK, Akcakaya M. Deep-Learning Methods for Parallel Magnetic Resonance Imaging Reconstruction: A Survey of the Current Approaches, Trends, and Issues. IEEE Signal Processing Magazine; 2020. p 128-140.

5. Wang S, Su Z, Ying L, Peng X, Zhu S, Liang F, Feng D, Liang D. Accelerating magnetic resonance imaging via deep learning. IEEE 13th International Symposium on Biomedical Imaging (ISBI); 2016. p 514-517.

6. Lee D, Yoo J, Tak S, Ye JC. Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks. IEEE Trans Biomed Eng 2018;65(9):1985-1995.

7. Akcakaya M, Moeller S, Weingartner S, Ugurbil K. Scan-specific robust artificial-neural-networks for k-space interpolation (RAKI) reconstruction: Database-free deep learning for fast imaging. Magn Reson Med 2019;81(1):439-453.

8. Hammernik K, Klatzer T, Kobler E, Recht MP, Sodickson DK, Pock T, Knoll F. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 2018;79(6):3055-3071.

9. Schlemper J, Caballero J, Hajnal JV, Price AN, Rueckert D. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans Med Imaging 2018;37(2):491-503.

10. Aggarwal HK, Mani MP, Jacob M. MoDL: Model-Based Deep Learning Architecture for Inverse Problems. IEEE Trans Med Imaging 2019;38(2):394-405.

11. Akcakaya M, Yaman B, Chung H, Ye JC. Unsupervised Deep Learning Methods for Biological Image Reconstruction. IEEE Signal Processing Magazine 2022.

12. Knoll F, Murrell T, Sriram A, Yakubova N, Zbontar J, Rabbat M, Defazio A, Muckley MJ, Sodickson DK, Zitnick CL, Recht MP. Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge. Magn Reson Med 2020;84(6):3054-3070.

13. Fan LXZ, Shen DM, Haji-Valizadeh H, Naresh NK, Carr JC, Freed BH, Lee DC, Kim D. Rapid dealiasing of undersampled, non-Cartesian cardiac perfusion images using U-net. NMR in Biomedicine 2020;33(5).

14. Wang JY, Weller DS, Kramer CM, Salerno M. DEep learning-based rapid Spiral Image REconstruction (DESIRE) for high-resolution spiral first-pass myocardial perfusion imaging. NMR in Biomedicine.

15. Kellman, M., Zhang, K., Markley, E., Tamir, J., Bostan, E., Lustig, M. & Waller, L. Memory-Efficient Learning for Large-Scale Computational Imaging. IEEE Transactions on Computational Imaging 2020; 6:1403-1414.

16. Qin, C., Schlemper, J., Caballero, J., Price, A. N., Hajnal, J. V. & Rueckert, D. Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans Med Imaging 2019; 38:280-290.

17. Fuin N, Bustin A, Küstner T, Oksuz I, Clough J, King AP, Schnabel JA, Botnar RM, Prieto C. A multi-scale variational neural network for accelerating motion-compensated whole-heart 3D coronary MR angiography. Magn Reson Imaging 2020 Jul;70:155-167.

18. Küstner T, Fuin N, Hammernik K, Bustin A, Qi H, Hajhosseiny R, Masci PG, Neji R, Rueckert D, Botnar RM, Prieto C. CINENet: deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions. Sci Rep 2020 Aug 13;10(1):13710.

19. Malavé MO, Baron CA, Koundinyan SP, Sandino CM, Ong F, Cheng JY, Nishimura DG. Reconstruction of undersampled 3D non-Cartesian image-based navigators for coronary MRA using an unrolled deep learning model. Magn Reson Med 2020 Aug;84(2):800-812.

20. Zhang C, Piccini D, Demirel OB, Bonanno G, Yaman B, Stuber M, Moeller S, Akcakaya M. Distributed Memory-Efficient Physics-Guided Deep Learning Reconstruction for Large-Scale 3D Non-Cartesian MRI. Proc IEEE ISBI 2022.

21. Yaman B, Hosseini SAH, Moeller S, Ellermann J, Uğurbil K, Akçakaya M. Self-supervised learning of physics-guided reconstruction neural networks without fully sampled reference data. Magn Reson Med 2020 Dec;84(6):3172-3191.

22. Yaman B, Shenoy C, Deng Z, Moeller S, El-Rewaidy H, Nezafat R, Akcakaya M. Self-supervised physics-guided deep learning reconstruction for high-resolution 3D LGE CMR. Proc IEEE ISBI 2021:100-104.

Proc. Intl. Soc. Mag. Reson. Med. 30 (2022)