0010

Rapid Motion Estimation and Motion-Corrected End-to-End Deep Learning Reconstruction for 1 Heartbeat CINE
Thomas James Fletcher1, Lina Felsner1, Andrew Phair1, Gastão Cruz2, Haikun Qi3, René Botnar1,4,5,6,7, and Claudia Prieto1,5,6
1School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom, 2Department of Radiology, University of Michigan, Ann Arbor, MI, United States, 3School of Biomedical Engineering, ShanghaiTech University, Shanghai, China, 4Instituto de Ingeniería Biológica y Médica, Pontificia Universidad Católica de Chile, Santiago, Chile, 5Escuela de Ingeniería, Pontificia Universidad Católica de Chile, Santiago, Chile, 6Millenium Institute for Intelligent Healthcare Engineering iHEALTH, Santiago, Chile, 7Institute of Advanced Study, Technical University of Munich, Munich, Germany

Synopsis

Keywords: AI/ML Image Reconstruction, Cardiovascular

Motivation: Cardiac CINE provides dynamic images of the heart for morphology and function assessment. Single-heartbeat CINE enables faster acquisition times and the study of heart rate variations, but conventional reconstruction methods incur significant computational cost.

Goal(s): This study aims to speed up single-heartbeat CINE reconstruction by using deep learning reconstruction.

Approach: We propose a novel, rapid, end-to-end deep learning pipeline for motion estimation and motion-corrected single-heartbeat CINE reconstruction with golden-angle radial acquisition.

Results: The network reconstructs each CINE slice in ~40 seconds (400 times faster than state-of-the-art), with comparable image quality, achieving SSIM values ranging from 0.75 to 0.84 across cardiac phases and slices.

Impact: The proposed approach enables reconstruction of single-heartbeat golden-angle radial CINE acquisition in ~40 seconds, making it clinically feasible. Single-heartbeat CINE could reduce scan times, achieve acquisitions of multiple slices in a single breath-hold and be robust to heart rate variations.

Introduction

Cardiac CINE provides dynamic images of the heart, allowing for assessment of morphology and function. Recent work has demonstrated the feasibility of single-heartbeat CINE imaging by using a golden-angle radial acquisition and non-rigid motion-corrected reconstruction for each cardiac phase, combined with a patch-based regularisation (MC-CINE1). However, this reconstruction can take more than 4.5 hours per slice. Recently, deep learning (DL) has been proposed for motion estimation2 as well as motion-corrected model-based (MoCo-MoDL) reconstructions3,4,5,6. We propose a novel, rapid, end-to-end DL reconstruction framework for motion-corrected single-heartbeat golden-angle radial CINE. The proposed framework receives as input zero-filled images denoised with a U-Net for each cardiac phase. It then employs a RAFT network7,8 to estimate the non-rigid motion between all cardiac phases, and a MoCo-MoDL reconstruction network that incorporates data consistency and regularisation using a U-Net. The proposed approach is evaluated on prospectively acquired 26.9x undersampled data, achieving comparable results to the MC-CINE1 method, in just ~40 seconds.

Methods

We employ DL to estimate cardiac motion so that the reconstruction of each cardiac phase uses data from the entire cardiac cycle, leading to ~30x more k-space data in the motion-corrected data consistency step, and a better-posed inverse problem. The reconstruction step also utilises a denoising U-Net for regularisation.

Reconstruction network: the proposed end-to-end network (Figure 1) receives as input the denoised zero-filled images which are input into the RAFT model to provide motion estimates. These motion estimates are used to warp all cardiac phases to the phase currently being reconstructed and then these warped images are summed. This image is then passed through an iterative reconstruction block with each iteration consisting of a U-Net for regularisation followed by a motion-corrected data consistency step that accounts for the radial acquisition. The output of the final iterative step is the image reconstructed by the network.

Data: Data was prospectively acquired with a gold-angle radial trajectory and an acceleration factor of R=26.9 (1-heartbeat) for 14 healthy subjects (8 slices per volunteer) at 1.5 T (Philips, Ingenia). Main imaging parameters included: (FOV)=256x256, 8 mm slice thickness, resolution=2x2mm2, TE/TR=1.16/2.3 ms, b-SSFP readout, radial tiny golden angle of 23.6°, flip angle 60°, 448 radial spokes acquired, ~1 second acquisition.

Training: Training was performed with a 10:2:2 split of subjects across the training, validation, and test datasets. Coil sensitivity maps, zero-filled reconstructions and Toeplitz masks9 were pre-computed before training. A pre-trained denoising U-Net was utilized to improve the image quality of the zero-filled images for motion estimation with the RAFT network, to reduce the sensitivity of motion estimates to undersampling artifacts. The RAFT network was pre-trained separately, to estimate motion between cardiac phases, on 778 short-axis CINE slices from the M&Ms-210 dataset. Pre-training enabled the RAFT network to generalize to a wide range of cardiac motions as well as estimating motion to systolic phases, which were under-represented in our training dataset compared to diastolic phases.

The proposed network was trained in an end-to-end manner with all losses, except those of the pre-trained denoising network, being updated together at each training step. The proposed approach was evaluated on separate held-out test data in comparison to the reference CINEs reconstructed with MC-CINE1.

Results

Diastolic and systolic phases from the CINEs reconstructed with the proposed approach are shown for two subjects in Figures 2 and 3. Good quality images are reconstructed with the proposed approach, showing excellent agreement with the reference MC-CINE images, and fewer undersampling artifacts. Figures 2 and 3 also show the RAFT network preserves details in the motion estimates where edges in the input images are present, preventing blurring in the reconstructed images.

A reconstructed CINE is presented for a mid-short-axis slice for subject 2 in Figure 4 and systolic and diastolic phases for all 8 slices for subject 1 are shown in Figure 5. The Figures show that the network accurately reconstructs all cardiac phases from systole to diastole, and diminishes the effect of undersampling artifacts. The SSIM between the proposed and reference reconstructions for subjects 1 and 2 respectively are 0.79 ± 0.02 and 0.79 ± 0.03, across different cardiac phases and slices.

Each CINE slice was reconstructed in just ~40 seconds with the proposed approach compared to ~4.5 hours using the MC-CINE reconstruction (~400 times faster).

Conclusions

An end-to-end deep learning framework combining motion estimation and motion-corrected reconstruction was used to accurately reconstruct 1 heartbeat CINE data, in just ~40 seconds. Future work will investigate applying this framework to data covering the whole left ventricle, acquired in a single breath-hold, to save scan time and result in improved inter-slice registration.

Acknowledgements

The authors acknowledge financial support from: (1) BHF RG/20/1/34802 (2) EPSRC EP/V044087/1 (3) Wellcome EPSRC Centre for Medical Engineering (NS/A000049/1), (4) ANID Millennium Institute iHEALTH, ICN2021_004; Fondecyt 1210637 and 1210638; Basal Funding, IMPACT, FB210024 and (6) the Technical University of Munich – Institute for Advanced Study.

References

  1. Cruz, Gastao, et al. "Single‐heartbeat cardiac cine imaging via jointly regularized nonrigid motion‐corrected reconstruction." NMR in Biomedicine (2023): e4942.
  2. Pan, Jiazhen, et al. "Efficient image registration network for non-rigid cardiac motion estimation." Machine Learning for Medical Image Reconstruction: 4th International Workshop, MLMIR 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, October 1, 2021, Proceedings 4. Springer International Publishing, 2021.
  3. Aggarwal, Hemant K., Merry P. Mani, and Mathews Jacob. "MoDL: Model-based deep learning architecture for inverse problems." IEEE transactions on medical imaging 38.2 (2018): 394-405.
  4. Qi, Haikun, et al. "End‐to‐end deep learning nonrigid motion‐corrected reconstruction for highly accelerated free‐breathing coronary MRA." Magnetic Resonance in Medicine 86.4 (2021): 1983-1996.
  5. Yang, Junwei, et al. "End-to-End Deep Learning of Non-rigid Groupwise Registration and Reconstruction of Dynamic MRI." Frontiers in cardiovascular medicine 9 (2022): 880186.
  6. Pan, Jiazhen, et al. "Reconstruction-driven motion estimation for motion-compensated MR CINE imaging." arXiv preprint arXiv:2302.02504 (2023).
  7. Teed, Zachary, and Jia Deng. "Raft: Recurrent all-pairs field transforms for optical flow." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer International Publishing, 2020.
  8. Stone, Austin, et al. "Smurf: Self-teaching multi-frame unsupervised raft with full-image warping." Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition. 2021.
  9. Ahmad, R., C. D. Austin, and L. C. Potter. "Toeplitz embedding for fast iterative regularized imaging." Algorithms for Synthetic Aperture Radar Imagery XVIII. Vol. 8051. SPIE, 2011.
  10. C. Martín-Isla et al., "Deep Learning Segmentation of the Right Ventricle in Cardiac MRI: The M&Ms Challenge," in IEEE Journal of Biomedical and Health Informatics, vol. 27, no. 7, pp. 3302-3313, July 2023, doi: 10.1109/JBHI.2023.3267857.

Figures

Figure 1: Overview of the end-to-end framework for reconstructing CINE images. Zero-filled images are denoised using a pre-trained network (U-Net) and input into a RAFT model to estimate the motion between cardiac phases. The original zero-filled images and motion estimates are then fed into an iterative MoCo-MoDL based reconstruction step which includes a MoCo data consistency step and a U-Net for regularisation. The network is trained end-to-end, with motion, regularisation and reconstruction losses all contributing to network weight updates during training.

Figure 2: Mid-short-axis slice views at different cardiac phases for healthy subject 1 from the test dataset. The left column shows images from the CINE reconstructed by the neural network. The second column shows the reference images reconstructed using the conventional MC-CINE1 method. Difference maps are shown in the 3rd column while the estimated motion from an opposite part of the cardiac sequence to the current phase is shown in the 4th column.

Figure 3: Mid-short-axis slice views at different cardiac phases for healthy subject 2 from the test dataset. The left column shows images from the CINE reconstructed by the neural network. The second column shows the reference images reconstructed using the conventional MC-CINE1 method. Difference maps are shown in the 3rd column while the estimated motion from an opposite part of the cardiac sequence to the current phase is shown in the 4th column.

Figure 4: [Video - Click to open] Mid-short-axis CINE for healthy subject 2, showing the CINE reconstructed with the proposed network in the first column, the reference MC-CINE reconstruction in the second and a difference map in the third column. Finally, the estimated motion from a cardiac phase at a different point in the cardiac sequence to the current frame is shown.

Figure 5: Diastolic and systolic frames for healthy subject 1 for all 8 slices that were acquired. The first and fourth row show the images reconstructed with the proposed network. The second and fifth rows show the reference MC-CINE reconstructions. Lastly, the third and sixth rows show the difference maps between the two methods.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
0010
DOI: https://doi.org/10.58530/2024/0010