4177

Data-consistent super resolution for 3D whole-heart MRI using a motion-corrected deep-learning reconstruction framework
Andrew Phair1, Anastasia Fotaki1, Lina Felsner1, Thomas J. Fletcher1, René M. Botnar1,2,3,4,5, and Claudia Prieto1,3,4
1School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom, 2Institute for Biological and Medical Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile, 3School of Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile, 4Millennium Institute for Intelligent Healthcare Engineering, Santiago, Chile, 5Institute for Advanced Study, Technical University of Munich, Munich, Germany

Synopsis

Keywords: Image Reconstruction, Cardiovascular

Motivation: Whole-heart CMR with high isotropic spatial resolution involves long and unpredictable scan times.

Goal(s): To propose and validate a super-resolution motion-corrected reconstruction framework to enable accelerated high-resolution whole-heart CMR from lower-resolution acquisitions.

Approach: Low resolution was treated as a k-space down-sampling problem, enabling the adaptation of an end-to-end motion-corrected iterative deep-learning network reconstruction, previously demonstrated for undersampled whole-heart CMRA.

Results: High-resolution whole-heart images (1.5×1.5×1.5 mm3) were obtained from prospective low-resolution data (1.5×6×6 mm3) using the proposed Super-MoCo-MoDL framework, with comparable image quality to a high-resolution acquisition. Scan times decreased from ~3.2 to ~1.2 minutes and reconstruction times were clinically feasible, at ~30 seconds.

Impact: The proposed Super-MoCo-MoDL framework enables data-consistent 3D whole-heart image reconstruction at high isotropic resolution from lower-resolution anisotropic scans. It has the potential to either accelerate whole-heart CMR, increase the feasibility of high-resolution clinical scanning, or a combination of the two.

Introduction

Whole-heart cardiovascular magnetic resonance (CMR) is a principal imaging modality for the assessment and management of patients with congenital heart disease (CHD)1. Traditional acquisition techniques utilise diaphragmatic respiratory gating to avoid respiratory motion artefacts, leading to long and unpredictable scan times. Scans can be accelerated by combining undersampled k-space trajectories with iterative reconstruction techniques, although these are often slow and computationally expensive. Recently, various deep-learning reconstruction methods have been proposed to provide fast reconstructions of undersampled data2. Among these is the motion-corrected model-based deep-learning (MoCo-MoDL) framework3, which incorporates a motion estimation network and an image regularisation network trained end-to-end.

Alternatively, faster scans can be realised by acquiring at lower image resolution, corresponding to a smaller k-space field-of-view (FOV). Super-resolution (SR) techniques can be employed to up-sample the images to the desired higher resolution. Deep-learning networks have also been proposed to perform this task, by learning the relationship between low-resolution and high-resolution images in large sets of training data4-7. These methods are often image based, and thus fidelity to the acquired low-resolution k-space data is not guaranteed.

In this work, we propose to adapt MoCo-MoDL to generate super-resolved 3D whole-heart images with isotropic spatial resolution. k-Space data is acquired with low resolution in the phase-encode and slice directions, and scan times are ~1.2 minutes. By treating SR as a k-space down-sampling problem, data consistency (DC) is maintained in the output high-resolution 3D images.

Methods

45 adult patients with CHD were scanned on a 1.5-T scanner (MAGNETOM Aera, Siemens Healthcare) using an ECG-triggered free-breathing T2-prepared bSSFP sequence8, a 3D variable-density Cartesian spiral-like trajectory (VD-CASPR)9 to ensure incoherent undersampling artefacts and the following imaging parameters: FOV=400×300×72-108 mm3, flip angle=90°, T2-prep duration=40 ms, TE=1.75 ms, coronal orientation. A 2D image navigator (iNAV) was acquired at each heartbeat to enable respiratory binning. Each patient was scanned once at high isotropic resolution (1.5×1.5×1.5 mm3) with four-fold undersampling and again with four-fold lower resolution in two dimensions (sixteen-fold overall,1.5×6×6 mm3) and full k-space sampling (within an elliptical shutter).

The data were randomly sorted into a training set (40 patients) and a test set (5 patients). For training-set patients, the four-fold-undersampled high-resolution data were reconstructed using the non-rigid-motion-corrected patch-based low-rank method NR-PROST10 (Figure 1) to generate ground-truth respiratory-bin images. Corresponding low-resolution images were formed by cropping and zero-padding the $$$k_y-k_z$$$ plane, followed by a zero-filled reconstruction (Figure 1). Since the high-resolution data were undersampled using a VD-CASPR9 trajectory, the cropped centre was sufficiently sampled to allow retrospective undersampling to be applied to match the sampling density of the fully sampled low-resolution acquisitions (with elliptical shutter). For test-set patients, before input to the network, the low-resolution data were sorted into respiratory bins, zero-padded in k-space and underwent a zero-filled reconstruction.

Our proposed deep-learning reconstruction framework, which we call Super-MoCo-MoDL, is depicted in Figure 2. It consists of a diffeomorphic motion estimation network11, which estimates the motion fields between respiratory bins, and a motion-informed model-based iterative reconstruction network, which alternates between a DC term, which enforces DC in the acquired k-space centre, and a super-resolving U-Net. To improve convergence stability during training, an ADMM iterative scheme was employed for the iterative reconstruction, as opposed to the alternating minimisation used in MoDL12.

The framework was trained on a 16-GB GPU over 1200 epochs and tested on retrospective and prospective low-resolution data for the 5 test-set patients.

Results

Average acquisition times were ~1.2 minutes (low resolution, fully sampled) and ~3.2 minutes (high resolution, four-fold undersampled). Transverse slices of the 3D high-resolution cardiac images obtained with Super-MoCo-MoDL are presented for the five test-set patients in Figures 3 (retrospective) and 4 (prospective). Additionally, prospective coronal slices, where SR is applied in one dimension, are shown in Figure 5. For comparison, the same slices are presented for high-resolution ground-truth images, bicubic interpolations of low-resolution NR-PROST reconstructions and NR-PROST reconstructions of zero-padded low-resolution data.

Discussion

The image quality seen in Figures 3-5 suggests the Super-MoCo-MoDL framework achieves comparable results to the four-fold-undersampled high-resolution NR-PROST reconstructions, despite the ~2.6-fold faster acquisition times. Additionally, network inference took ~30 seconds per subject, a ~240-fold computational speedup relative to the iterative NR-PROST algorithm.

Future work will focus on generalising the framework for a range of image resolutions, fine-tuning the network hyperparameters and expanding the training set to improve robustness.

Conclusion

Super-MoCo-MoDL, a data-consistent super-resolution framework, was introduced to obtain 3D whole-heart CMR images for CHD patients with isotropic spatial resolution (1.5 mm3) from scans acquired at low $$$k_y-k_z$$$ resolution (1.5×6×6 mm3). Scan times were ~1.2 minutes and super-resolved reconstruction times were ~30 seconds.

Acknowledgements

The authors acknowledge financial support from: (1) King’s BHF Centre for Award Excellence PG/18/59/33955 and RG/20/1/34802, (2) EPSRC EP/V044087/1, EP/P001009/1, EP/P032311/1, EP/P007619, (3) Wellcome EPSRC Centre for Medical Engineering (NS/A000049/1), (4) Millennium Institute for Intelligent Healthcare Engineering ICN2021 004, FONDECYT 1210637 and 1210638, (5) IMPACT, Center of Interventional Medicine for Precision and Advanced Cellular Therapy, Santiago, Chile. ANID-Basal funding for Scientific and Technological Center of Excellence, IMPACT, #FB210024 (6) the Department of Health through the National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award, (7) NIHR Cardiovascular MedTech Co-operative, (8) the Technical University of Munich – Institute for Advanced Study and (9) the Government of Denmark.

References

  1. Fratz, S., Chung, T., Greil, G. F., Samyn, M. M., Taylor, A. M., Valsangiacomo Buechel, E. R., Yoo, S. & Powell, A. J. (2013). Guidelines and protocols for cardiovascular magnetic resonance in children and adults with congenital heart disease: SCMR expert consensus group on congenital heart disease. Journal of Cardiovascular Magnetic Resonance, 15(1), 1-26.
  2. Oscanoa, J. A., Middione, M. J., Alkan, C., Yurt, M., Loecher, M., Vasanawala, S. S., & Ennis, D. B. (2023). Deep learning-based reconstruction for cardiac MRI: A Review. Bioengineering, 10(3), 334.
  3. Qi, H., Hajhosseiny, R., Cruz, G., Kuestner, T., Kunze, K., Neji, R., Bonar, R. M. & Prieto, C. (2021). End‐to‐end deep learning nonrigid motion‐corrected reconstruction for highly accelerated free‐breathing coronary MRA. Magnetic Resonance in Medicine, 86(4), 1983-1996.
  4. Steeden, J. A., Quail, M., Gotschy, A., Mortensen, K. H., Hauptmann, A., Arridge, S., Jones, R. & Muthurangu, V. (2020). Rapid whole-heart CMR with single volume super-resolution. Journal of Cardiovascular Magnetic Resonance, 22(1), 1-13.
  5. Küstner, T., Munoz, C., Psenicny, A., Bustin, A., Fuin, N., Qi, H., Neji, R., Kunze, K., Hajhosseiny, R., Prieto, C. & Botnar, R. M. (2021). Deep‐learning based super‐resolution for 3D isotropic coronary MR angiography in less than a minute. Magnetic Resonance in Medicine, 86(5), 2837-2852.
  6. Qiu, D., Cheng, Y., & Wang, X. (2022). Dual U-Net residual networks for cardiac magnetic resonance images super-resolution. Computer Methods and Programs in Biomedicine, 218, 106707.
  7. Zhao, M., Wei, Y., & Wong, K. K. (2022). A generative adversarial network technique for high-quality super-resolution reconstruction of cardiac magnetic resonance images. Magnetic Resonance Imaging, 85, 153-160.
  8. Bustin, A., Ginami, G., Cruz, G., Correia, T., Ismail, T. F., Rashid, I., Neji, R., Botnar R. M. & Prieto, C. (2019). Five‐minute whole‐heart coronary MRA with sub‐millimeter isotropic resolution, 100% respiratory scan efficiency, and 3D‐PROST reconstruction. Magnetic resonance in medicine, 81(1), 102-115.
  9. Prieto, C., Doneva, M., Usman, M., Henningsson, M., Greil, G., Schaeffter, T., & Botnar, R. M. (2015). Highly efficient respiratory motion compensated free‐breathing coronary MRA using golden‐step Cartesian acquisition. Journal of magnetic resonance imaging, 41(3), 738-746.
  10. Bustin, A., Rashid, I., Cruz, G., Hajhosseiny, R., Correia, T., Neji, R., Rajani, R., Ismail, T. F., Botnar, R. M. & Prieto, C. (2020). 3D whole-heart isotropic sub-millimeter resolution coronary magnetic resonance angiography with non-rigid motion-compensated PROST. Journal of Cardiovascular Magnetic Resonance, 22(1), 1-16.
  11. Munoz, C., Qi, H., Cruz, G., Küstner, T., Botnar, R. M., & Prieto, C. (2022). Self-supervised learning-based diffeomorphic non-rigid motion estimation for fast motion-compensated coronary MR angiography. Magnetic resonance imaging, 85, 10-18.
  12. Aggarwal, H. K., Mani, M. P., & Jacob, M. (2018). MoDL: Model-based deep learning architecture for inverse problems. IEEE transactions on medical imaging, 38(2), 394-405.

Figures

Fig 1. Generation of training data. An iterative SENSE reconstruction of each respiratory bin was used to estimate non-rigid inter-bin motion fields, which were used in an NR-PROST reconstruction of the reference bin. This was then warped by the motion fields to provide a high-resolution ground-truth 3D image for each bin. Corresponding low-resolution bin images were formed by cropping and zero-padding the high-resolution undersampled k-space, and then applying a zero-filled reconstruction.

Fig 2. The Super-MoCo-MoDL framework. Low-resolution bin images are input to the motion estimation network, and the output motion fields are incorporated in the iterative motion-informed MoDL reconstruction which alternates between DC in the k-space centre and U-Net denoising. The training loss consists of a motion loss (difference between ground-truth images warped to the same bin + motion smoothness), a reconstruction loss (difference between output and ground truth) and network regularisation loss.

Fig 3. Transverse slices of the high-resolution (HR) ground-truth images (reconstructed with NR-PROST) (1st row), the zero-padded low-resolution (LR) network-input images (2nd row) and three SR methods applied to retrospective LR data (obtained by cropping the HR k-space) for the five test-set patients. The SR methods are: bicubic interpolation applied to a NR-PROST reconstruction performed at LR (3rd row), NR-PROST applied to zero-padded k-space (4th row) and the proposed Super-MoCo-MoDL method (5th row). Four-fold SR is applied in each of the dimensions pictured.

Fig 4. Transverse slices of the high-resolution (HR) ground-truth images (reconstructed with NR-PROST) (1st row), the zero-padded low-resolution (LR) network-input images (2nd row) and three SR methods applied to prospective LR data for the five test-set patients. The SR methods are: bicubic interpolation applied to a NR-PROST reconstruction performed at LR (3rd row), NR-PROST applied to zero-padded k-space (4th row) and the proposed Super-MoCo-MoDL method (5th row). Four-fold SR is applied in each of the dimensions pictured.

Fig 5. Coronal slices of the HR ground-truth images (reconstructed with NR-PROST) (1st row), the zero-padded LR network-input images (2nd row) and three SR methods applied to prospective LR data for the five test-set patients. The SR methods are: bicubic interpolation applied to a LR NR-PROST reconstruction (3rd row), NR-PROST applied to zero-padded k-space (4th row) and the proposed Super-MoCo-MoDL method (5th row). In this orientation four-fold SR is applied to the horizontal (left-right) direction only; the vertical (foot-head) direction corresponds to the readout direction.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
4177
DOI: https://doi.org/10.58530/2024/4177