2653
End-to-end Motion Corrected Reconstruction using Deep Learning for Accelerated Free-breathing Cardiac MRI1School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom, 2MR Research Collaborations, Siemens Healthcare Limited, Frimley, United Kingdom
Synopsis
A non-rigid respiratory motion-corrected reconstruction technique (non-rigid PROST) has achieved high-quality coronary MRA (CMRA). However, non-rigid PROST requires respiratory-resolved (bin) image reconstruction, bin-to-bin non-rigid registration and regularized reconstruction, leading to long computation time. In this study, we propose an end-to-end deep learning non-rigid motion-corrected reconstruction technique for highly undersampled free-breathing CMRA. It consists of a diffeomorphic motion estimation network and a motion-informed model-based deep learning reconstruction network that were trained jointly for motion-corrected undersampled reconstruction. Compared with non-rigid PROST, the proposed technique achieved better reconstruction performance in both retrospectively and prospectively 9x-accelerated CMRA, while operating orders of magnitude faster.
Introduction
Whole-heart coronary MR angiography (CMRA) is a promising technique for non-invasive assessment of coronary arteries. An accelerated and non-rigid motion-corrected (MoCo) reconstruction approach (non-rigid PROST) has been proposed to provide high-quality CMRA (1). However, this approach relies on respiratory-resolved (bin) image reconstruction, bin-to-bin non-rigid registration and regularized MoCo reconstruction, leading to long computation times. Respiratory-resolved images with sufficient image quality are required to reliably estimate the non-rigid inter-bin motion, which may limit the achievable acceleration factor of the CMRA acquisition. In this study, we propose an end-to-end deep learning framework for highly (9x) undersampled free-breathing CMRA. The proposed approach consists of a diffeomorphic motion estimation network (2,3) to estimate 3D non-rigid motion from highly undersampled (~22x) respiratory bin images and a motion-informed model-based deep learning (MoDL) (4) reconstruction network. Both networks are trained jointly for simultaneous non-rigid motion estimation and motion corrected regularized reconstruction (MoCo-MoDL).Methods
CMRA acquisition: Data was acquired in 17 healthy subjects and 17 patients referred to cardiac MRI, with 1.2mm3 isotropic free-breathing CMRA (5), which uses a VD-CASPR sampling trajectory (6) and acquires a 2D iNAV (7,8) in each heartbeat to estimate beat-to-beat 2D translational respiratory motion. After acquisition, the data was sorted into four respiratory bins according to the iNAV estimated respiratory motion curve followed by 2D translational intra-bin motion correction (1). MoCo-MoDL: The proposed end-to-end deep learning framework consists of a diffeomorphic non-rigid respiratory motion estimation network (RespME-net) (2) and a motion-aware MoDL reconstruction network (Fig. 1). RespME-net takes inputs of the undersampled respiratory bin images reconstructed using zero-filling (ZF) and outputs the non-rigid inter-bin motion, whereas MoDL takes inputs of the undersampled bin images, the predicted bin-to-bin motion and coil sensitivity maps, outputting the motion-corrected, dealiased CMRA. The motion-aware MoDL reconstruction equation is:$$\it z_k=D_{w}(x_k) (1a)$$
$$\it x_{k+1}=argmin\frac{1}{2}\sum_b||A_{b}W(x,u_{ref\rightarrow b})-y_{b}||_2^2+\frac{1}{2}\lambda||x-z_k||_2^2 (1b)$$
where k is the iteration number, x is the motion-free image, W(x,uref→b) warps x from reference bin to bin b with the forward motion uref→b predicted by RespME-net, yb is the binned k-space, Ab is the encoding operator, and z is the denoised version of x with Dw being the denoising network, λ is the learnable regularization parameter. Eq. (1a) and Eq. (1b) was optimized alternatively. Eq. (1b) is known as the data-consistency (DC) step, which can be solved by CG-SENSE to reduce the number of unrolled iterations (4), which was set to 3 in this work. Similar U-net architecture (9) was adopted for RespME-net and the denoising network. Training: Special efforts have been made to generate realistic training data with three major steps (Fig. 2): 1) Reconstruct reference bin image for 15 fully sampled (healthy subjects) and 15 2x-3x undersampled (patients) data using direct FFT or Wavelet-CS, respectively; 2) Generate fully sampled respiratory-resolved images by applying the realistic bin-to-bin respiratory motion to the reference bin, with the bin-to-bin motion estimated by binning the fully sampled or modestly undersampled data into 4 respiratory phases; 3) Simulate realistic undersampling by binning the 9x-accelearted VD-CASPR sampling based on the randomly selected iNAV motion curve. MoCo-MoDL was trained end-to-end with the training loss composed of motion estimation loss and reconstruction loss. The motion estimation loss is constructed to measure the difference between the warped moving images and reference image and the motion smoothness: $$$\it \sum_b||W(x_{gt}^b,u_{b\rightarrow1})-x_{gt}^1||_2^2+\alpha TV(u_{b\rightarrow1})$$$, where $$$\it x_{gt}^b$$$ is the ground truth image of bin b, ub→1 is the backward motion from moving bin b to the to the end-expiration reference bin (bin 1), TV means total variation and $$$\it \alpha$$$ is the regularization parameter. The reconstruction loss is the mean-squared-error between the network reconstruction and $$$\it x_{gt}^1$$$. To fit the GPU memory and extract large numbers of training samples, the input image is randomly cropped along the fully sampled readout dimension with size of 64 during training. Twenty-four randomly selected subjects were used for training and the remaining six subjects were used for validation. Evaluation: Prospectively 9x-undersampled CMRA acquisitions were performed in 2 healthy subjects and 2 patients with acquisition time of ~2.5mins. The non-rigid PROST reconstruction was performed as the baseline. For validation datasets where the ground truth is available, the PSNR and SSIM were computed in the heart region. For testing subjects, the right coronary artery (RCA), left anterior descending artery (LAD) were reformatted to measure the vessel sharpness.
Results
Representative validation results are shown in Fig. 3. MoCo-MoDL achieved higher PSNR (27.29±2.66) and SSIM (0.78±0.05) than non-rigid PROST (PSNR: 26.36±2.48; SSIM: 0.73±0.06). Both non-rigid PROST and MoCo-MoDL demonstrated good performance for the prospectively undersampled data (Fig. 4, 5). Improved vessel sharpness was observed for MoCo-MoDL (RCA: 58.6±13.2%; LAD: 55.1±10.7%) compared with non-rigid PROST (RCA: 52.8±10.1%; LAD: 48.1±10.6%). MoCo-MoDL has much shorter reconstruction time (~23s) than non-rigid PROST (~1h).Discussion
An end-to-end motion-corrected reconstruction network has been proposed for fast reconstruction of highly undersampled CMRA. The proposed method directly estimated motion from highly undersampled bin images without the need of additional efforts of reconstructing respiratory-resolved images. The predicted motion was then exploited in a motion-informed model-based deep learning network for simultaneous motion correction and undersampled reconstruction. Promising results were obtained from preliminary test data. Additional healthy subjects and patients will be recruited for further validation.Acknowledgements
No acknowledgement found.References
1. Bustin A, Rashid I, Cruz G, Hajhosseiny R, Correia T, Neji R, Rajani R, Ismail TF, Botnar RM, Prieto C. 3D whole-heart isotropic sub-millimeter resolution coronary magnetic resonance angiography with non-rigid motion-compensated PROST. J CARDIOVASC MAGN R 2020;22(1).
2. Qi H, Cruz G, Kuestner T, Prieto C, Botnar R. Unsupervised Learning-based Non-Rigid Registration for Fast Motion-compensated Coronary MR Image Reconstruction. 2020; Online. p 058.
3. Qi H, Fuin N, Cruz G, Pan J, Kuestner T, Bustin A, Botnar RM, Prieto C. Non-rigid Respiratory Motion Estimation of Whole-heart Coronary MR Images using Unsupervised Deep Learning. IEEE Trans Med Imaging 2020;PP.
4. Aggarwal HK, Mani MP, Jacob M. MoDL: Model-Based Deep Learning Architecture for Inverse Problems. IEEE Trans Med Imaging 2019;38(2):394-405.
5. Bustin A, Ginami G, Cruz G, Correia T, Ismail TF, Rashid I, Neji R, Botnar RM, Prieto C. Five-minute whole-heart coronary MRA with sub-millimeter isotropic resolution, 100% respiratory scan efficiency, and 3D-PROST reconstruction. Magn Reson Med 2019;81(1):102-115.
6. Prieto C, Doneva M, Usman M, Henningsson M, Greil G, Schaeffter T, Botnar RM. Highly efficient respiratory motion compensated free-breathing coronary MRA using golden-step Cartesian acquisition. J Magn Reson Imaging 2015;41(3):738-746.
7. Henningsson M, Koken P, Stehning C, Razavi R, Prieto C, Botnar RM. Whole-heart coronary MR angiography with 2D self-navigated image reconstruction. Magn Reson Med 2012;67(2):437-445.
8. Wu HH, Gurney PT, Hu BS, Nishimura DG, McConnell MV. Free-breathing multiphase whole-heart coronary MR angiography using image-based navigators and three-dimensional cones imaging. Magn Reson Med 2013;69(4):1083-1093.
9. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015; 2015; Cham. Springer International Publishing. p 234-241. (Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015).
Figures
