4179
A Self-Consistency Guided Multi-Prior Deep Learning Framework for Reconstruction of Fast Spatiotemporal MRI and Its Applications in Cardiac MRI1Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China
Synopsis
Keywords: Image Reconstruction, Heart
Motivation: Cardiac MRI (CMR) is widely used for assessment of cardiac diseases, but long acquisition time can cause patient discomfort and motion artifacts. Existing methods face challenges in reconstructing detailed information from highly undersampled spatiotemporal CMR acquisitions.
Goal(s): We propose a self-consistency guided multi-prior deep learning framework termed $$$k$$$-$$$t$$$ CLAIR to address this challenge.
Approach: This method exploits spatiotemporal correlations in data and incorporates calibration information to learn complementary priors across the $$$x$$$-$$$t$$$, $$$x$$$-$$$f$$$, and $$$k$$$-$$$t$$$ domains.
Results: Evaluation performed on publicly available cardiac cine and T1/T2 mapping datasets demonstrated that the proposed method can effectively reconstruct detailed information from highly undersampled CMR data.
Impact: The proposed method achieves high-quality reconstruction of highly undersampled CMR datasets including both cine imaging and T1/T2 mapping. This method has potential to improve CMR in clinical use.
Introduction
Cardiac MRI (CMR) is widely used for diagnosing cardiac diseases, with cine imaging employed for cardiac function assessment and T1/T2 mapping utilized for evaluating intracellular disturbances of cardiomyocytes. Both cine imaging and T1/T2 mapping involve data acquisition along temporal direction. Developing techniques to reconstruct highly undersampled 3D datasets with temporal acquisition (3D+1D) is challenging but highly desirable for these applications. Recent studies have demonstrated the superior performance of deep learning (DL) methods in MRI reconstruction compared to traditional methods 1-3. However, existing research of DL-based MRI reconstruction primarily focuses on static MRI 4-9. Limited work has been reported for DL-based reconstruction of fast CMR with spatiotemporal acquisitions 10,11. In this work, we propose a self-consistency guided multi-prior DL framework termed $$$k$$$-$$$t$$$ CLAIR to achieve faithful MR reconstruction of highly undersampled 3D+1D CMR.Methods
We formulate the inverse reconstruction process as a multi-prior optimization problem across various domains:$$\mathop{\min}\limits_\mathbf{m}{\| \mathcal{A}\mathbf{m} - \tilde{\mathbf{v}} \| ^2_2 + \lambda_\text{xt} \mathcal{R}_\text{xt}(\mathbf{m};\theta_\text{xt}) + \lambda_\text{xf} \mathcal{R}_\text{xf}(\mathcal{F}_\text{t}\mathbf{m};\theta_\text{xf}) + \lambda_\text{kt} \mathcal{R}_\text{kt}(\mathcal{F}_\text{s}\mathbf{m};\theta_\text{kt})},$$
where $$$\mathcal{R}_\text{xt}(\mathbf{m};\theta_\text{xt})$$$, $$$\mathcal{R}_\text{xf}(\mathcal{F}_\text{t}\mathbf{m};\theta_\text{xf})$$$, and $$$\mathcal{R}_\text{kt}(\mathcal{F}_\text{s}\mathbf{m};\theta_\text{kt})$$$ are data-adaptive priors with learnable parameters $$$\theta_\text{xt}$$$, $$$\theta_\text{xf}$$$, and $$$\theta_\text{kt}$$$ to regularize the data in the $$$x$$$-$$$t$$$, $$$x$$$-$$$f$$$, and $$$k$$$-$$$t$$$ spaces, respectively. $$$\mathcal{F}_\text{t}$$$ and $$$\mathcal{F}_\text{s}$$$ represent Fourier transform along temporal and spatial dimensions of the image series $$$\mathbf{m}$$$, respectively. $$$\lambda_\text{xt}$$$, $$$\lambda_\text{xf}$$$, and $$$\lambda_\text{kt}$$$ control the balance between the influence of the imposed priors and the fidelity to the acquired data.
We solve the above equation through our iterative method, $$$k$$$-$$$t$$$ CLAIR, as depicted in Fig. 1. It comprises several components, including $$$sen$$$-CNN, $$$xt$$$-CNN, $$$xf$$$-CNN, $$$kt$$$-CNN, $$$calib$$$-CNN, and a frequency fusion block. These components exploit spatiotemporal correlations to learn complementary priors in CMR reconstruction. Specifically, the $$$sen$$$-CNN extracts coil sensitivity maps from autocalibration signal (ACS) data using a U-Net inspired by the work 12. The $$$xt$$$-CNN, employing Gradient Descent (GD) with 3D U-Net, eliminates aliasing artifacts and restores overall anatomical structures in the $$$x$$$-$$$t$$$ domain. The $$$xf$$$-CNN, designed similarly to the $xt$-CNN, captures dynamic details by extracting spatiotemporal correlations in the $$$x$$$-$$$f$$$ domain. The $$$kt$$$-CNN and $$$calib$$$-CNN comprise four convolution layers to model dynamic multi-coil k-space correlation. The $$$calib$$$-CNN aims to extract self-consistency features to enforce the $$$kt$$$-CNN to interpolate consistent k-space data. A frequency fusion block merges reconstructed results from $$$x$$$-$$$t$$$, $$$x$$$-$$$f$$$, and $$$k$$$-$$$t$$$ domains, allowing coordinated feature learning across domains. Note that all components utilize 3D convolutions to capture dynamic information in the time domain during 3D+1D MRI reconstruction.
Data: Experiments were conducted on two CMR reconstruction tasks: CINE and T1/T2 Mapping 13,14. Cine imaging included SA and LA (2CH, 3CH, 4CH) views with a cardiac cycle of 12–25 phases, while T1/T2 mapping involved 9 T1W and 3 T2W images. The investigated data consisted of 180 healthy volunteers with 120 fully sampled and 60 retrospectively accelerated data. One-dimensional equispaced sampling masks of acceleration factors 4, 8, and 10 with an ACS of 24 lines were used.
Implementation: The fully sampled k-space data were randomly divided into two subsets: 80% for training and the remaining 20% for validation. The 60 retrospectively accelerated data served as the testing set with withheld ground truth images for online evaluation. The $$$k$$$-$$$t$$$ CLAIR underwent 12 iterations for both tasks. For each task, a single model was trained for reconstruction of all different acceleration factors.
Results and Discussion
We compared our method against Zero-filled, FastMRI U-Net 12, and E2EVarNet 15. Additionally, we extended the FastMRI U-Net and E2EVarNet by incorporating 3D Convolution to capture temporal information. Detailed evaluation results on the validation (Table 1) and testing (Table 2) sets are shown in Fig. 2. The results on the validation set show the improvements achieved by the proposed method across all metrics, views (LAX and SAX), and contrasts (T1W and T2W). We also highlighted the improvement of the proposed method in terms of the statistics of the performance on the validation set, including the outliers at each acceleration factor, for both CINE (Fig. 3(a)) and T1W/T2W (Fig. 3(b)) reconstruction. Furthermore, our method demonstrated good generalization on the testing set for both tasks. Representative images from the CINE and T1/T2 Mapping tasks at the 10X acceleration factor are displayed in Fig. 4 and Fig. 5, respectively, illustrating the faithful reconstructions produced by our method across all views and contrasts. The reconstruction error maps consistently demonstrated the improved performance of our method in rapid CMR reconstruction compared to the other methods.Conclusion
We presented $$$k$$$-$$$t$$$ CLAIR for CMR reconstruction and demonstrated its potential in both fast CINE and T1/T2 mapping tasks. Future work includes exploring its applications in fast MRI of other anatomies which involves spatiotemporal acquisitions.Acknowledgements
This work was supported by a grant from Innovation and Technology Commission of the Hong Kong SAR (MRP/046/20X); and by a Faculty Innovation Award from the Faculty of Medicine of the Chinese University of Hong Kong.References
1. M. Lustig, D. Donoho, and J. M. Pauly, “Sparse mri: The application of compressed sensing for rapid mr imaging,” Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 58, no. 6, pp. 1182–1195, 2007.
2. H. Jung, K. Sung, K. S. Nayak, E. Y. Kim, and J. C. Ye, “k-t focuss: a general compressed sensing framework for high resolution dynamic mri,” Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 61, no. 1, pp. 103–116, 2009.
3. S. G. Lingala, Y. Hu, E. DiBella, and M. Jacob, “Accelerated dynamic mri exploiting sparsity and low-rank structure: kt slr,” IEEE transactions on medical imaging, vol. 30, no. 5, pp. 1042–1054, 2011.
4. K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll, “Learning a variational network for reconstruction of accelerated mri data,” Magnetic resonance in medicine, vol. 79, no. 6, pp. 3055–3071, 2018.
5. T.Eo,Y.Jun,T.Kim,J.Jang,H.-J.Lee,andD.Hwang,“Kiki-net:cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images,” Magnetic resonance in medicine, vol. 80, no. 5, pp. 2188–2201, 2018.
6. M. Ran, W. Xia, Y. Huang, Z. Lu, P. Bao, Y. Liu, H. Sun, J. Zhou, and Y. Zhang, “Md-recon-net: A parallel dual-domain convolutional neural network for compressed sensing mri,” IEEE Transactions on Radiation and Plasma Medical Sciences, vol. 5, no. 1, pp. 120–135, 2020.
7. S. Wang, Z. Ke, H. Cheng, S. Jia, L. Ying, H. Zheng, and D. Liang, “Dimension: dynamic mr imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training,” NMR in Biomedicine, vol. 35, no. 4, p. e4131, 2022.
8. Z. Fabian, B. Tinaz, and M. Soltanolkotabi, “Humus-net: Hybrid unrolled multi-scale network architecture for accelerated mri reconstruction,” Advances in Neural Information Processing Systems, vol. 35, pp. 25 306–25 319, 2022.
9. L. Zhang, X. Li, and W. Chen, “Camp-net: Consistency-aware multi-prior network for accelerated mri reconstruction,” arXiv preprint arXiv:2306.11238, 2023.
10. T. Ku ̈stner, N. Fuin, K. Hammernik, A. Bustin, H. Qi, R. Hajhosseiny, P. G. Masci, R. Neji, D. Rueckert, R. M. Botnar et al., “Cinenet: deep learning-based 3d cardiac cine mri reconstruction with multi-coil complex-valued 4d spatio-temporal convolutions,” Scientific reports, vol. 10, no. 1, p. 13710, 2020.
11. C. Qin, J. Duan, K. Hammernik, J. Schlemper, T. Ku ̈stner, R. Botnar, C. Prieto, A. N. Price, J. V. Hajnal, and D. Rueckert, “Complementary time-frequency domain networks for dynamic parallel mr image reconstruction,” Magnetic Resonance in Medicine, vol. 86, no. 6, pp. 3274–3291, 2021.
12. J. Zbontar, F. Knoll, A. Sriram, T. Murrell, Z. Huang, M. J. Muckley, A. Defazio, R. Stern, P. Johnson, M. Bruno et al., “fastmri: An open dataset and benchmarks for accelerated mri,” arXiv preprint arXiv:1811.08839, 2018.
13. C. Wang, Y. Li, J. Lv, J. Jin, X. Hu, X. Kuang, W. Chen, and H. Wang, “Recommendation for cardiac magnetic resonance imaging-based phenotypic study: imaging part,” Phenomics, vol. 1, pp. 151–170, 2021.
14. C. Wang, J. Lyu, S. Wang, C. Qin, K. Guo, X. Zhang, X. Yu, Y. Li, F. Wang, J. Jin et al., “Cmrxrecon: An open cardiac mri dataset for the competition of accelerated image reconstruction,” arXiv preprint arXiv:2309.10836, 2023.
15. A. Sriram, J. Zbontar, T. Murrell, A. Defazio, C. L. Zitnick, N. Yakubova, F. Knoll, and P. Johnson, “End-to-end variational networks for accelerated mri reconstruction,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23. Springer, 2020, pp. 64–73.
Figures
Fig. 1: Overall architecture of the proposed $$$k$$$-$$$t$$$ CLAIR, a self-consistency guided multi-prior deep learning framework, for reconstruction of fast spatiotemporal MRI. (Best viewed in color.)
Fig. 3: Violin plots illustrating PSNR, NMSE, and SSIM measurements of the proposed $$$k$$$-$$$t$$$ CLAIR model compared to zero-filled, UNet, U-Net3D, E2EVarNet, and E2EVarNet3D models for CINE and T1W/T2W reconstruction at acceleration factors of 4X, 8X, and 10X. The proposed $$$k$$$-$$$t$$$ CLAIR model demonstrates improved performance across all measurements and tasks. (Best viewed in color.)
