0352
Physics-informed Synthetic Data Learning Boosts Multi-Scenario Fast MRI Reconstruction1Xiamen University, Xiamen, China, 2Fudan University, Shanghai, China, 3Philips Healthcare, Shanghai, China, 4Siemens Healthineers Ltd., Shanghai, China, 5United Imaging Research Institute of Intelligent Imaging, Beijing, China, 6Nanjing First Hospital, Nanjing, China, 7Shandong Aoxin Medical Technology Company, Weifang, China, 8The First Affiliated Hospital of Xiamen University, Xiamen, China, 9Zhongshan Hospital Affiliated to Xiamen University, Xiamen, China, 10Zhongshan Hospital, Fudan University (Xiamen Branch), Xiamen, China, 11Zhongshan Hospital, Fudan University, Shanghai, China, 12Ruijin Hospital Lu Wan Branch, Shanghai Jiaotong University School of Medicine, Shanghai, China, 13Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, China, 14The First Affiliated Hospital of Kunming Medical University, Shanghai, China, 15Xiamen University of Technology, Xiamen, China, 16Department of Electronic Science, Xiamen University, Xiamen, China
Synopsis
Keywords: AI/ML Image Reconstruction, Machine Learning/Artificial Intelligence, MR Physics Model
Motivation: Deep learning (DL) is powerful for fast MRI reconstruction, but remains largely untapped in multiple clinical imaging scenarios.
Goal(s): To provide a feasible and cost-effective way to markedly boost the widespread usage of DL in various fast MRI applications.
Approach: In this work, we present a Physics-Informed Synthetic data learning framework for Fast MRI, called PISF, which is the first to enable generalizable DL for multi-scenario MRI reconstruction using solely one trained model.
Results: PISF trained on synthetic data enables high-quality, ultra-fast, and robust MRI reconstruction from different 4contrasts, 5 anatomies, 5 vendors and centers, and 2 pathologies, without further re-training.
Impact: Physics-informed synthetic data learning (DL) provides a feasible and cost-effective way to markedly boost the widespread usage of DL in various fast MRI applications, while freeing from the intractable ethical and practical considerations of in vivo human data acquisitions.
Purpose
Magnetic resonance imaging (MRI) is a principal radiological modality that provides radiation-free, abundant, and diverse information about the whole human body for medical diagnosis, but suffers from prolonged scan time [1]. Although deep learning (DL) has emerged as a powerful tool for image reconstruction in fast MRI, its potential in multiple imaging scenarios remains largely untapped. This is because not only collecting large-scale and diverse realistic training data is generally costly and privacy-restricted [2], but also existing DL methods are hard to handle the practically inevitable mismatch between training and target data [3-4]. Here, we present a Physics-Informed Synthetic data learning framework for Fast MRI, called PISF, which is the first to enable generalizable DL for multi-scenario MRI reconstruction using solely one synthetic-data-trained model.Method
Figure 1 shows that our proposed PISF first generates large-scale and diverse synthetic data based on physical forward models for training a DL foundation network (Figure 2(a)), and then utilizes the data-specific enhancement to achieve robust performance under multi-scenarios.For data synthesis, although the synthetic texture may be quite different from realistic MR images, our usage of 1D learning [4] alleviates reliance on spatial relationships and enhances the correlation between the synthetic and realistic data. For example, latent features show that most features of the target in vivo MRI data are covered by those of the synthetic training data (Figure 2(b)). Besides, we can see that the statistical distribution of the synthetic data also encompasses the realistic data with a wider intensity range in the histogram (Figure 2(c)). Thus, we can expect synthetic data learning to yield a more versatile nonlinear mapping between images with and without artifacts, and then handle well on the realistic data.
Using the synthetic database, the main task of the DL network design is to narrow the gap between synthetic and realistic data (Figure 2(d)). Thus, we present the core idea of “separate-first-enhance-then”, which means that for a 2D reconstruction problem, we first separate it into many 1D basic problems in the training stage [4], to train a data-versatile 1D foundation network for adaptive image de-aliasing. While in the reconstruction stage, the solution is regularized by using a data-specific 2D k-space enhancement obtained from the target data itself, to achieve strong generalization into various practical applications. Remarkably, although this approach is tailored towards the widely used 1D undersampling, it can naturally extend to 2D undersampling without any re-training.
Results
We compare the proposed synthetic-data-trained PISF with two state-of-the-art DL methods: DOTA [5] and HDSLR [6], both of which are trained using large-scale realistic data from fastMRI dataset [7]. The detailed descriptions of training and test datasets can be found in our full-length paper [8].To quantitatively evaluate the reconstruction performance, two objective criteria including peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) [9] are utilized.
Since objective criteria might not reflect image quality in terms of diagnostically important features, three clinical-concerned subjective criteria including SNR, artifacts suppression, and overall image quality are used for reader study. The score of each criterion has a range from 0 to 5 with precision of 0.1 (i.e., 1: Non-diagnostic; 2: Poor; 3: Adequate; 4: Good; 5: Excellent). The reader study is conducted online through our CloudBrain-ReconAI platform [10-11] (https://csrc.xmu.edu.cn/CloudBrain.html).
(i) Substitutability of PISF framework for conventional DL: In Figure 3, PISF achieves comparable performance to realistic data training methods while reducing the demand for real-world MRI data by up to 96%.
(ii) Multi-vendor multi-center imaging: In Figure 4, PISF can reconstruct high-quality images of 4 anatomies and 5 contrasts across 5 vendors and centers using a single trained network.
(iii) Adaptability to patients and reader study: Considering the morphology of pathological tissues is extremely complicated and diverse, we examine the adaptability of our method to patients, which is essential in clinical diagnosis. Figure 5 shows that, its overall image quality steps into the excellent level (i.e., 4 of a 5-point Likert scale) under 10 experienced doctors’ evaluations.
Conclusion
In this work, we present a proof-of-concept demonstration of applying physics-informed synthetic data learning to achieve high-quality, ultra-fast, and robust MRI reconstruction of different contrasts, anatomies, vendors, centers, and pathologies. It opens a new avenue for the widespread application of DL in MRI, without the need to consider the intractable ethical and practical issues of in vivo human data acquisitions. Thanks to the flexibility of our PISF framework, we anticipate that it is also applicable to higher dimensional imaging (e.g., dynamic, diffusion, and quantitative MRI) after proper modifications.Acknowledgements
See more details in the full-length preprint: https://arxiv.org/abs/2307.13220. This work was supported in part by the National Natural Science Foundation of China under grants 62331021, 62122064, 61971361, and 61871341, Natural Science Foundation of Fujian Province of China under grants 2023J02005 and 2021J011184, President Fund of Xiamen University under grant 20720220063, and Xiamen University Nanqiang Outstanding Talents Program. The authors thank Drs. Michael Lustig, Dosik Hwang, and Mathews Jacob for sharing their codes online.
The correspondence should be sent to Prof. Xiaobo Qu (Email: quxiaobo@xmu.edu.cn)
References
[1] M. Lustig, D. Donoho, and J. M. Pauly, "Sparse MRI: The application of compressed sensing for rapid MR imaging," Magnetic Resonance in Medicine, vol. 58, no. 6, pp. 1182-1195, 2007.
[2] Q. Yang, Z. Wang, K. Guo, C. Cai, and X. Qu, "Physics-driven synthetic data learning for biomedical magnetic resonance: The imaging physics-based data synthesis paradigm for artificial intelligence," IEEE Signal Processing Magazine, vol. 40, no. 2, pp. 129-140, 2023.
[3] V. Antun, F. Renna, C. Poon, B. Adcock, and A. C. Hansen, "On instabilities of deep learning in image reconstruction and the potential costs of AI". Proceedings of the National Academy of Sciences, vol. 117, no. 48, p. 30088, 2020.
[4] Z. Wang et al., "One-dimensional deep low-rank and sparse network for accelerated MRI," IEEE Transactions on Medical Imaging, vol. 42, no. 1, pp. 79-90, 2023.
[5] T. Eo, H. Shin, Y. Jun, T. Kim, and D. Hwang, "Accelerating Cartesian MRI by domain-transform manifold learning in phase-encoding direction". Medical Image Analysis, vol. 63, 101689, 2020.
[6] A. Pramanik, H. Aggarwal, and M. Jacob, "Deep generalization of structured low-rank algorithms (Deep-SLR)," IEEE Transactions on Medical Imaging, vol. 39, no. 12, pp. 4186-4197, 2020.
[7] F. K. Zbontar et al., "FastMRI: An open dataset and benchmarks for accelerated MRI," arXiv: 1811.08839, 2019.
[8] Z. Wang et al., "One for multiple: Physics-informed synthetic data boosts generalizable deep learning for fast MRI reconstruction," arXiv: 2307.13220, 2023.
[9] W. Zhou, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.
[10] Y. Zhou et al., "XCloud-pFISTA: A medical intelligence cloud for accelerated MRI," in 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021, pp. 3289-3292.
[11] Y. Zhou et al., "CloudBrain-ReconAI: An online platform for MRI reconstruction and image quality evaluation," arXiv: 2212.01878, 2022.
Figures
