3650
fastMRI Prostate: A Public, Biparametric MRI Dataset to Advance Machine Learning for Prostate Cancer Imaging1Bernard and Irene Schwartz Center for Biomedical Imaging, Department of Radiology, New York University Grossman School of Medicine, New York, NY, United States, 2Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine, New York, NY, United States, 3Vilcek Institute of Graduate Biomedical Sciences, New York University Grossman School of Medicine, New York, NY, United States, 4Siemens Medical Solutions USA, New York, NY, United States
Synopsis
Keywords: Prostate, Prostate, k-space data, public
Motivation: There is a lack of publicly available, raw k-space data for prostate MRI.
Goal(s): To compile and release raw k-space data for clinical prostate MRI and demonstrate its utility for development of deep learning methods for image reconstruction and automated diagnosis.
Approach: Biparametric MRI data from 312 patients with associated prostate cancer labels were added to the public fastMRI repository. Deep-learning models were trained on the data to reconstruct images from undersampled k-space and perform automated diagnosis of prostate cancer (PCa) on these images.
Results: SSIM > 0.866 and AUC > 0.80 (test set) for the deep-learning reconstruction and automated PCa diagnosis respectively.
Impact: Raw k-space data with clinical labels from fastMRI prostate will enable researchers to develop clinically relevant deep-learning reconstruction and automated diagnosis models which may ultimately advance the diagnosis and management of prostate cancer.
Introduction
The public release of large raw k-space datasets and data labels over the last four years (fastMRI, SKM-TEA, fastMRI+)1-3 have significantly advanced MR image reconstruction and classification research. Public access to raw k-space data has enabled novel reconstruction techniques to accelerate MRI acquisitions4,5, supporting downstream AI processes like classification and segmentation and enabling upstream optimization of data acquisition and image reconstruction. Prostate cancer (PCa) is a significant healthcare burden, with high mortality and substantial economic impact6,7. Early detection and efficient management are vital. MRI, especially with the use of T2-weighted and diffusion-weighted image contrasts, is a crucial diagnostic tool in PCa management8-11. Accelerating MRI exams through advanced reconstruction of undersampled data can enhance cost-effectiveness, while automated image interpretation can improve MRI accessibility. The PI-CAI dataset (2022) underscored the importance of integrating AI into the prostate cancer (PCa) diagnostic pipeline but the released data included only reconstructed images. To address the gap of raw prostate k-space data availability, the fastMRI prostate dataset was introduced in 202312. It comprises raw T2-weighted and diffusion-weighted data and image labels for suspicion of clinically significant PCa (csPCa). This abstract describes the dataset and demonstrates its potential for deep-learning-based reconstruction and automated diagnosis.Methods
Figure 1 depicts the overall workflow utilized to generate the fastMRI prostate dataset. The fastMRI prostate dataset (https://fastmri.med.nyu.edu/) is generated from biparametric 3T (Table 1) clinical MRI scans (MAGNETOM Vida, Siemens Healthineers, Germany), encompassing T2-weighted and diffusion-weighted images (DWI) acquired from 312 patients (age 66 ± 8 years) at our institution between March 2020 and April 2021.The k-space data, the associated offline reconstructions and DICOM data were made available in the publicly released H5 files13,14. Each under-sampled average of the T2 data was reconstructed using GRAPPA15 and a RSS combination followed by an averaging in image space. The DWI reconstruction includes EPI gridding, GRAPPA reconstruction, and an SNR-optimizing matched filter coil combination16 with coil sensitivities estimated using ESPIRIT17. The offline reconstructions serve as a benchmark for developers for validation of new image reconstruction methods. The DICOM files, while post-processed by the scanner, are compatible with various existing image analysis and machine learning tools.
Additionally, the dataset includes labels comprising slice-by-slice PI-RADS scores for assessing prostate cancer lesions. The patients were divided into training, validation, and test subsets. Scripts for image reconstruction, label files are provided on the GitHub repository: https://github.com/cai2r/fastMRI_prostate.
To show the utility of the dataset in accelerated MRI reconstruction and automated diagnosis pipelines, four deep-learning networks were trained. Two end-to-end variational networks (E2E-VarNet)18,19 were trained for the axial 6x under-sampled T2-weighted and 3x under-sampled diffusion-weighted sequences using an SSIM-based loss function.
Two 2D classification models were developed to diagnose clinically significant prostate cancer (csPCa) using the ConvNext20 architecture on the T2-weighted images and B1500 and ADC maps using a binary cross-entropy loss function. The architecture was modified to accommodate one channel for T2 images and two channels for diffusion models. Training was performed on a slice-level basis with the labels provided in the dataset. For binary classification, the PIRAD labels were sorted into two bins, PIRADS<2 vs PIRADS>3, to distinguish stable features from csPCa.
All models were trained with an Adam optimizer and a learning rate of 1e-03 with early stopping to prevent overfitting.
Results
Figure 2 shows an example slice for a reconstructed T2-weighted image, B50 trace, B1000 trace, and calculated ADC map for two subjects with PI-RADS scores of 1 and 5, respectively. For the VarNet based reconstruction, the averaged SSIM for the test set was 0.952±0.014, 0.930±0.014 and 0.855±0.061 for T2, B50 trace and B1000 trace respectively.Figure 3 shows an example from the test set comparing the conventional reconstruction to the VarNet reconstruction. AUCs of 0.83 (95% CI:0.77–0.87) and 0.80 (95% CI:0.74–0.86) were obtained for the identification of csPCa (PI-RADS>3) in the test T2 and diffusion images, respectively.
Figure 4 displays the ROC curves of both models applied to the test subset.
Discussion and Conclusion
The fastMRI prostate dataset represents a valuable resource for advancing data-driven image reconstruction techniques in MRI. The results from the VarNet and the classification models show the suitability of the dataset for investigating accelerated imaging methods and automated diagnosis. Though it is smaller than previous fastMRI datasets, this dataset includes real-world clinical data along with raw diffusion-weighted images, making it a uniquely valuable resource for developing clinically relevant AI models. The dataset is designed to serve as a foundation for the future development of state-of-the-art deep learning models and reconstruction techniques with clinical impact.Acknowledgements
The work reported in this publication was supported by the Center for Advanced Imaging Innovation and Research (CAI2R), a National Center for Biomedical Imaging and Bioengineering operated by NYU Langone Health and funded by the National Institute of Biomedical Imaging and Bioengineering through award P41EB017183. Additional NIH/NIBIB funding was provided by grant R01EB024532.References
1. Knoll, F. et al. fastMRI: A Publicly Available Raw k-Space and DICOM Dataset of Knee Images for Accelerated MR Image Reconstruction Using Machine Learning. Radiol Artif Intell 2, e190007 (2020). https://doi.org:10.1148/ryai.2020190007
2. Desai, A. D. et al. SKM-TEA: A Dataset for Accelerated MRI Reconstruction with Dense Image Labels for Quantitative Clinical Evaluation. (2022).
3. Zhao, R. et al. fastMRI+, Clinical pathology annotations for knee and brain fully sampled magnetic resonance imaging data. Sci Data 9, 152 (2022). https://doi.org:10.1038/s41597-022-01255-z
4. Knoll, F. et al. Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge. Magn Reson Med 84, 3054-3070 (2020). https://doi.org:10.1002/mrm.28338
5. Muckley, M. J. et al. Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction. IEEE Trans Med Imaging 40, 2306-2317 (2021). https://doi.org:10.1109/TMI.2021.3075856
6. Siegel, R. L., Miller, K. D., Wagle, N. S. & Jemal, A. Cancer statistics, 2023. CA Cancer J Clin 73, 17-48 (2023). https://doi.org:10.3322/caac.21763
7. Mariotto, A. B., Enewold, L., Zhao, J., Zeruto, C. A. & Yabroff, K. R. Medical Care Costs Associated with Cancer Survivorship in the United States. Cancer Epidemiol Biomarkers Prev 29, 1304-1312 (2020). https://doi.org:10.1158/1055-9965.EPI-19-1534
8. Penzkofer, T. & Tempany-Afdhal, C. M. Prostate cancer detection and diagnosis: the role of MR and its comparison with other diagnostic modalities--a radiologist's perspective. NMR Biomed 27, 3-15 (2014). https://doi.org:10.1002/nbm.3002
9. Drost, F. H. et al. Prostate Magnetic Resonance Imaging, with or Without Magnetic Resonance Imaging-targeted Biopsy, and Systematic Biopsy for Detecting Prostate Cancer: A Cochrane Systematic Review and Meta-analysis. Eur Urol 77, 78-94 (2020). https://doi.org:10.1016/j.eururo.2019.06.023
10. Rosenkrantz, A. B. et al. Computed diffusion-weighted imaging of the prostate at 3 T: impact on image quality and tumour detection. Eur Radiol 23, 3170-3177 (2013). https://doi.org:10.1007/s00330-013-2917-8
11. Greenberg, J. W., Koller, C. R., Casado, C., Triche, B. L. & Krane, L. S. A narrative review of biparametric MRI (bpMRI) implementation on screening, detection, and the overall accuracy for prostate cancer. Ther Adv Urol 14, 17562872221096377 (2022). https://doi.org:10.1177/17562872221096377
12. Tibrewala, R. et al. FastMRI Prostate: A Publicly Available, Biparametric MRI Dataset to Advance Machine Learning for Prostate Cancer Imaging. arXiv preprint arXiv:2304.09254 (2023).
13. Koranne, S. in Handbook of Open Source Tools 191-200 (Springer US, 2011).
14. Knoll, F., Patricia M. Johnson, Daniel K. Sodickson, Michael P. Recht, Yvonne W. Lui. (2018).
15. Griswold, M. A. et al. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn Reson Med 47, 1202-1210 (2002). https://doi.org:10.1002/mrm.10171
16. Roemer, P. B., Edelstein, W. A., Hayes, C. E., Souza, S. P. & Mueller, O. M. The NMR phased array. Magn Reson Med 16, 192-225 (1990). https://doi.org:10.1002/mrm.1910160203
17. Uecker, M. et al. ESPIRiT--an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magn Reson Med 71, 990-1001 (2014). https://doi.org:10.1002/mrm.24751
18. Hammernik, K. et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 79, 3055-3071 (2018). https://doi.org:10.1002/mrm.26977
19. Johnson, P. M. et al. Deep Learning Reconstruction Enables Highly Accelerated Biparametric MR Imaging of the Prostate. J Magn Reson Imaging 56, 184-195 (2022). https://doi.org:10.1002/jmri.28024
20. Liu, Z. et al. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11976-11986.
Figures
