0245
The ISMRM Open Science Initiative for Perfusion Imaging (OSIPI): A Challenge for Reproducible DCE-MRI AI-based Analysis1Cancer Center, University of Colorado, Aurora, CO, United States, 2Edinburgh Imaging, The University of Edinburgh, Edinburgh, United Kingdom, 3Centre for Cardiovascular Science, The University of Edinburgh, Edinburgh, United Kingdom, 4Department of Neurosurgery, University of Pennsylvania, Philadelphia, PA, United States, 5Clinical Imaging Group, Genentech, South San Francisco, CA, United States, 6Department of Radiology & Nuclear Medicine, Erasmus MC University Medical Center, Rotterdam, Netherlands, 7Radiology and Nuclear Medicine, Vrije Universiteit Amsterdam, Amsterdam, Netherlands, 8Amsterdam Neuroscience, Brain Imaging, Amsterdam, Netherlands, 9Department of Radiology and Nuclear Medicine, University of Amsterdam, Amsterdam, Netherlands, 10Imaging and Biomarkers, Cancer Center Amsterdam, Amsterdam, Netherlands, 11School of Physics and Astronomy, University of Leeds, Leeds, United Kingdom, 12Institute of Physics, University of Campinas, Campinas, Brazil, 13Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States, 14Department of Radiology, Mayo Clinic, Rochester, MN, United States, 15Department of Radiology, University of Colorado, Aurora, CO, United States, 16Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, Netherlands, 17Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom, 18Radiology, University of Alabama in Birmingham, Birmingham, AL, United States
Synopsis
Keywords: Data Processing, DSC & DCE Perfusion, Deep Learning
Motivation: There is a need for reproducibility, repeatability, and accuracy. Previously, OSIPI organized a challenge for benchmarking DCE software. As the use of artificial intelligence grows, we now set out to repeat the challenge, focusing on deep learning techniques.
Goal(s): To encourage researchers put their quantitative methods to test and stimulate collaboration and to charter the heterogeneity of DCE analysis software.
Approach: To use deep learning techniques to estimate perfusion parameters in DCE-MRI of the uterus. We share repeated in-vivo data to assess the algorithm’s precision, and simulated DCE-data to test the accuracy.
Results: Top three winners may present their method at ISMRM 2025.
Impact: As quantitative perfusion MRI receives more importance and attention, the need for reproducibility, repeatability, and accuracy is inevitable. A public challenge within the MRI community is a great way to highlight the quantification of DCE-MRI.
Introduction
DCE-MRI has successfully been used in staging and assessing therapy for various cancerous, inflammatory, and fibrotic diseases1,2. However, to get DCE-MRI from a research tool to clinic, reproducibility and repeatability in quantitative methods are essential3. Specifically, in perfusion imaging, the variability in the estimation of perfusion parameters is high due to several reasons: e.g, arterial input function (AIF) selection (patient-specific vs. population-based), perfusion model (Tofts4, Extended Tofts5, etc.), fitting algorithm, and T1 values. The ISMRM Open Science Initiative for Perfusion Imaging (OSIPI), an initiative of the ISMRM perfusion study group, was founded to support reproducible research and open science in perfusion imaging and enable the translation of software tools into clinical practice. This movement will eventually facilitate the translation of quantitative methods to clinical practice across institutions worldwide. In 2021, the OSIPI Task Force 6.2 designed and published a challenge to test the reproducibility of perfusion analysis in brain tumors6 in which the winner was the only deep learning contribution. Therefore, this year, we have designed a challenge for which we require the use of Deep Learning techniques to estimate perfusion parameters in DCE-MRI images of the uterus7.Challenge Overview
The synthetic data will be created using the SPGR signal model with various T1,Ktrans, and ve pixel values. Gaussian noise will be added to the synthetic data, and a population-based AIF will be provided. The in vivo data set includes two DCE-MRI exams of each individual.The participants will receive synthetic and in vivo DCE-MRI data, combined with inversion-recovery data for T1 mapping and will submit their estimated Ktrans and ve maps for both in vivo and synthetic data. The synthetic data will be used to evaluate the accuracy of their method, and the test-retest in vivo data will be used to measure the repeatability. The neutral evaluators will generate the Ktrans and ve maps using the standard operating procedure (SOP) provided by the participants to calculate the reproducibility. The scores for each category will be added to determine the winner (Fig.1).OSIPI-DCE will be an ISMRM challenge8, that will be launched at the ISMRM Annual Meeting in May 2024, and the winners will be announced at the 2025 ISMRM Annual Meeting (Fig.2).Data - In Vivo and Synthetic
1.In Vivo Data- Patient population: The clinical data for this challenge were previously used for studying endometrial hypoxia9. The dataset includes the MR images of the uterus of 12 healthy female volunteers (18–55 years) with regular menstrual cycles (21–35 days). Each volunteer was scanned twice:during days 1–3 of menstruation and the early/mid-secretory phase of their cycle. The two relevant MRI sequences for this challenge are DCE-MRI and IR-TrueFISP (Figure.3). All images were acquired on a 3T scanner (Siemens Verio or Skyrafit, Siemens Healthineers, Erlangen, Germany)
- T1 measurement: The protocol included sagittal IR-TrueFISP (inversion recovery true fast imaging with steady-state precession) for T1 measurement with voxel size of 2×2×5mm. The Inversion Times are:220,300,600,1000,1500,2500 msec (Table.1).
- DCE-MRI: Sagittal 3D FLASH (fast low angle shot) which was acquired during injection of 0.1mg/kg Gadovist followed by 20ml saline chaser at 2ml/s. 150 frames were acquired at a temporal resolution of 1.6sec with the same voxel size as the IR-TrueFISP (2×2×5mm).
- AIF: derived at each visit from the common iliac arteries.
- A morphological phantom will be generated by assuming a series of perfusion parameters, T1 values, and a population-based AIF10,11. A two-compartment exchange model and the spoiled gradient-echo steady-state signal model will be used to create the voxel-wise signal. The assumed T1 values for blood and tissue will be provided to convert the signal to concentration.
Submissions and Evaluation
Each team will train their deep learning models based on their preferred method (on their in vivo or simulated data). Then, their trained model will be tested on the data (in vivo and synthetic) that is provided by this challenge. The participants will submit the perfusion maps for each dataset. In addition, they submit SOPs so that the evaluators will reproduce their results following their SOP and their code. The SOP should describe how to install and use the provided trained model. Finally, the results will be evaluated based on the OSIPI-DCE scoring system defined in Table.2.Conclusion
After evaluating the results submitted by the participants, and ranking the best models, the winner will be announced, and the top three participants may present their work in ISMRM 2025. After the challenge is concluded, the overall results will be published in an article to help the MRI community learn about different methods and their reproducibility. We hope that the OSIPI-DCE challenge ultimately improves and highlights more quantitative methods in DCE-MRI.Acknowledgements
We would like to acknowledge Dr. Jackie Maybin for providing the dataset for this challenge. This data was acquired for a work that was funded by Wellbeing of Women grant RG1820, Wellcome Trust Fellowship 209589/Z/17/Z and undertaken in the MRC Centre for Reproductive Health, funded by grants G1002033 and MR/N022556/1.References
1. A. R. Padhani and K. A. Miles, “Multiparametric imaging of tumor response to therapy,” Radiology, vol. 256, no. 2, pp. 348–364, 2010.
2. J. P. O’Connor et al., “Imaging biomarker roadmap for cancer studies,” Nature reviews Clinical oncology, vol. 14, no. 3, pp. 169–186, 2017.
3. Stikov, N., Trzasko, J. D. & Bernstein, M. A. Reproducibility and the future of MRI research. Magn. Reson. Med. 82, 1981–1983 (2019).
4. Tofts PS, Kermode AG. Measurement of the blood-brain barrier permeability and leakage space using dynamic MR imaging. 1. Fundamental concepts. Magn Reson Med 1991;17:357–67.
5. Tofts PS. Modeling tracer kinetics in dynamic Gd-DTPA MR imaging. J Magn Reson Imaging 1997;7:91–101.
6. Kazerooni A, The Open Source Initiative for Perfusion Imaging (OSIPI): DCE-MRI Challenge, ISMRM 2021, Abstract 1094
7. Maybin JA, Murray AA, Saunders PTK, Hirani N, Carmeliet P, Critchley HOD. Hypoxia and hypoxia inducible factor-1alpha are required for normal endometrial repair during menstruation. Nat Commun 2018;9:295.
8. http://challenge.ismrm.org/
9. Reavey JJ, Walker C, Nicol M, Murray AA, Critchley HOD, Kershaw LE, Maybin JA. Markers of human endometrial hypoxia can be detected in vivo and ex vivo during physiological menstruation. Hum Reprod. 2021 Mar 18;36(4):941-950.
10. G. J. Parker et al., “Experimentally-derived functional form for a population-averaged high-temporal-resolution arterial input function for dynamic contrast-enhanced MRI,” Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 56, no. 5, pp. 993–1000, 2006.
11. Harrison Kim, “Modification of population based arterial input function to incorporate individual variation”, Magnetic Resonance Imaging, Volume 45, 2018, Pages 66-71.
Figures
Figure 3. Examples of in vivo data. (A) T2 sagittal, (B) DCE sagittal, (C) T2 Axial, (D) T2 Coronal.
