0540

Towards Reproducible Intravoxel Incoherent Motion (IVIM) Analysis: The ISMRM Open Science Initiative for Perfusion Imaging
Oscar Jalnefjord1,2, Ivan A. Rashid3,4, Daan Kuppens5,6, Merel van der Thiel7,8, Petra van Houdt9, Paulien HM Voorter7,8, Eric T Peterson10, and Oliver Gurney-Champion5,6
1Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden, 2Department of Medical Physics and Biomedical Engineering, Sahlgrenska University Hospital, Region Västra Götaland, Gothenburg, Sweden, 3Medical Radiation Physics, Department of Translational Medicine, Lund University, Malmö, Sweden, 4Radiation Physics, Department of Hematology, Oncology and Radiation Physics, Skåne University Hospital, Lund, Sweden, 5Department of Radiology and Nuclear Imaging, Amsterdam UMC location University of Amsterdam, Amsterdam, Netherlands, 6Imaging and Biomarkers, Cancer Center Amsterdam, Amsterdam, Netherlands, 7Department of Radiology & Nuclear Medicine, School for Mental Health & Neuroscience, Maastricht University Medical Center, Maastricht, Netherlands, 8School for Mental Health & Neuroscience, Maastricht University, Maastricht, Netherlands, 9Department of Radiation Oncology, Netherlands Cancer Institute, Amsterdam, Netherlands, 10Biosciences, Neuroscience Program, SRI International, Menlo Park, CA, United States

Synopsis

Keywords: Software Tools, Perfusion, Reproducible research

Motivation: Lack of validated and open-source intravoxel incoherent motion (IVIM) post-processing and fitting code is hindering reproducible research, limiting the validation and large-scale roll-out of IVIM imaging.

Goal(s): To create an open-source code repository for IVIM-related code.

Approach: Scientists interested in IVIM are encouraged to upload their code to our open-source code repository built by the ISMRM OSIPI task force 2.4, where automated testing and evaluation based on reference data are used to enable quality control of the code.

Results: As of November 2023, 19 code contributions have been submitted by 6 different institutes, all passing automated testing.

Impact: The work of ISMRM OSIPI task force 2.4 enables an open-source platform for validated code relevant to intravoxel incoherent motion (IVIM) imaging, thus reducing duplicate development, improving reproducibility, and serving as a benchmark for future methods.

Introduction

Intravoxel incoherent motion (IVIM) analysis of diffusion-weighted MRI data provides information on both diffusion and perfusion from a single MRI scan.1 IVIM has great potential for several purposes, e.g. for cancer treatment evaluation.2 However, IVIM has not found its way into daily routine yet as results in literature are inconclusive, for example, regarding histological validation.3 This can partly be attributed to varying methodological choices, including choice of model-fitting algorithms.

To address the problem of inconsistent IVIM results, and to promote transparency and reproducibility within the IVIM research field, a task force dedicated to sharing IVIM-related software code has been initiated as part of the ISMRM open science initiative for perfusion imaging (OSIPI).4 Here, we present the OSIPI IVIM code repository and initial evaluations of current code contributions.

Repository

An OSIPI Github repository was created (https://github.com/OSIPI/TF2.4_IVIM-MRI_CodeCollection), featuring 1) instructions for code contribution, 2) wrapper code, 3) automated tests, and 4) reference datasets for testing and evaluation (Fig. 1).

Code contribution: Instructions for contributing IVIM-related code are available at the repository main page. To facilitate simple code contribution, no strict rules regarding code language or input arguments are applied.

Wrapper code: A wrapper class providing a common programming interface to all code contributions was implemented for simple usage, testing and evaluation. For each submission of model-fitting code, a standardized fit class inheriting from the wrapper is defined for easy-to-use fitting with standardized inputs and outputs.

Automated tests: Code adapted to the wrapper is evaluated by automated unit tests, using simulated data, to ensure that code and wrapping is working as intended. A test is considered successful if the output of tested code is within predefined margins from ground truth.

Datasets: Reference data sets consist of simulated and in vivo diffusion MRIs of the brain and abdomen. Simulations use IVIM implementations of brain5,6 and abdominal7,8 digital phantoms with literature-based IVIM parameter values. In vivo data were obtained in healthy volunteers.

Experiments

Unit tests were run as described above, with relative and absolute tolerances of 5/5/25 and 0.01/0.01/0.1 for f/D/D*, respectively, using simulated signal values from the abdominal phantom.

To further characterize code contributions, biexponential IVIM model-fitting approaches were applied to simulated data using all applicable contributions (excluding non-Python code and algorithms with other outputs than f/D/D*, like triexponential fits).

Results

As of November 2023, 19 model-fitting approaches were submitted by 6 different institutes. Code included least-squares (6), segmented (9), Bayesian (2), and variable-projection (2) fitting written in Python (17) or MATLAB (2). All code passed unit testing.

At high SNR levels, most algorithms describe data accurately (Fig. 2), and show accurate and precise parameter estimates (Figs. 3-5). For lower SNR levels, predictions start to differ (Fig. 2), and the variation in systematic and random errors increases (Figs. 3-5).

Discussion

The fact that different code contributions result in substantially different parameter estimates from the same data underlines the need for open-source and standardized IVIM fitting algorithms. Current variety of in-house IVIM software is hence likely a source of the variation found in IVIM literature. Our comparisons can aid when comparing results of previous studies, and suggest choices of model-fitting code for future studies.

Code contributions submitted were all model-fitting algorithms and mainly written in Python. Much code used in the IVIM literature is however written in MATLAB, from which we hope to see additional submissions. Code contributions for other purposes like b-value optimization and preprocessing are also encouraged.

Currently, our main goal is to gather IVIM-related code from different groups, such that code snippets that otherwise would have remained private become open to the IVIM community. A long-term goal is to build upon this to form a common open-source (Python) package available to all IVIM researchers.

The testing is currently limited to simulations of healthy tissue. Evaluation of code based on in vivo data is an upcoming prioritized topic. Introducing phantoms with pathology can also increase the impact of the testing and evaluation.

In conclusion, our results highlight the need for transparency and standardization in the IVIM research field. Use of a common code base has potential to increase reproducibility and move IVIM towards clinical routine.

How to contribute

There will be a continuous call for code contributions during the whole roadmap (2023-2025) and we strongly encourage researchers to share their IVIM-related code. Information on how to submit code or getting involved in the task force can be found here: https://osipi.ismrm.org/task-forces/task-force-2-4/.

Acknowledgements

We thank Stefan Zijlema for his work in the establishment of the task force and Koen Baas for code contribution.

OJ is funded by grants from the Swedish state under the agreement between the Swedish government and the county councils, the ALF-agreement (ALFGBG-942664). DK and OG-C are funded by the KWF Dutch Cancer Society (KWF-UVA 2021.13785). ETP is funded by the NIH grant Free Water Imaging in Parkinson’s Disease (R21 NS 13210101).

References

  1. Le Bihan D. What can we see with IVIM MRI? NeuroImage. 2019;187:56-67.
  2. Iima M. Perfusion-driven Intravoxel Incoherent Motion (IVIM) MRI in Oncology: Applications, Challenges, and Future Trends. Magn Reson Med Sci. 2021;20(2)125-138.
  3. Federau C. Intravoxel incoherent motion MRI as a means to measure in vivo perfusion: A review of the evidence. NMR Biomed. 2017;30(11):e3780.
  4. https://osipi.ismrm.org/task-forces/task-force-2-4/
  5. Fonov V, Evans AC, Botteron K, et al. Unbiased average age-appropriate atlases for pediatric studies. NeuroImage. 2011;54(1):313-327.
  6. Fonov V, Evans AC, McKinstry RC, et al. Unbiased nonlinear average age-appropriate brain templates from birth to adulthood. NeuroImage. 2009;47:S102.
  7. Wissman L, Santelli C, Segars WP, Kozerke S. MRXCAT: Realistic numerical phantoms for cardiovascular magnetic resonance. J Cardiovasc Magn Reson. 2014;16(1):63.
  8. Schrauben E. The ERIC phantom - Extra-dimensional respiration and inflow of contrast: an open-source DCE digital simulation tool. Annual Meeting ISMRM. 2023;#3063.

Figures

Figure 1. The created repository (https://github.com/OSIPI/TF2.4_IVIM-MRI_CodeCollection) contains wrapper code to harmonize calls to all code contributions and reference data for testing and evaluation

Figure 2. Signal vs. b as predicted by different fitting algorithms (color) and ground truth (black). The columns are different SNR levels and the rows are different tissue types

Figure 3. Estimated values of f using different fitting algorithms with ground truth in red. The rows are different SNR levels and the columns are different tissue types

Figure 4. Estimated values of D using different fitting algorithms with ground truth in red. The rows are different SNR levels and the columns are different tissue types

Figure 5. Estimated values of D* using different fitting algorithms with ground truth in red. The rows are different SNR levels and the columns are different tissue types

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
0540
DOI: https://doi.org/10.58530/2024/0540