3820
The Open Source Initiative for Perfusion Imaging (OSIPI)
Laura C. Bell1, Henk Mutsaerts2, Andrey Fedorov3, Zaki Ahmed4, Patricia Clement5, Simon Levy6, Frank G Zollner7, Jan Petr8, Sudipto Dolui9, Kathleen Schmainda10, Melissa Prah10, Matthias Schabel11, Ananth Madhuranthakam12, Li Zhao13, Michael Thrippleton14, Petra van Houdt15, James Holmes16, C. Chad Quarles1, Greg Cron17, David L Thomas18, Yuriko Suzuki19, Ina Kompan20, David L Buckley21, Paula Croal22, Udunna Anazodo23, Anahita Fathi Kazerooni9, Hamidreza Saligheh Rad24, Charlotte Debus25, and Steven P Sourbron26
1Barrow Neurological Institute, Phoenix, AZ, United States, 2Amsterdam University Medical Cente, Amsterdam, Netherlands, 3Brigham and Women’s Hospita, Boston, MA, United States, 4Mayo Clinic, Rochester, MN, United States, 5Ghent University, Ghent, Belgium, 6Aix-Marseille Univ, Marseille, France, 7Heidelberg University, Mannheim, Germany, 8Helmholtz-Zentrum Dresden-Rossendorf, Dresden, Germany, 9University of Pennsylvania, Philadelphia, PA, United States, 10Medical College of Wisconsin, Milwaukee, WI, United States, 11Oregon Health and Science University, Portland, OR, United States, 12UT Southwestern Medical Center, Dallas, TX, United States, 13Children’s National Medical Center, Washington DC, DC, United States, 14University of Edinburgh, Edinburgh, United Kingdom, 15the Netherlands Cancer Institute, Amsterdam, Netherlands, 16University of Wisconsin - Madison, Madison, WI, United States, 17Ottawa Hospital Research Institute, Ottawa, ON, Canada, 18University College London, London, United Kingdom, 19University of Oxford, Oxford, United Kingdom, 20German Cancer Research Center, Heidelberg, Germany, 21University of Leeds, Leeds, United Kingdom, 22University of Nottingham, Nottingham, United Kingdom, 23Western University, London, ON, Canada, 24Tehran University of Medical Sciences, Tehran, Iran (Islamic Republic of), 25Karlsruhe Institute of Technology, Karlsruhe, Germany, 26University of Sheffield, Sheffield, United Kingdom
1Barrow Neurological Institute, Phoenix, AZ, United States, 2Amsterdam University Medical Cente, Amsterdam, Netherlands, 3Brigham and Women’s Hospita, Boston, MA, United States, 4Mayo Clinic, Rochester, MN, United States, 5Ghent University, Ghent, Belgium, 6Aix-Marseille Univ, Marseille, France, 7Heidelberg University, Mannheim, Germany, 8Helmholtz-Zentrum Dresden-Rossendorf, Dresden, Germany, 9University of Pennsylvania, Philadelphia, PA, United States, 10Medical College of Wisconsin, Milwaukee, WI, United States, 11Oregon Health and Science University, Portland, OR, United States, 12UT Southwestern Medical Center, Dallas, TX, United States, 13Children’s National Medical Center, Washington DC, DC, United States, 14University of Edinburgh, Edinburgh, United Kingdom, 15the Netherlands Cancer Institute, Amsterdam, Netherlands, 16University of Wisconsin - Madison, Madison, WI, United States, 17Ottawa Hospital Research Institute, Ottawa, ON, Canada, 18University College London, London, United Kingdom, 19University of Oxford, Oxford, United Kingdom, 20German Cancer Research Center, Heidelberg, Germany, 21University of Leeds, Leeds, United Kingdom, 22University of Nottingham, Nottingham, United Kingdom, 23Western University, London, ON, Canada, 24Tehran University of Medical Sciences, Tehran, Iran (Islamic Republic of), 25Karlsruhe Institute of Technology, Karlsruhe, Germany, 26University of Sheffield, Sheffield, United Kingdom
Synopsis
Open Source Initiative for Perfusion Imaging (OSIPI) was founded by the ISMRM Perfusion Study Group as a community-driven initiative. Supported by six distinct aims, its mission is “to promote the sharing of perfusion imaging open-source software in order to eliminate the practice of duplicate development, improve the reproducibility of perfusion imaging research, and speed up the translation into tools for discovery science, drug development, and clinical practice”. OSIPI seeks to provide centralized resources to deliver reproducible perfusion research. In addition, it provides a platform for exchange where new and more advanced methods may be validated for perfusion accuracy.
Introduction
In 2018, members of the ISMRM Perfusion Study Group recognized reproducibility as a primary blocker for adoption of perfusion MRI in clinical decision making. A community-led initiative to promote open-science was created to address this issue [1-2], starting with a survey to probe the support and particular interests of the community. We present results of the survey, and the implementation of the conclusions leading up to the Open Source Initiative for Perfusion Imaging (OSIPI).Study Group Survey
In the fall of 2018, a call-to-action survey was sent out through the ISMRM Perfusion Study Group (n = 626 members) to assess support for a perfusion-based open-science initiative. This survey asked “How are you interested in helping out?” with five categories presented that revolved around open-source software development and sharing data, plus an opportunity to comment in free text (Table 1). survey participants were matched to the perfusion study group demographics provided by the ISMRM office for 2018-2019 years to assess demographics of interest (Table 2).102 individual responses were returned (study group response rate of 16.3%) with some individuals willing to help out in more than one way (n = 122) (Table 1). Table 2 lists the demographics of the survey participants. Strong interest came from Europe (60% of participants) when compared to the makeup of the study group (37% of members). ISMRM Trainees are a majority within the study group (74% of members) but represented only 41% of survey participants. All responses were in support of this proposed initiative. Most indicated interest to develop code (29%), provide data for testing (25%), and share code (24%). 20% of the responses indicated willingness to help in non-technical ways such as managing, consulting, and advertising. Many participants (47%) provided additional comments. Table 3 lists a subsampling of the responses not directly pertaining to software development. Figure 1 is a word cloud of all the responses in order to visualize comments provided.
The survey disclosed a strong need and community support for a perfusion open source initiative. While the survey originally centered on open-source software development, which is supported by the community, it also brought to light a strong desire for more emphasis on standardization and validation.
Implementation of Survey Conclusions
Early in 2019, founding members were identified from the survey respondents. They organized launch events during and after the ISMRM in Montreal, and defined the strategy and governance of OSIPI (Figure 2). OSIPI’s mission is defined as: “to promote the sharing of perfusion imaging open-source software in order to eliminate the practice of duplicate development, improve the reproducibility of perfusion imaging research, and speed up the translation into tools for discovery science, drug development, and clinical practice”. The mission will be accomplished by six aims that are addressed by 1-3 task forces generally split by non-contrast (ASL) and contrast agent-based (DSC and DCE) perfusion. The aims are software inventory, software toolbox, data and reference objects inventory, reporting standards, a platform for exchange, and benchmarking (Figure 3). Task forces were defined to address specific objectives.The plans were accepted by the ISMRM Perfusion Study Group and the ISMRM Executive Committee in November 2019. The OSIPI executive and management board was populated with respondents on the survey and OSIPI was launched in December 2019. OSIPI has no predefined end date and is designed to evolve as the need for research coordination changes.
Current Status and Outlook
The work in OSIPI is carried out in 2-year delivery cycles, with the first cycle starting formally in May 2020, and with the milestones and deliverables outlined in a roadmap which is revised and updated at 3-monthly intervals (osipi.org/quicklinks/). In this first cycle, OSIPI expects 11-12 manuscripts focused on a comprehensive inventory of software platforms, classification and validated contributed code, inventory and sharing guidelines for digital reference objects and open-access perfusion data for both clinical and preclinical purposes, reporting guidelines and formats, and results from challenges focused on software validation.The task forces are on track to deliver against those timelines (www.osipi.org/news/). ASL/DCE/DSC software is identified and summarized into central inventories, code is being collected that will be tested, refined, and publicly available, contributions for phantoms and reference objects are being collected, as well as an inventory of publicly available pre-clinical and clinical datasets, lexicons for ASL and DCE/DSC are in draft stage, and two challenge designs are finalized. All papers will be published in the same issue of Magnetic Resonance in Medicine in 2022 if accepted after peer-review and will also feature in a dedicated virtual issue of the journal.
Acknowledgements
No acknowledgement found.References
[1] Stikov N, Trzasko JD, Bernstein MA. Reproducibility and the future of MRI research. MRM 2019; 82(6): 1981-1983.
[2] Kapur T, Pieper S, Fedorov A, et al. Increasing the impact of medical image computing using community-based open-access hackathons: The NA-MIC and 3D Slicer experience. MedIA 2016; 33: 176-180.
Figures
Figure 1: A word cloud of the comments (produced by wordart.com). Words such as “standard”, “evaluation”, “tested”, “validated” indicate a consensus that instead of another software platform, the community really needs perfusion to be benchmarked and standardized.
Figure 2: The OSIPI governance structure is designed to ensure continuous progress and unify the separate aims that support its mission. This ideally makes the initiative a sustainable long term initiative that can adapt to current needs within the perfusion community.
Figure 3: OSIPI’s mission to support open perfusion science is done through the 6 aims as outlined here each with the associated task forces.
Table 1: The call-to-action survey results from the 2018 call-to-action survey indicating how volunteers wanted to participate in an open-source perfusion platform.
Table 2: Demographics of the perfusion study group and the survey participants. Career stage (trainee/non-trainee) was defined by ISMRM SG membership. Some survey participants were not members as it’s possible the survey was forwarded. This accounts for the “unknown” career stage category since it was not required to be filled in.
Table 3: An abbreviated selection of the feedback from the perfusion community regarding current perfusion needs outside of software development from the 2018 call-to-action survey.