3005
An extendable web-based tool for online radiological assessment of imaging protocols1Centre for Medical Image Computing (CMIC), Medical Physics & Biomedical Engineering, University College London, London, United Kingdom, 2Advanced Research Computing (ARC) Centre, University College London, London, United Kingdom, 3Dementia Research Centre (DRC), UCL Queen Square Institute of Neurology, University College London, London, United Kingdom, 4Department of Brain Repair and Rehabilitation, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom, 5NMR Research Unit, Queen Square MS Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom, 6Centre for Medical Image Computing (CMIC), Computer Science, University College London, London, United Kingdom, 7Department of Radiology and Nuclear Medicine, Amsterdam UMC, Amsterdam, Netherlands, 8Bioxydyn Limited, Manchester, United Kingdom
Synopsis
Keywords: Software Tools, Software Tools
Motivation: Available online reading tools were not readily extendable for our comparative and inter-rater radiological assessment of an ultra-fast MRI protocol for dementia diagnosis.
Goal(s): We sought to develop a review tool to enable scalable, geographically-distributed review of imaging protocols for diagnostic use.
Approach: We designed a web-based review tool and evaluated it for our needs, whilst ensuring flexibility for use across MRI studies.
Results: We developed the Extendable Review Tool using popular technologies to ease configuration for future studies. We tested the tool using a dementia-specific questionnaire to compare ultra-fast and standard of care MRI scans across a range of contrasts.
Impact: Our Extendable Review Tool is customisable for any MRI-based study where geographically-distributed clinical reading of a set of images is needed. As an extendable and flexible web-based tool it has potential to aid a broad range of studies.
Introduction
The translation of novel imaging techniques into the clinic requires that the images produced can be used diagnostically by clinicians within the disease area of interest. In our work to develop and validate ultra-fast dementia diagnosis MRI protocols1 we encountered the challenge of presenting ultra-fast and standard clinical images to a diverse expert audience. This was needed to allow blinded assessment of the two protocols for clinical utility in dementia diagnosis, and to do so in a way that reduced burden for busy clinicians, as well as that of downstream data analysis.To enable this, we developed a flexible, web-based image review tool that allows reviewers to progress through a set of image series from a study and assess each using a simultaneously presented customisable questionnaire (figure 1).
Design
The Extendable Review Tool (ERT) was co-designed by a research software engineer with medical image computing expertise and a neurologist specialising in dementia; this aided in ensuring shared understanding of the clinical context and the technical implementation.We wrote the software using Flask2, a lightweight and extensible Python web framework. This allows the ERT to be accessible to reviewers solely through a modern web-browser reducing barriers to use on locked down systems. The choice of Python enables researchers with existing expertise in the language to modify the system more easily to their needs.
As illustrated in figure 2, researchers can interact with the ERT to define projects that store information about imaging sessions. These imaging sessions can contain one or more series. A set of questions can then be defined using JSON and associated with the project. Reviewers are then assigned to these studies that associate imaging data and a question set and can proceed through the set of images with their clinical reads recorded.
Features
Support for over 30 imaging formats through NiiVue Viewing of imaging data is achieved through NiiVue3 , a WebGL2 based imaging viewer that supports loading NIfTI, DICOM and many other common formats. We implemented basic image viewing controls to manipulate the NiiVue canvas.Customisable clinical read questionnaire through SurveyJS The tool uses the open-source SurveyJS4 library to allow full customisation of the clinical read questionnaire. A visual online survey designer can be used to generate the questionnaire JSON, or this can be edited directly. This allows full specification of the questions, options, form validation and visual presentation – allowing researchers to maintain integrity of the data collected whilst also easing the review workflow (figure 3).
Reviewer Blinding and De-identification Where multiple imaging protocols are being compared, metadata about the series can bias reviewers – the ERT allows researchers to hide this information from reviewers, as well as define the order in which image sessions are presented so that imaging from the same subject is not adjacent. Reviewer personal information is kept separate to their responses, so results can be easily extracted without providing details about individual users.
Results
This tool is planned for use within an ongoing dementia study1 where it has been implemented with a dementia-specific question set – the JSON description and the rendered form are shown in figure 3.For our use case, we required that several contrasts are presented for review, and the questionnaire filled with respect to the whole session (figure 4). Results from reviewed sessions can be exported in a structured JSON format (figure 5) for further analysis.
Discussion
Whilst similar tools exist such, they are either study-specific and difficult to extend, or part of non-free commercial packages (e.g. Flywheel’s multi-reader study functionality5, Nora medical imaging platform6) that reduce availability to the community.The extendable nature of our tool enables it to be easily used for other reader studies, involving any contrast or anatomy, where a questionnaire needs to be presented alongside a set of images through a web-browser. A new reading study with its own images and customised questionnaire can be set-up with zero code changes reducing time spent configuring the tool. The use of Python, a popular programming language in the community, enables further customisation of the base tool.
Beyond extendibility and availability, through NiiVue our approach can support loading and overlay of mesh-based formats such as those outputted from FreeSurfer7. This will be beneficial in multi-reader studies where segmentation and parcellation are being assessed.
Conclusion
Our web-based Extendable Review Tool provides a flexible base for comparing images from different protocols. It fulfils our use case to assess ultra-fast MRI protocols for dementia diagnosis. In addition, the design and implementation choices enable its usefulness in our future work and other MRI-based studies.Acknowledgements
This research was supported by funding from Alzheimer’s Society (grant number 577 (AS-PG-21-045)). This work is also partially supported by the EPSRC-funded UCL Centre for Doctoral Training in Intelligent, Integrated Imaging in Healthcare (i4health) (EP/S021930/1), UCL’s Advanced Research Computing (ARC) Centre, and by the National Institute for Health and Care Research University College London Hospitals Biomedical Research Centre (NIHR UCLH BRC).References
1. Chughtai HR, Thomas DL, Rosa-Grilo M, Mulroy E, Beament M, Coath W, Prosser L, Malone I, Alexander D, Barkhof F, Mummery CJ, Fox NC, Parker GJM. An ultra-fast dementia diagnosis MRI protocol enabled by Wave-CAIPI. In: ISMRM & ISMRT Annual Meeting & Exhibition. ; 2023.
2. pallets/flask. Published online October 30, 2023. Accessed October 30, 2023. https://github.com/pallets/flask
3. Taylor Hanayik, Chris Rorden, Christopher Drake, alexis, Paul Taylor, Nell Hardcastle, Paul McCarthy, Anthony Androulakis, Chris Markiewicz, Pierre. niivue/niivue: 0.37.0. Published online 2023. doi:10.5281/zenodo.8320937
4. surveyjs/survey-library. Published online October 30, 2023. Accessed October 30, 2023. https://github.com/surveyjs/survey-library
5. Designing Custom Multi-Reader Studies is Streamlined with Robust Viewer Configuration Options. Published May 5, 2023. Accessed October 30, 2023. https://flywheel.io/insights/academic-research-data-management/multi-reader-studies-setup
6. Anastasopoulos C, Reisert M, Kellner E. “Nora Imaging”: A Web-Based Platform for Medical Imaging. In: Neuropediatrics. Vol 48. Georg Thieme Verlag KG; 2017:P26. doi:10.1055/s-0037-1602977
7. Fischl B. FreeSurfer. NeuroImage. 2012;62(2):774-781. doi:10.1016/j.neuroimage.2012.01.021
Figures
a) Study-specific questionnaires (e.g. for dementia assessment as shown) can be defined using a JSON data structure. Presentation and validation rules are also defined in the same manner.
b) The defined information is automatically rendered into a web form using the SurveyJS library
A) Sagittal view of a T1w scan. B) Axial view of a T2w scan C) Coronal view of a FLAIR scan.
Each read produces:
a) JSON output that can be exported from the application alongside
b) information about the associated images to perform downstream statistical analysis between any groups present in the project (e.g. standard of care vs ultra-fast sequences).