0234

Smart-Uploader: an automatic tool for medical image classification and quality protocol adherence

Óscar Peña-Nogales¹, Evie Neylon¹, Tommy Boshkovski¹, Marc Ramos¹, Paulo Rodrigues¹, Vesna Prčkovska¹, and Kire Trivodaliev¹
¹QMENTA Inc, Boston, MA, United States

Synopsis

Keywords: Software Tools, Software Tools

Motivation: Imaging biomarkers are becoming a cornerstone to increase throughput and efficiency of large clinical trials. However, the diversity of imaging modalities used to derive them creates complexity for both imaging protocols and image archiving systems.

Goal(s): To develop a tool to automatically classify imaging modalities and assess their adherence to the predefined acquisition protocol.

Approach: The combination of a few-shot learning classifier trained to classify image modalities according to their contrast characteristics and a deterministic heuristic approach based on the DICOM headers.

Results: The proposed approach displays potential for automatic online image classification and identification of protocol deviations, increasing clinical trial operational efficiency.

Impact: The proposed joint approach automatically classifies all medical imaging data and assesses its adherence to the predefined acquisition protocol. Consequently, it not only facilitates data management but also identifies protocol deviations increasing the operational efficiency of clinical trials.

Purpose

Imaging biomarkers are becoming a cornerstone in both daily clinical practice and large clinical trials due to their potential to objectively measure underlying biological processes of clinical relevance^1,2. Multiple biomarkers can be identified using a diversity of MRI acquisitions (i.e., T1 weighted imaging, T2 weighted imaging, diffusion weighted imaging, and susceptibility weighted imaging among others). These biomarkers assist diagnosis, facilitate understanding of diseases, evaluate disease progression, and assess the efficacy of pharmaceutical treatments. However, the complexity of advanced MRI acquisitions and their lack of standardization hinder their usage in large clinical trials³. This usage can be additionally hampered by complicated MRI consoles and insufficient training of operators. Consequently, acquisition protocol deviations are not uncommon and they can lead to the need for patient re-scans or even data disqualification^4,5; both actions result in cost increases. Furthermore, the aforementioned diversity of MRI modalities creates complexity for image archival systems such as PACS and cloud-based solutions.

In this work we propose a tool, named Smart-Uploader, to automatically classify medical imaging files and to verify their adherence to the predefined acquisition protocol. This tool is based on the joint contribution approach of a deep learning method and a deterministic heuristic algorithm.

Methods

The Smart-Uploader is coded in Python and integrated within a centralized cloud-based platform for data management and analysis. After the MRI acquisition, the imaging data is uploaded to the platform via PACS or drag&drop. The Smart-Uploader first removes all the protected health information to subsequently extract the meta-information available in the DICOM headers and group the files. Next, it sequentially combines the information from: 1) a few-shot learning classifier based on triplet ranking networks trained to classify image modalities according to their contrast characteristics (see Puch et al. ⁶ for further details), and 2) a deterministic heuristic applied to the DICOM headers based on a decision tree. Following this procedure, the Smart-Uploader is able to classify and tag all the files uploaded. Finally, based on a set of predefined rules, the Smart-Uploader analyzes the compliance of the uploaded data to the imaging protocol. Some examples of these rules in JSON format are shown in Table 1. After execution, the Smart-Uploader provides a summary report of the data available and its adherence to the acquisition protocol.

Results

The execution time of the Smart-Uploader is around 3 min for a standard MRI session containing T1-weighted, T2-weighted, T2*-weighted, T2 FLAIR, fMRI, and DTI data. Further, the Smart-Uploader is able to classify 20 MRI modalities as shown in Figure 1. A sub-classification is also performed when appropriate. Protocol adherence is assessed by applying up to 50 rules (see Figure 2 for an example summary report). The versatility of the rules allows to perform an across-vendors comparison to facilitate data harmonization and to avoid human error.

A case study of a phase 2 Alzheimer’s Disease clinical trial is considered, with 100 patients and 13 institutions with varying MR hardware and software. Considering it would take an expert up to 25 min to perform the classification and protocol adherence assessment manually. Burden reduction was estimated to be 88%, with increased site responsiveness and engagement reducing errors and delays. The trial attained 95% of its initial data collection projection, improving the overall quality of the results.

Discussion

We propose a joint deep learning and deterministic heuristic approach for the automatic classification of medical images as well as to check the adherence of the data to the acquisition protocol. The classification is capable of successfully identifying as many as 20 MRI modalities and the protocol adherence facilitates the quality assessment of the acquired data. See for example the acquisition deviation shown in Figure 2 where the voxel size of the T1 acquisition is not compliant with the protocol. See also the conformance of the acquired diffusion images with opposite phase encoding directions to allow performing gradient field map corrections.

Future lines of research include the extension of the classification to also identify other MRI modalities such as magnetization transfer imaging or MR spectroscopy. In addition, the extension of the quality protocol adherence to assess image quality and identify artifacts is also desirable.

Conclusion

The proposed automatic tool for medical image classification and protocol adherence facilitates data management and identification of protocol deviations. The results suggest the potential of the tool to increase the quality of imaging endpoints, generate time-savings, and leverage operational efficiency in clinical trials.

Acknowledgements

No acknowledgement found.

References

1. Dregely I, et al. Imaging biomarkers in oncology: Basics and application to MRI. J of Magn Reson Imaging. 48.1 (2018): 13-26.

2. Salvatore C, et al. Magnetic resonance imaging biomarkers for the early diagnosis of Alzheimer's disease: a machine learning approach. Front in neuroscience 9 (2015): 307.

3. Yankeelov T. E., et al. The role of magnetic resonance imaging biomarkers in clinical trials of treatment response in cancer. Sem in oncology. Vol. 38. No. 1. WB Saunders, 2011.

4. Ohri N, et al. Radiotherapy protocol deviations and clinical outcomes: a meta-analysis of cooperative group clinical trials. J of the Nat Canc Inst 105.6 (2013): 387-393.

5. Ghooi, R B., et al. Assessment and classification of protocol deviations. Perspectives in clinical research 7.3 (2016): 132.

6. Puch S., Few-shot learning with deep triplet networks for brain imaging modality recognition. First MICCAI Workshop, DART 2019, and First International Workshop, MIL3ID 2019, Shenzhen, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13 and 17, 2019

Figures

Table 1: Example rules in JSON format and their definition used to assess the adherence of the acquired data to the predefined protocol. “operator” indicates the operation to perform; one of: “==“, “>=“, “<=“, “>“, “<“, “!=“, “in“, “in lower“, or “not in“. “value_type” is the value type of the “key”; one of: “number“, “string“, or “list“. “key” is either the corresponding DICOM tag to be checked or “SU_Modality” to check the classification output provided by the Smart-Uploader. “value” corresponds to the value that the “key” is compared to.

Figure 1: Set of MRI modalities the classifier is able to identify. The sub-classification tags are also added, when appropriate, to distinguish between acquisitions. For instance, spin echo (SE) versus gradient echo (GR) or T2-weighted versus T2 FLAIR. The color code of the different modalities is used to facilitate visual identification. Plain text shows the name of the image and its size is between brackets.

Figure 2: Example summary report of the protocol adherence assessment. All files are checked for multiple rules applied to different acquisition parameters. It can be seen how the protocol adherence assessment failed on the 3D_T1 acquisition due to an incorrect voxel size. Additionally, see how the phase encoding direction of both the diffusion tensor imaging (DTI) acquisition and the DTI gradient field map (DTI GFM) acquisition is the opposite (i.e., “j-”: anterior -> posterior versus “j”: posterior -> anterior, respectively).

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)

0234

DOI: https://doi.org/10.58530/2024/0234