Purpose

The automated classification of MRI sequences remains mostly unsolved. Existing machine learning methods, while available, are marked by substantial computational demands and limited adaptability across varying datasets1. The need for larger and more diverse data sources keeps rising but the lack of an established toolkit and the limited adoption of standard terminology3 for labeling MRI acquisitions makes the curation of any large-scale dataset challenging. To address this challenge, we developed fw-classification, an open-source python package designed to programmatically classify MRI acquisitions based on their DICOM header properties and a set of configurable rules, written in YAML. In this work, we present the functionality of the package as we set out to define rules, based on heterogeneous DICOM metadata from various manufacturers and guided by MR physics2, to robustly classify MR images. This innovative package aims to address the challenge of large and diverse metadata standards, thereby facilitating a standardized and accurate classification process for MRI scans across sites and vendors.

Software Package

At the core of fw-classification is a flexible engine that is configured using a YAML profile which defines a set of rules. Once configured, this engine can process input JSON metadata extracted from DICOM series to produce metadata enhanced with the inferred classification (Figure 1). These rules are crafted to carry out logical assessments and executing actions that include setting and updating metadata fields contingent upon matching conditions. The declarative rules are highly adaptable and allow for comprehensive control over a wide spectrum of metadata structures. This adaptability is crucial, allowing the package to be customized and enriched to cater to a vast array of metadata structures from different modalities and manufacturers. At the highest level, the YAML profiles allow for inheritance and cross-referencing, which then allows for breaking profiles into functional modules. Each YAML profile is further composed of block components that include `match` statements for identifying relevant metadata and `actions` to define the subsequent classification logic. The `match` component uses key-value pairs to locate specific metadata within the JSON, employing pattern matching for complex queries. Specifically, a set of 12 operators is available to define arbitrarily complex matching criteria. The `actions` component then specifies how to modify the JSON metadata, hence classifying the scan based on the matched metadata, by setting/appending/updating classification values. An example profile for the classification of Siemens T2* scans can be seen in Figure 2.

We make available a set of profiles using logic derived from MR physics to classify T1, T2, T2*, T2 FLAIR, and PD Scans based on MRI DICOM metadata acquired from three major MRI manufacturers: GE, Siemens, and Philips. These profiles are released as an open-source repository named fw-classification-profiles on GitLab. The profiles underwent a rigorous calibration and validation process, wherein the resulting output was aligned with two proprietary ground truth datasets consisting of >10k MRI scans each using the Flywheel platform. These datasets were provided and curated by Contract Research Organizations (CROs), and were instrumental in calibrating the rules, ensuring consistent and accurate classification across a wide variety of scan parameters and features.

Application

Empirical tests of fw-classification have demonstrated an accuracy rate exceeding 98% in the classification of various MRI scans (Table 1). This high level of accuracy affirms the package's capacity to efficiently manage complex metadata from a multitude of manufacturers and to accurately classify scan attributes across a variety of scan parameters. The use of YAML-based rules facilitates the definition of a sophisticated and flexible classification logic while preserving human readability, enabling the package to be easily customized to handle diverse metadata from several manufacturers.

Discussion

Although the package's performance on private datasets is promising, further verification and validation with public datasets is essential. This additional step will ensure the package's capability and stability across a broader range of clinical and research environments. Expanding the validation will help to establish the package's utility as a universal standard and reinforce its reliability in the field. Additionally, exposure to more data will test the package’s applicability to a diverse range of modalities beyond MR.

Conclusion

We introduced the fw-classification package and its associated YAML-based profiles as a method to classify MRI scans using an approach that is scalable, robust, and precise. This method provides a streamlined and accurate framework for MRI scan classification, enhancing the efficiency and precision of data organization in medical imaging and enabling standardization at scale across a diverse dataset. The flexible architecture of the package and classification profiles allows for applicability to a wide array of modalities and manufacturers.

Acknowledgements

This work was supported by Flywheel-io.