Introduction

Whole-Organ Magnetic Resonance Imaging Score(WORMS)¹ is a semi-quantitative scoring system developed in an attempt to standardize the Knee MRI readings. Such grading systems are widely used in a research setting. However, the clinical application is hampered by the time and level of expertise needed to reliably perform the reading, making the automation of this task appealing for a smoother and faster clinical translation. The goal of this study is to fill this void by capitalizing on recent developments in Deep Learning (DL) applied to medical imaging. Specifically, we aim to (i) identify cartilage, meniscus, bone marrow edema (BEM) and ACL ligament lesions and assess severity providing full knee lesion severity assessment, and (ii) provide a condensed clinical history of patients in an automated manner.

Methods and Data

The present study was based on a dataset composed of 1,435 knee MRI from subjects with and without OA and after ACL injury. Images were acquired from three previous studies, all of which used a high-resolution 3D fast spin-echo(FSE) CUBE sequence TR/TE=1500/26.69ms, field-of-view=14cm, matrix=512-by-512, slice-thickness=0.5mm, bandwidth=50.0kHz. Distributions of patient demographics are as follows: age=42.79±14.75 years, BMI=24.28±3.22Kg/m², 48/52 male/female split. A 3D V-Net² neural network(NN) architecture was used to learn segmentations of the 11 ROIs as described above, using as labels 399 subject volumes(3D images) that were manually segmented as training data. In order to optimize the segmentation problem, we utilized two V-net architectures. The first V-net outputs segmentations for 5 classes(Figure 1A), namely 3 cartilage classes(femur, tibia and patella), one class for meniscus horns, and one for the background(BG). The second V-net(Figure 1B), takes as input the same preprocessed MRI used in the first V-Net and also uses, as a second channel, the output of the 5-class segmentation generated by the first V-Net. The second V-Net then assigns 11 labels to the compartments segmented by the first V-net. The 11 classes are 6 cartilages(patella, trochlea, medial and lateral tibia and medial and lateral femur) +4 menisci horns +BG. After applying the segmentation bounding boxes(BBs) around the 6 cartilage compartments and 4 meniscus horns were extracted, resulting in the volumes of interest(cVOIs)(Figure 1C). A rule-based approach was applied to isolate the ACL, using the limits of the cartilage BBs. cVOIs were randomly divided with a 70/15/15% split into training, validation, and testing/holdout datasets, keeping the distributions of lesion severity per compartment. 3 class labels were generated for each knee tissue(Figure 2A). The cropped volumes were input into a 3D CNN composed by six layers, including one skip connection after the first convolutional layer, intended to preserve multi-scaled features and mitigate overfitting(Figure 2B).

Results

The hierarchical 3D deep learning architecture was evaluated on a holdout-set in two levels, namelly lesion detection(Figure 3), where AUC metrics are presented, and severity assessment(Figure 4) on a finer grained classification(3 classes for cartilage, BME and meniscus and 4 classes for ACL). Proposed system reports high lesion detection(2classes) sensitivity/specificity for cartilage(85.2/89.2%), meniscus(85.1/83.8%), BME(70.1/88.0%). For multiclass severity staging, sensitivity per class and confusion matrices are reported in Figure4.

Discussion and Conclusion

By automatically detecting and assessing the severity of the lesions in different anatomical structures of full knee, we are able to present a condensed clinical history of patients, presented in Figure 5. The baseline scan indicates the presence of a topical ACL injury as well as partial thickness cartilage lesions located in the lateral compartment. These are accompanied by severe and diffuse BME lesions in the lateral tibia and moderate BME lesions in the lateral femur. The 3 year follow up scan (post-reconstruction) indicates the presence of diffuse abnormalities in the medial side and in the patella femoral joint, suggesting the development of post traumatic OA. Meanwhile, the BME lesions (lateral tibia injury related) appear to have resolved.

Acknowledgements

This work was funded by GE Healthcare and the National Institutes of Health - National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIH-NIAMS) Grant number: R61AR073552 (SM/VP)