1137

Automatic Segmentation of Knee Cartilage and Menisci from MRI Data: Efficient Multiclass Solution Based on Deep Learning
Egor Panfilov1, Aleksei Tiulpin1, Miika Nieminen1,2, and Simo Saarakkala1,2

1Research Unit of Medical Imaging, Physics and Technology, University of Oulu, Oulu, Finland, 2Department of Diagnostic Radiology, Oulu University Hospital, Oulu, Finland

Synopsis

Manual segmentation of articular cartilage and menisci from magnetic resonance data is time-consuming and can be challenging. Recently, deep learning has shown promising results in medical image segmentation. The aim of this study was to develop a method for automatic segmentation of articular cartilage and menisci that performs more accurately and efficiently than the previously published methods. On OAI/iMorphics dataset the method achieves Dice score of 90.7±1.9 for femoral cartilage, 89.7±2.8 for tibial, 87.1±4.7 for patellar, and 86.3±3.4 for menisci. The presented results could facilitate the osteoarthritis research and enhance clinical practice. Source codes and pretrained models will be open-sourced.

Introduction

Osteoarthritis (OA) is the most common musculoskeletal disease in the world. Magnetic resonance imaging (MRI) has been actively used to assess the condition of the affected tissues and to study the onset and progression of OA. 3D double-echo steady-state (3D-DESS) MRI sequence provides a clear contrast between cartilage, bone, meniscus and synovial tissues, which is generally sufficient for assessment of tissue lesions by a radiologist. Automatic segmentation of articular cartilage and menisci is of high relevance for OA diagnostics and basic research, but it still remains a challenging task. Once solved, it can lead to a significant reduction of time required for the automatic analysis of the MRI data.

During the past years, deep learning (DL) has become a gold standard in medical image segmentation. However, despite multiple attempts, its value in the task of segmentation of cartilage and meniscal tissue has not been fully clarified. In this study, we introduce a fully-automatic method to segment the cartilage tissue and menisci from 3D-DESS MRI data, demonstrating the improvement over the previously published methods.

Methods

We utilized a subset of Osteoarthritis Initiative (OAI) dataset[1] and the corresponding annotations produced by iMorphics[2]. The data consisted of sagittal 3D-DESS MRI scans from 88 subjects (2 scans per subject: at baseline and at 12 months), acquired with Siemens 3.0T MRI scanners. Each scan contained 160 slices of 384x384 pixels (slice thickness = 0.7mm, field of view = 14x14 cm2). From these data, the following tissues were segmented: femoral cartilage, tibial cartilage (lateral and medial as one class), patellar cartilage, and menisci (lateral and medial as one class).

We randomly split the whole dataset subject-wise into train and test subsets maintaining the similar distribution of the Kellgren-Lawrence (KL) gradings (derived from the corresponding radiographs) between the subsets. Subsequently, we used 5-fold stratified cross-validation splits of the train data to develop the segmentation models. The test set was used solely for the final stage of our analysis.

Our segmentation approach is based on U-Net[3] with several modifications (24 channels in the first convolutional block output, doubled at each level, 6 levels total), performing cartilage tissue segmentation slice-wise. We trained the same model for each cross-validation fold for 50 epochs using multi-class cross-entropy loss function and Adam optimizer. The models were trained with a starting learning rate of 0.001, reduced by the factor of 10 at 30th epoch, weight decay of 5e-5, and batch size of 48. To improve the generalization of our solution, during the training we used a set of data augmentations: histogram percentile clipping (lower 5% and upper 1% of sample intensity), horizontal flips, gamma correction, artificial worsening (downscaling followed by upscaling), and bilateral filtering. For training the models, we used PyTorch DL framework and 2xNvidia GTX 1080 graphical processing units.

In the testing phase, we produced the predictions using all the 5 trained models and averaged their outputs. To assess the results, we used volumetric Dice score coefficient (DSC). In order to gain more insights regarding the performance of our approach, and assuming the data being registered, we calculated the statistics of DSC for each spatial slice across the test subset scans.

Results and Discussion

Volumetric DSC for each scan, and averaged coefficients over the full dataset (Table 1) and for every KL-grade score independently (Table 2) were calculated. We compared our model with other DL-based solutions on OAI/iMorphics dataset and presented the results in Table 1. The examples of the predictions and the corresponding errors are presented in Figure 1. Finally, the distributions of the DSC for each slice index are visualized in Figure 2. Our results indicate that the developed method performs more accurately with the current dataset than the previously published methods, while at the same time requiring less training data. Additionally, from Figure 2, it can be seen that the method performs with high DSC for the most anatomically relevant parts of the knee joint and performs poorly on the sides of the knee condyles, where the cartilage tissues and menisci are barely visible by a naked eye. Our source codes and the pretrained models will be publicly available.

Conclusions

We developed a new open-source DL-based method for automatic segmentation of knee articular cartilage and menisci from 3D-DESS MRI data. Our method is able to produce the segmentation masks for a single scan in a number of seconds, roughly 100 times faster than an average radiologist. We believe that DL-based approaches will allow to drastically speed up the analysis of MRI data and facilitate the development of OA treatment.

Acknowledgements

Support from Infotech Oulu is gratefully acknowledged.

References

  1. Peterfy, C. G., Schneider, E., & Nevitt, M. (2008). The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. Osteoarthritis and cartilage, 16(12), 1433-41.
  2. Data from 3D cartilage/meniscus segmentations of knee MRI scans. https://oai.epi-ucsf.org/datarelease/iMorphics.asp
  3. Ronneberger O., Fischer P., Brox T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N., Hornegger J., Wells W., Frangi A. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, vol 9351. Springer, Cham
  4. Chaudhari A., Fang Z., Lee J., Gold G., Hargreaves B. (2018) Open-sourced deep-learning for cartilage and meniscus segmentation. IWOAI 2018 Program Book, p. 26. https://github.com/akshaysc/msk_segmentation
  5. Norman, B. D., Pedoia, V., Link, T. M., & Majumdar, S. (2018). Artificial intelligence pipeline for meniscus segmentation and lesion detection. Osteoarthritis and Cartilage, 26(2018), S440–S441. https://doi.org/10.1016/j.joca.2018.02.844
  6. Gatti, A. A. (2018). NEURALSEG: state-of-the-art cartilage segmentation using deep learning – analyses of data from the osteoarthritis initiative. Osteoarthritis and Cartilage, 26(2018), S47–S48. https://doi.org/10.1016/j.joca.2018.02.110
  7. Tack, A., Mukhopadhyay, A., & Zachow, S. (2018). Knee menisci segmentation using convolutional neural networks: data from the Osteoarthritis Initiative. Osteoarthritis and Cartilage, 26(5), 680–688. https://doi.org/10.1016/j.joca.2018.02.907

Figures

Figure 1

Figure 2

Table 1

Table 2

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)
1137