2653

Compartment-specific Knee Cartilage Segmentation using Deep Learning
Egor Panfilov1, Aleksei Tiulpin1,2, Victor Casula1, Simo Saarakkala1,2, and Miika T. Nieminen1,2
1Research Unit of Medical Imaging, Physics and Technology, University of Oulu, Oulu, Finland, 2Department of Diagnostic Radiology, Oulu University Hospital, Oulu, Finland

Synopsis

In this study, we developed a method for compartment-specific segmentation of knee cartilage from 3D-DESS MR images which jointly utilizes deep learning and atlas-based approaches. The method was applied to compare the performance of two deep learning-based segmentation models on two independent datasets. One of the models achieved new state-of-the-art in knee cartilage segmentation on the Osteoarthritis Initiative data and was more robust to the changes in MRI protocol. Detailed analysis performed using our method showed how the performance improvements are localized compartment-wise. The method can be used to select the most accurate segmentation model for the considered clinical problem.

Introduction

Automatic methods based on deep learning (DL) have recently achieved promising results in knee cartilage segmentation from MRI data 1,2,3. Such methods, however, are sensitive to inhomogeneities in input data, e.g., caused by changing the MRI protocol 4. These failures may remain unnoticed, since the DL-based segmentation can introduce bias within tissue compartments across the datasets, while knee MRI segmentation methods are typically assessed tissue-wise. Additionally, measured at tissue-level, segmentation metrics do not fully reflect the morphological changes of articular cartilage. Accordingly, such evaluation may prioritize the method with lower accuracy in the regions of interest, but higher in the ones that are trivial to segment. Therefore, assessment and comparison of automatic segmentation methods have to be done compartment-wise, in a clinically meaningful manner.
In this study, we propose a multi-stage method for compartment-specific segmentation and analysis of knee cartilage tissues. The first stage of the pipeline employs a fast and accurate DL-based segmentation model. The second stage uses an atlas-based method for automatic compartmentalization of the segmented tissues.

Methods

Our multi-stage method performs segmentation of the cartilage tissues and their subsequent compartmentalization (Figure 1). For training and evaluation of the segmentation models, we used a subset of data from the Osteoarthritis Initiative (OAI) 5. The dataset consisted of 176 3D-DESS MRI scans (3T Siemens MAGNETOM Trio, quadrature transmit-receive knee coil, voxel 0.37x0.37x0.7mm3, matrix 384x384, FOV 140mm, 160 slices, TR 16.3ms, TE 4.7ms, flip angle 25°) with semi-automatic annotations by iMorphics 6. The data was split patient-wise into training and test subsets, 140 and 36 scans respectively, stratified by radiographic OA severity (Kellgren-Lawrence scale). For evaluation, we additionally employed the OKOA dataset 7, which contains 44 3D-DESS scans and mainly differs in acquisition hardware and spatial resolution (3T Siemens MAGNETOM Skyra, 15-channel transmit-receive knee coil, voxel 0.59x0.59x0.6mm3, matrix 256x256, FOV 150mm, 160 slices, TR 14.1ms, TE 5ms, flip angle 25°). Annotations for femoral and tibial cartilage tissues were produced by our research group. Two DL-based methods were developed using a training subset of OAI to segment femoral, tibial, patellar cartilages, and menisci. Each method consisted of an ensemble of models trained in 5-fold cross-validation setting. As a first method (A), we used the previous state-of-the-art in the domain 3. There, a modified U-Net architecture with 6 depth levels and 24 base filters was trained from scratch. The second method (B) was based on transfer learning and used VGG19 8 backbone pre-trained on ImageNet as an encoder, otherwise being similar to A. To improve the method robustness, we used mixup regularization 9.
The second stage of our approach performed compartmentalization of the tissues. We selected ten scans from the training subset of OAI to construct a multi-atlas, maintaining similar distribution of Body Mass Index as in the whole subset. The annotation of multi-atlas was done in accordance with MRI Osteoarthritis Knee Score (MOAKS) guidelines 10.
During evaluation, we segmented the scans from the test subset of OAI and whole OKOA using both methods – A and B. Next, we used rigid registration of the multi-atlas to each scan to estimate the similarity transforms. These transforms were used to warp the masks from multi-atlas to the reference and the automatic segmentations. The segmentation masks for each cartilage tissue were then divided into compartments voxel-wise based on the Euclidean proximity to the valid compartments from the atlas. Finally, we used majority voting to fuse the compartment-level maps across the multi-atlas. To quantify the segmentation accuracy, we used volumetric Dice score coefficient (DSC) computed tissue- and compartment-wise.
For training the models, we used PyTorch framework and NVIDIA RTX 2080ti graphics card. For registration, we employed elastix 11. For compartmentalization we used SciPy 12.

Results

Tissue-wise volumetric DSCs showed that method B significantly outperformed A on both datasets (Table 1 and 2, Figure 2a). On OKOA both methods yielded lower relative segmentation accuracy due to the differences in MRI data acquisition protocols 3, yet method B showed better robustness (Figure 3a).
We visually inspected the compartmentalization results and confirmed that they were consistent and in agreement with the multi-atlas.
Compartment-wise, both methods performed similarly on OAI (Table 1, Figure 2b), yet method B significantly improved (p<0.05) the DSCs for 4 compartments. On OKOA, method B achieved significantly higher (p<0.001) DSCs in most of the compartments (Table 2, Figure 3b). Method A was significantly more accurate in the posterior areas of femoral cartilage.

Discussion and Conclusions

In this work we proposed a new multi-stage method for compartment-specific segmentation of knee cartilage tissues from MRI data. Using this method, we were able to perform fine-grained comparison of tissue-level segmentation models and showed how such analysis can be used to select the most accurate model for the considered problem.
Our segmentation approach showed performance superior to the previous state-of-the-art at both tissue and compartment levels. Notably, the improvements were observed in femur load-bearing areas on OAI and in most of the compartments on OKOA.
Even though the study considered only knee MRIs, it can be further extended to other domains. Particularly, the problems, where a trade-off between segmentation accuracies in multiple anatomical regions has to be made, would directly benefit from our approach.

Acknowledgements

The OAI is a public-private partnership comprised of five contracts (N01-AR-2-2258; N01-AR-2-2259; N01-AR-2- 2260; N01-AR-2-2261; N01-AR-2-2262) funded by the National Institutes of Health, a branch of the Department of Health and Human Services, and conducted by the OAI Study Investigators. Private funding partners include Merck Research Laboratories; Novartis Pharmaceuticals Corpora- tion, GlaxoSmithKline; and Pfizer, Inc. Private sector fund- ing for the OAI is managed by the Foundation for the Na- tional Institutes of Health. This manuscript was prepared using an OAI public use data set and does not necessarily reflect the opinions or views of the OAI investigators, the NIH, or the private funding partners.

The authors would like to acknowledge the following funding sources: strategic funding of University of Oulu (Infotech Oulu), Sigrid Juselius foundation, and KAUTE foundation, Finland.

References

  1. Tack, A., Mukhopadhyay, A., & Zachow, S. (2018). Knee menisci segmentation using convolutional neural networks: data from the osteoarthritis initiative. Osteoarthritis and cartilage, 26(5), 680-688.
  2. Tack, A., & Zachow, S. (2019). Accurate Automated Volumetry of Cartilage of the Knee using Convolutional Neural Networks: Data from the Osteoarthritis Initiative. In IEEE 16th International Symposium on Biomedical Imaging, 40-43.
  3. Panfilov, E., Tiulpin, A., Klein, S., Nieminen, M. T., & Saarakkala, S. (2019). Improving Robustness of Deep Learning Based Knee MRI Segmentation: Mixup and Adversarial Domain Adaptation. In Proceedings of the IEEE International Conference on Computer Vision Workshops (pp. 0-0).
  4. Karani, N., Chaitanya, K., Baumgartner, C., & Konukoglu, E. (2018, September). A lifelong learning approach to brain mr segmentation across scanners and protocols. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 476-484). Springer, Cham.
  5. Peterfy, C. G., Schneider, E., & Nevitt, M. (2008). The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. Osteoarthritis and cartilage, 16(12), 1433-41.
  6. Data from 3D cartilage/meniscus segmentations of knee MRI scans. https://oai.epi-ucsf.org/datarelease/iMorphics.asp
  7. Podlipská, J., Guermazi, A., Lehenkari, P., Niinimäki, J., Roemer, F. W., Arokoski, J. P., … Saarakkala, S. (2016). Comparison of Diagnostic Performance of Semi-Quantitative Knee Ultrasound and Knee Radiography with MRI: Oulu Knee Osteoarthritis Study. Scientific reports, 6, 22365. doi:10.1038/srep22365
  8. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
  9. Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412.
  10. Hunter, D. J., Guermazi, A., Lo, G. H., Grainger, A. J., Conaghan, P. G., Boudreau, R. M., & Roemer, F. W. (2011). Evolution of semi-quantitative whole joint assessment of knee OA: MOAKS (MRI Osteoarthritis Knee Score). Osteoarthritis and cartilage, 19(8), 990–1002. doi:10.1016/j.joca.2011.05.004
  11. Klein, S., Staring, M., Murphy, K., Viergever, M. A., & Pluim, J. P. (2009). Elastix: a toolbox for intensity-based medical image registration. IEEE transactions on medical imaging, 29(1), 196-205.
  12. Jones, E., Oliphant, T., & Peterson, P. (2001). SciPy: Open source scientific tools for Python.

Figures

Figure 1

Figure 2

Figure 3

Table 1

Table 2

Proc. Intl. Soc. Mag. Reson. Med. 28 (2020)
2653