1894
Estimating time-to-total knee replacement surgery using deep learning
Eisa Hedayati1, Haresh Rajamohan2, Lily Zhou3, Kyunghyun Cho2, Gregory Chang1, Richard Kijowski1, and Cem M Deniz1
1Radiology, New York University Langone health, New York, NY, United States, 2Center for data Science, New York University, New York, NY, United States, 3Radiology & diagnostic Imaging, University of Alberta, Edmonton, AB, Canada
1Radiology, New York University Langone health, New York, NY, United States, 2Center for data Science, New York University, New York, NY, United States, 3Radiology & diagnostic Imaging, University of Alberta, Edmonton, AB, Canada
Synopsis
Keywords: Machine Learning/Artificial Intelligence, Osteoarthritis
"When do I need my knee replaced? " is a common question of patients with progressed osteoarthritis conditions. However, answering that question is not trivial, especially for cases that do not result in an immediate surgery. To help doctors addressing this question we employed neural networks to recommend an estimated subject-specific date for the total knee replacement (TKR).Introduction
To date, there has been little work on estimating when TKR surgery is needed for individuals with Knee Osteoarthritis (KOA) condition. It is a challenging task to accurately identify the time-to-TKR surgery date due to multiple factors associated with surgery decision. . Heisinger et al [1] built a model based on 14 factors monitored over a 4 year period of individuals with KOA to predict the need for TKR surgery they were able to predict whether the patient will undergo the surgery in the immediate year after the visit with 84% positive rate and negative predictive value of 73%. However, they were not addressing any specific future date for the TKR surgery. Prediction of total knee replacement as a binary outcome variable has been studied using MR images and radiographs [2][3]. Focusing on the time-to-TKR predictions, Afshin Jamshidi [4] used several machine learning methods (cox, DeepSurv, etc.) to find the factors with most influence in time-to-TKR prediction by applying feature selection resulted in a knee specific survival analysis with concordance index of 0.85.In this work, we propose to estimate the time-to-TKR by using the sagittal turbo spin echo (TSE) MR images and training a deep learning model over the Osteoarthritis Initiative (OAI) dataset [5].
Methods
A subset of OAI dataset that includes sagittal TSE MR images was compiled to extract individuals who had their knee replaced due to progressed KOA. Multiple MRIs from a single patient at different dates were included in the dataset. Total of 416 unique individuals who had TKR during the study period were investigated. The training, validation and test sets have 292, 41, 83 individuals respectively. Each patient might have KOA for both or single legs. Table 1 represents the study cohorts used for DL model development and testing.The 3D-Resnet34 [6] was used as a backbone neural network architecture and the classifier was repurposed from [7]. The classifier on top of 3D_Resnet34 consists of one 3D transposed convolution layer followed by two 3D convolution layers and a pooling layer. ReLU was used as an activation function. The details of the deep learning architecture is depicted in Fig 1.
For each MRI, the label (remaining days to the surgery) is first converted to month, then mapped to a normal distribution (discretized to 128 bins) with variance of 4 and mean equal to the number of remaining months to the TKR. We trained our model using KL-Divergence loss that ensures the output of the model (128 classes) has a similar distribution with the enforced normal distribution of the label. The prediction was calculating by the area under the predicted distribution: $$date=\sum_{i=0}^{127} \hat{p} i.$$
Results
Table 2 shows the accuracy of the proposed model when different discretizations of time are used. Our preliminary results on our test set can predict a time-to-TKR date with 37.2% one year , 66.1% two years and 84.6% three years accuracy. In a survival analysis, the test set’s concordance index is 0.55. The best results were achieved in the range 2 to 5 years to TKR. This behavior was expected because the dataset has more participants in a range of less than 5 years. We can see the model can predict when the TKR should be done within 1 year of the actual TKR date for 51.7% of the given MRIs. More detailed results can be found in table 2.Conclusion
We used deep learning models for predicting the time-to-TKR surgery date using TSE MRIs of patients. Even using a small dataset, our preliminary results suggested that there can be a measurable relation between the current state of the knee as seen on the MRI to the final decision of doctors/patient on when the TKR procedure should be done. Further improvements can be investigated by looking at different MRI contrasts and employing self-supervised pre-training approaches. Moreover, incorporating clinical measurements during the development of the model could improve time-to-TKR predictions.Acknowledgements
This work was supported in part by NIH grant R01 AR074453, and was performed under the rubric of the Center for Advanced Imaging Innovation and Research (CAI2R, www.cai2r.net), a NIBIB Biomedical Technology Resource Center (NIH P41 EB017183).References
[1] Heisinger, Stephan, et al. "Predicting total knee replacement from symptomology and radiographic structural change using artificial neural networks—data from the osteoarthritis initiative (OAI)." Journal of clinical medicine 9.5 (2020): 1298.
[2] Leung, Kevin, et al. "Prediction of total knee replacement and diagnosis of osteoarthritis by using deep learning on knee radiographs: data from the osteoarthritis initiative." Radiology 296.3 (2020): 584.
[3] Tolpadi, Aniket A., et al. "Deep learning predicts total knee replacement from magnetic resonance images." Scientific reports 10.1 (2020): 1-12.
[4] Jamshidi, Afshin, et al. "Machine Learning–Based Individualized Survival Prediction Model for Total Knee Replacement in Osteoarthritis: Data From the Osteoarthritis Initiative." Arthritis Care & Research 73.10 (2021): 1518-1527.
[5] Peterfy, Charles G., Erika Schneider, and M. Nevitt. "The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee." Osteoarthritis and cartilage 16.12 (2008): 1433-1441.
[6] Hara, Kensho, Hirokatsu Kataoka, and Yutaka Satoh. "Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?." Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2018.
[7] Yang, Yanwu, et al. "Regularizing Brain Age Prediction via Gated Knowledge Distillation." Medical Imaging with Deep Learning. 2021.
Figures
Fig 1: The model. The classifier takes the 3D-Resnet34 output of the last layer before its final pooling layer as an input. The output proposed model is a predicted discrete distribution.
Fig 2: Time-to-TKR date predictions for a given MRI scan in the test set. The blue lines are isolating the range of 2-5 years where the proposed model performs the best. For the MRIs with time-to-TKR date of more than 5 years, the number of MRIs is limited which could be one of the reasons for decrease in model performance.
Table 1: The study cohort: Sagittal TSE MRIs of the patients in the OAI dataset who had their knees replaced during the study period.
Table 2: The model performance results on the test set. The test set is separated in different intervals and the accuracy and concordance index (c-index) is evaluated for each interval.
DOI: https://doi.org/10.58530/2023/1894