3797
Estimation of Time-to-Total Knee Replacement Surgery with Deep Learning using MRI and Clinical Data1Bernard and Irene Schwartz Center for Biomedical Imaging, Department of Radiology, New York University Grossman School of Medicine, New York, NY, United States, 2Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York, NY, United States, 3Center of Data Science, New York University, New York, NY, United States, 4Department of Radiology, New York University Grossman School of Medicine, New York, NY, United States
Synopsis
Keywords: Diagnosis/Prediction, Machine Learning/Artificial Intelligence, Osteoarthritis
Motivation: The combination of deep learning, MRI, and clinical data for predicting the time to total knee replacement surgery in knee osteoarthritis patients has been investigated.
Goal(s): The 3D Resnet18 model was employed to extract features from MRI scans, and relevant clinical variables were integrated to establish a comprehensive predictive model.
Approach: Time-to-surgery probabilities were estimated using the ensemble random survival forest model. The model’s performance was evaluated across clinical variables, two MRI sequences, and their combinations.
Results: The proposed approach aims to help the precision of TKR surgery decision-making using artificial intelligence.
Impact: This study fuses deep learning, survival analysis, MRI, and clinical data to accurately predict time-to-TKR surgery. Our approach has the potential to enhance TKR surgery decision precision.
Introduction
Knee osteoarthritis (KOA) is a prevalent and disabling chronic condition, affecting approximately 10% of men and 13% of women aged 60 and above. At advanced disease stages, total knee replacement (TKR) becomes a viable intervention for KOA. Recent studies using deep learning (DL) models have highlighted MRI’s capability in detection of knee osteoarthritis progression [1 ]. In [ 2 ], various machine learning algorithms, including random survival forest (RSF), were assessed for time-to-TKR prediction using only the ten most informative clinical variables. In [ 3], the DenseNet169 model was employed to predict KOA progression, defined with both radiographic and pain progression within 24–48months. The model employed a combination of three different 3D knee MRI sequences collected at baseline, 12, and 24 months for binary classification into progression and non-progression. The [3] concluded that as the disease progressed over time, the AUC of the classification increased, indicating improved disease progression detection by the model. This study aims to estimate the time-to-TKR surgery within a 9-year timeframe by employing a novel approach that combines DL, survival analysis, MRI, and clinical data. Using 3D Resnet18 models, the features were extracted from two separate MRI scans and they were concatenated with clinical variables to predict the outcome. Subsequently, an RSF model was applied to the concatenated features to estimate the probabilities of time-to-TKR surgery within the 9-year prediction horizon [4, 2]. The performance of the proposed model was evaluated with different input data combinations and it demonstrated the potential for more precise TKR surgery decisions.Materials and Methods
The study used OAI dataset [5] for training and in-distribution testing, and the MOST dataset [6] for external testing of the developed model. The details of study cohort are presented in Table 1. The models were assessed using an internal hold-out testing set from the OAI database, which was not included in the model’s training and validation. Furthermore, the model trained with SAG IW TSE data from OAI was also tested on an external group of TKR-positive patients using SAG PD FS MRI data from the MOST database. Two separate 3D Resnet18 models were trained using SAG IW TSE and SAG DESS MR images. For each MRI data, the labels representing time-to-TKR surgery were mapped to a normal distribution (discretized into 30 bins) with a variance of 4 and a mean equal to the label [ 7 ]. The model was trained using the KL-divergence loss, which ensures that the output of the model has a distribution similar to the normal distribution of the labels. Predictions were calculated based on the area under the predicted distribution. The best model was selected based on the highest validation accuracy [8]. Features from MRI sequences were extracted using the best models and they were concatenated with clinical variables. An RSF model developed to predict survival probabilities over a 9-year period, which were utilized to estimate the time-to-TKR surgery. The model performance was evaluated using clinical variables, SAG IW TSE, SAG DESS MRI, and their combinations. The flowchart of the model is provided in Figure 1.Results
The combination of features from both SAG IW TSE and SAG DESS MRI sequences, along with clinical variables, yielded the best predictions for time-to-TKR surgery when compared to the models utilizing features from a single input stream. The confusion matrices for exact year and ±1 year predictions are presented in Figure 3. The results of the study is given in Table 2. Using the SAG DESS MRI sequence resulted in better predictive performance compared to SAG IW TSE, likely due to its higher spatial resolution, allowing for more detailed feature extraction in the slice dimension. The model using combination of features from two MRI sequences and clinical variables was tested with an external testing group from MOST database. The prediction performance was decreased. This could be considered a “worse-case scenario” external testing group with MRI sequence acquired on 1.0T extremity scanners with lower image quality than MRI protocols performed on 3.0T scanners within OAI study.Conclusions
Accurately predicting the time-to-TKR surgery is a challenging task due to various factors associated with the surgery decision. The use of a random survival forest model, incorporating clinical variables and MRI sequence features extracted from 3D Resnet18 models, represented promise in estimating the time-to-TKR surgery. The proposed model in this study has the potential to help TKR surgery decisions and make contributions to the field of OA research. Further enhancements can be explored by considering additional imaging modalities, implementing self-supervised pre-training methods for feature extraction, accommodating censored data, and optimizing predictive models.Acknowledgements
This work was supported in part by the NIH R01 AR074453, and was performed under the rubric of the Center for Advanced Imaging Innovation and Research (CAI2R, www.cai2r.net), an NIBIB National Center for Biomedical Imaging and Bioengineering (NIH P41 EB017183).References
[1] H. R. Rajamohan, T. Wang, K. Leung, G. Chang, K. Cho, R. Kijowski, C. M. Deniz, Prediction of total knee replacement using deep learning analysis of knee MRI, Scientific Reports 13 (1) (2023) 6922.URL https://doi.org/10.1038/s41598
[2] A. Jamshidi, J.-P. Pelletier, A. Labbe, F. Abram, J. Martel-Pelletier, A. Droit, Machine learning-based individualized survival prediction model for total knee replacement in osteoarthritis: Data from the osteoarthritis initiative, Arthritis Care Res. (Hoboken) 73 (10) (2021) 1518–1527.
[3] J. Hu, C. Zheng, Q. Yu, L. Zhong, K. Yu, Y. Chen, Z. Wang, B. Zhang, Q. Dou, X. Zhang,DeepKOA: a deep-learning model for predicting progression in knee osteoarthritis using multimodal magnetic resonance images from the osteoarthritis initiative, Quant. Imaging Med. Surg. 13 (8) (2023)4852–4866
[4] K. Suresh, C. Severn, D. Ghosh, Survival prediction models: an introduction to discrete-time modeling,BMC Med. Res. Methodol. 22 (1) (2022) 207.
[5] G. Lester, Clinical research in OA–the NIH osteoarthritis initiative, J. Musculoskelet. Neuronal Interact. 8 (4) (2008) 313–314.
[6] Multicenter osteoarthritis study (most) public data sharing.URL https://most.ucsf.edu/multicenter-osteoarthritis-study-most-public-data-sharing
[7] Y. Yang, G. Xutao, C. Ye, Y. Xiang, T. Ma, Regularizing brain age prediction via gated knowledge distillation, in: Medical Imaging with Deep Learning, 2022.URL https://openreview.net/forum?id=nxA2bZff3iQ
[8] E. Hedayati, H. Rajamohand, L. Zhou, K. Cho, G. Chang, R. Kijowski, C. M. Deniz, Estimating time-to-total knee replacement surgery using deep learning, in: Proceedings of the 2023 ISMRM &ISMRT Annual Meeting & Exhibition, Toronto, Canada, 2023.6
Figures
Figure 1: The 25 clinical variables available for more than 90% of subjects and with highest variance are selected from the 1224 baseline ones. The tuning of the RSF model was performed by minimizing the integrated Brier score [ 4]. Ten threshold value ranging from 0.1 to 1 is employed to determine the time-to-TKR surgery. When the survival probability falls below the threshold, the predicted time-to-TKR surgery corresponds to the latest year in which it surpasses the threshold. The threshold value was chosen based on the highest accuracy achieved with the validation data.
