3102
External Validation of IDH Mutation Detection Using Radiomics Extracted from T2-weighted MRI with Machine Learning1Institute of Biomedical Engineering, Boğaziçi University, Üsküdar, Turkey, 2Department of Medical Pathology, Acibadem University, İstanbul, Turkey, 3Brain Tumor Research Group, Acibadem University, İstanbul, Turkey, 4Department of Neurosurgery, Acibadem University, İstanbul, Turkey, 5Department of Radiology, Acibadem University, İstanbul, Turkey
Synopsis
Keywords: Analysis/Processing, Radiomics, glioma, machine learning, generalizability
Motivation: Isocitrate dehydrogenase (IDH) mutation plays a key role in the prognosis of gliomas. Several studies have detected the IDH mutation using radiomics. However, few studies focused on the generalizability of radiomics-based machine learning models.
Goal(s): To externally validate the ability of radiomics for noninvasive detection of IDH mutation using multi-site data.
Approach: Radiomics of T2w MRI of UCSF-PDGM dataset (Cohort 1) was used for training machine learning models, then externally validated at the local dataset (Cohort 2).
Results: T2w MRI-radiomics could identify the IDH mutation with an accuracy of 0.89 on Cohort 1, which was externally validated with an accuracy of 0.73.
Impact: External validation studies are important for investigating the generalizability of machine learning models. The models based on T2w MRI-radiomics resulted in 0.89 accuracy in the training dataset, with a slightly lower accuracy on external validation dataset for identifying IDH mutation.
Introduction
Gliomas are infiltrative primary intracranial tumors accounting for about 24.5% of all primary brain and other central nervous system (CNS) tumors, and 80.9% of malignant tumors.1 Molecular biomarkers have been integrated into the classification of gliomas. Isocitrate dehydrogenase (IDH) mutation restricts the glioma subtypes into two groups: i) IDH-mutant (IDH-mut) “astrocytomas and oligodendrogliomas” and ii) IDH-wildtype (IDH-wt) “glioblastomas”.2 It has been reported that IDH mutation is a significant prognostic factor for glioma patients and has been associated with better treatment response and overall survival.3 Although IDH mutation is detected by immunochemistry and genomic sequence analyses, intra-tumoral heterogeneity of gliomas might lead to incorrect results.4,5 Radiomics extracts a large number of quantitative and mineable features from standard imaging modalities.6 Many publications demonstrated that radiomics have the ability to detect the IDH mutation status of gliomas from various MRI modalities.7,8 However, it is of utmost importance to externally validate the radiomics-based machine learning models to show the reliability of the predictive power. The aim of this study is to evaluate and externally validate the predictive power of machine learning models based on radiomics extracted from T2-weighted (T2w) MRI for noninvasive identification of IDH mutation status in gliomas.Methods
In this study, we used two independent patient cohorts. The first cohort (Cohort 1), used for model development, consisted of 425 gliomas (IDH-mut/IDH-wt: 90/335, f/m: 168/257, (mean±sd) age: 57.28±14.89 years) from The University of California San Francisco Preoperative Diffuse Glioma MRI dataset.9 The second cohort (Cohort 2), used for external validation, was local data of 213 gliomas (IDH-mut/IDH-wt: 106/107, f/m: 90/123, (mean±sd) age: 45.52±14.46 years) scanned at a clinical 3T MRI scanner (Siemens Healthcare, Erlangen, Germany). The tumor segmentation was manually conducted using 3D Slicer.10To minimize the inhomogeneity between the two cohorts, MR images were first preprocessed as shown in Figure 1-A. Then, feature extraction was performed using PyRadiomics yielding 1820 features, consisting of intensity, shape, and texture radiomics features.11 The overall machine-learning workflow is given in Figure 1-B. The most relevant features for the machine learning classifiers (kNN, logistic regression, SVM, linear discriminant analysis, and decision trees) were chosen through LASSO and grid search with 5-fold cross-validation was applied for selecting the best hyperparameters. In the final step, we trained the machine learning models with the complete data of Cohort 1 and tested on the independent external validation dataset (Cohort 2). The differences between IDH-mut and IDH-wt gliomas in selected radiomics features were assessed by a Mann-Whitney U test with Bonferroni correction (P<0.0025).Results
The mean values of the 20 best-performing features in IDH-wt and IDH-mut gliomas of Cohort 1 are provided in Table 1. The same features were used for identifying the IDH-mutational subgroups in Cohort 2, and their mean values are given in Table 2. In Cohort 1, 13 of the selected features significantly differed between IDH-wt and IDH-mut gliomas. In Cohort 2, five of the selected radiomic features were found significantly different. Except for interquartile range with LBP filtering, four radiomic features were significantly different for IDH-wt and IDH-mut patients in both cohorts. The best performance for identifying IDH mutational status of Cohort 1 was obtained with a linear discriminant analysis classifier that resulted in an accuracy of 0.89 (precision±std= 0.72±0.11, recall±std= 0.80±0.16, and area under the ROC curve (ROC AUC±std)= 0.91±0.06) (Table 3). On Cohort 2, IDH mutation was detected best with the logistic regression classifier, and the accuracy, precision, recall, and ROC AUC were 0.73, 0.89, 0.67, and 0.75, respectively (Table 4).Discussion
The generalizability of predictive models on different datasets depends on various factors such as differences in image acquisition, tumor segmentation, and the preprocessing steps of radiomic feature extraction.12 In this study, we harmonized two independent datasets and evaluated the generalizability of traditional classifiers. Model performance was reduced by the site change. Still, relevant features showed some agreement. Four radiomics features significantly differed in both cohorts. We noticed lower sphericity in IDH-wt tumors, which suggests increased shape irregularity compared to IDH-mut tumors. Radiomic texture features have the potential to quantify intra-tumoral heterogeneity that is invisible to the human eye.13 In our study, IDH-wt tumors had significantly higher grey level dependence matrix (GLDM) dependence variance and grey level co-occurrence matrix (GLCM) cluster shade, which are related to prominent intra-tumoral heterogeneity.Conclusion
This work utilized multicenter data for developing radiomics-based classifiers to predict the IDH mutational status of gliomas. Although the classification accuracy was slightly lower for the external validation dataset, MRI-based radiomics features have demonstrated potential for noninvasively identifying IDH mutational status in gliomas.Acknowledgements
This study was supported by the Scientific and Technological Research Council of Turkey (TUBITAK) 1003 grant 216S432.References
1. Ostrom QT, Cioffi G, Waite K, Kruchko C, Barnholtz-Sloan JS. CBTRUS statistical report: primary brain and other central nervous system tumors diagnosed in the United States in 2014–2018. Neuro-Oncol. 2021;23(Supplement_3):iii1-iii105.
2. Louis DN, Perry A, Wesseling P, et al. The 2021 WHO classification of tumors of the central nervous system: a summary. Neuro-Oncol. 2021;23(8):1231-1251.
3. Yan H, Parsons DW, Jin G, et al. IDH1 and IDH2 mutations in gliomas. N Engl J Med. 2009;360(8):765-773.
4. Çelik S, Öven BB, Demir MK, et al. Magnetic resonance imaging criteria for prediction of isocitrate dehydrogenase (IDH) mutation status in patients with grade II-III astrocytoma and oligodendroglioma. Clin Neurol Neurosurg. 2021;207:106745. doi:10.1016/j.clineuro.2021.106745
5. Perizzolo M, Winkfein B, Hui S, Krulicki W, Chan JA, Demetrick DJ. IDH mutation detection in formalin-fixed paraffin-embedded gliomas using multiplex PCR and single-base extension. Brain Pathol Zurich Switz. 2012;22(5):619-624. doi:10.1111/j.1750-3639.2012.00579.x
6. Lambin P, Rios-Velazquez E, Leijenaar R, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48(4):441-446.
7. Han L, Wang S, Miao Y, et al. MRI texture analysis based on 3D tumor measurement reflects the IDH1 mutations in gliomas – A preliminary study. Eur J Radiol. 2019;112:169-179. doi:10.1016/j.ejrad.2019.01.025
8. Huang W yuan, Wen L hua, Wu G, et al. Comparison of Radiomics Analyses Based on Different Magnetic Resonance Imaging Sequences in Grading and Molecular Genomic Typing of Glioma. J Comput Assist Tomogr. 2021;45(1):110-120. doi:10.1097/RCT.0000000000001114
9. Calabrese E, Villanueva-Meyer J, Rudie J, et al. The University of California San Francisco Preoperative Diffuse Glioma MRI (UCSF-PDGM). Published online April 7, 2023. doi:10.7937/TCIA.BDGF-8V37
10. Fedorov A, Beichel R, Kalpathy-Cramer J, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging. 2012;30(9):1323-1341. doi:10.1016/j.mri.2012.05.001
11. Van Griethuysen JJM, Fedorov A, Parmar C, et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017;77(21):e104-e107. doi:10.1158/0008-5472.CAN-17-0339
12. Jha AK, Sherkhane UB, Mthun S, et al. External Validation of Robust Radiomic Signature to Predict 2-Year Overall Survival in Non-Small-Cell Lung Cancer. J Digit Imaging. Published online September 21, 2023. doi:10.1007/s10278-023-00835-8
13. Molina D, Pérez-Beteta J, Luque B, et al. Tumour heterogeneity in glioblastoma assessed by MRI texture analysis: a potential marker of survival. Br J Radiol. 2016;89(1064):20160242. doi:10.1259/bjr.20160242
Figures
