Introduction

The amount of quantitative data that can be extracted from a single medical image has exponentially increased with the use of radiomic analysis[1]. Imaging biomarkers can aid in the generation of prediction models for disease diagnosis, characterization and prognosis[2, 3]. However, generalizability of the models is dependent on the robustness of these features. This study aims develop a task-specific robustness preselection step to improve the generalizability of radiomics model for prediction of sentinel lymph node status in breast cancer patients.

Methods

Phantom Study: A phantom study[4] (Fig. 1) was performed to divide radiomic features (histogram, gray level cooccurrence matrix (GLCM)[5], gray level run length matrix (GLRLM)[6], neighborhood gray level different matrix (NGLDM) [7], gray level zone length matrix (GLZLM)[8], Laws [9]) into robustness groups relative to voxel size differences. Robustness was assessed by intraclass correlation coefficient analysis (ICC values: low (<0.5), moderate (0.5-0.9), and high (>0.9)).

Sentinel Lymph Node Status Prediction: 212 patients with breast cancer undergoing primary surgery were analyzed in this portion of the study. From the standard of care dynamic contrast enhancement sequence, wash-in, wash-out and signal enhancement ratio maps were generated; in-breast tumor was manually segmented by a radiologist with 11 years of experience. Similar to the phantom study, radiomic features were calculated for each map. Additionally, clinical data was collected including location of tumor, multifocality, age, pathological type and grade, molecular subtype and lymphovascular invasion. These patients were separated into training set (n=109, 37 positive(+) for SLN metastasis), testing set (n=54, 18+) and an additional testing set (n=48, 13+). The training set and testing set had in-plane resolution of 0.70mm (referred to as Testing_0.70), whereas the additional testing set had in-plane resolution of 0.78-1.0mm (referred to as Testing_0.78-1.0). A predictive model was generated using radiomic and clinical data using least absolute shrinkage and selection operator (LASSO) with and without a step for removal of the low-moderately robust features to voxel size differences. Predictive performance was evaluated using receiver operating curve (ROC) analysis. Specifically, the metrics of most interest for this study include specificity, negative predictive value and accuracy. In order to assess stability and reproducibility of the model, the predictive model was run 100 times with different randomization seeds and the most commonly chosen features are reported.

Results

Phantom Study: Fig. 2 and Fig. 3 summarize the result of ICC analysis. All of GLCM, GLRLM, GLZLM features were found to be highly robust. Only one of first order and one of NGLDM features were found to be of less than high robustness. For Laws features, a majority were found to be of low and moderate robustness (27 out of 56 and 16 out of 56, respectively).

Sentinel Lymph Node Status Prediction: Prior to robustness preselection step, the specificity and accuracy of Testing_0.78-1.0 was poorer than Testing_0.70, (specificity: 86% vs 66%, accuracy: 82% vs 71%). However, the negative predictive value was higher for Testing_0.78-1.0 compared to Testing_0.70. Accuracy and specificity of Testing_0.78-1.0 improved after preselection based on robustness assessment. Results are summarized in Fig. 4 and Fig. 5. In Testing_0.78-1.0, specificity increased from 66% to 71%, and accuracy increased from 71% to 75%.

Discussion

Voxel size can vary based on different sizes of patients or across different institutions based on their imaging protocols. The effect of small voxel size variations, and its relation to radiomic feature robustness, is understudied. This study demonstrates that a model trained and tested on images of a particular voxel size performs worse in a test set comprised of different resolution. Furthermore, it indicates that removal of features deemed to be sensitive to voxel size variation improved performance.

Conclusion

Removal of radiomic features that are sensitive to voxel size variation results in a more generalizable and accurate predictive model for additional testing set with different voxel size compared to the training set. This finding could aid in the incorporation of radiomic tools for use in multi-institutional settings.

Acknowledgements

This work is in part funded by National Institutes of Health (R03CA223052), Walk-for-Beauty Foundation and Carol M. Baldwin Breast Cancer Research Foundation.