1909
A Deep Transfer Learning Model to Predict Patient Outcome in ICH using the Fusion of Clinical and Fluid-Attenuated Inversion Recovery Imaging Data1Radiology, University of Cincinnati, Cincinnati, OH, United States, 2The Perinatal Institute and Section of Neonatology, Perinatal and Pulmonary Biology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, United States, 3Electrical Engineering and Computer Science, University of Cincinnati, Cincinnati, OH, United States, 4Pediatrics, University of Cincinnati, Cincinnati, OH, United States, 5Neurology and Rehabilitation Medicine, University of Cincinnati, Cincinnati, OH, United States
Synopsis
Timely and reliable prognostic tools for intracerebral hemorrhage (ICH) have great potential to guide physician decision making. They are potentially useful for targeting patients for interventions and optimizing rehabilitation strategies. The objective of this study is to investigate if a deep transfer learning model can capture individual variability to predict clinical outcome for ICH patients at 3 months using the integration of clinical and T2-weighted fluid-attenuated inversion recovery (FLAIR) imaging data. Our model was able to correctly identify patients likely to have unfavorable outcomes with an AUC of 0.87 (95% confidence interval: 0.86, 0.89).
Introduction
Intracerebral hemorrhage (ICH) is a life-threatening type of stroke – with an incidence rate of 24.6 per 100,000 person-years .1 The fatality rate is about 40% and only 12-39% of patients regain independence after an ICH.2 Reliable prognostic tools can help optimize rehabilitation strategies and guide the decision making of physicians including discussion with patients and families; there is a critical need for improving currently used traditional ICH scores. Recent advances in deep learning have demonstrated that such techniques are well-suited for extracting meaningful pathological features and revealing discriminative information from MRI data. 3, 4 Deep learning has achieved breakthroughs in applications with large sample sizes, such as image recognition and speech recognition. However, when facing high-dimensional, low-sample-size annotated neuroimaging datasets with clinical and outcome information, deep learning suffers from insufficient training data. 3 Transfer learning represents an important key to solve the fundamental problem of insufficient training data in deep learning. 5 The aim of this study is to investigate if a deep transfer learning model can accurately predict clinical outcome in patients with ICH at 3 months using the integration of clinical and T2-weighted fluid-attenuated inversion recovery (FLAIR) imaging data.Method
This study included a convenience sample of 53 left thalamocapsular patients with ICH (hemorrhagic volume < 20cc; mean age = 52.4 years) from the Ethnic/Racial Variation in Intracerebral Hemorrhage (ERICH) study. T2-weighted FLAIR data were acquired using clinical protocols in this NIH funded multicenter cohort. A total of 135 clinical features (including demographics, vascular risk factors) and 3-month modified ranking scale (mRS) scores were obtained using the ERICH study data. We considered mRS scores of 0-2 as favorable outcomes, and mRS scores of 3-6 as unfavorable outcomes. We conducted the following schema for FLAIR data preprocessing: 1) skull stripping (comprises the process of removing skull, extra-meningeal and non-brain tissues from the MRI data); 6 2) bias field correction (reducing the signal intensity inhomogeneity mainly caused by radiofrequency coils); 7 and 3) intensity normalization (reducing the variations of signal intensity and contrast across subjects). 8 Figure 1 shows an overview of the proposed deep transfer learning model using both clinical and T2-weighted FLAIR data to predict the recovery outcomes in patients with ICH. More specifically, the proposed model included two separate input channels, one for imaging and the other for clinical data. To extract high-level discriminative imaging features in imaging channel, we designed a module with 24 layers by reusing the weights of a pre-trained VGG-19 model (1st to 21st layers), 9 and then training the weights of two additional convolutional layers with [64, 128] neurons and 3 x 3 filters and one fully-connected layer with 64 neurons. For each patient, the model analyzed n = 8 slices containing the entire hemorrhagic lesion. For the clinical channel, one fully-connected layers with 64 neurons was applied to learn the discriminative features from the clinical data. Finally, a fully connected fusion layer with 64 neurons was applied to integrate the extracted discriminative information from both imaging and clinical data. A two-way softmax classifier was then utilized to identify the patients likely to have unfavorable outcomes. Rotation and shift-based data augmentation strategy 10 was implemented to increase the training samples by 10 times (but not testing samples). Performance was evaluated using 5-fold cross-validation with the metrics of accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC).Result
We tested the performance of the proposed model using only clinical and imaging data alone; and then using combined clinical and imaging. As shown in Table 1, our model was able to correctly identify patients likely to have unfavorable outcomes with an AUC of 0.87 (95% confidence interval: 0.86, 0.89) using the combined imaging data and clinical data. This was significantly greater than using image data alone (p=0.005) or clinical data alone (p<0.0001).Discussion and Conclusion
In this study, we set out to explore the potential of deep learning model using clinical data alone, imaging data alone, and the combination of the both, as features for clinical outcome prediction in patients with ICH. T2-weighted FLAIR imaging data demonstrated better predictive power over clinical data. The predictive power of using the integration of both clinical and imaging data outperformed that of using each individual feature set. To mitigate concerns regarding insufficient data for training a deep learning model, transfer learning and data augmentation techniques were employed. We employed a pre-trained VGGNet-19 model as a feature generator to learn high-level features from input FLAIR images. Our study is hypothesis generating and a larger multidimensional study is required to validate our approach.Acknowledgements
References
1. Go, A.S., et al. Executive summary: heart disease and stroke statistics—2013 update: a report from the American Heart Association. Circulation 127, 143-152 (2013).
2. van Asch, C.J., et al. Incidence, case fatality, and functional outcome of intracerebral haemorrhage over time, according to age, sex, and ethnic origin: a systematic review and meta-analysis. The Lancet Neurology 9, 167-176 (2010).
3. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. nature 521, 436-444 (2015).
4. Sahiner, B., et al. Deep learning in medical imaging and radiation therapy. Medical physics 46, e1-e36 (2019).
5. Bengio, Y. Deep learning of representations for unsupervised and transfer learning. in Proceedings of ICML workshop on unsupervised and transfer learning 17-36 (2012).
6. Jenkinson, M., Pechaud, M. & Smith, S. BET2: MR-based estimation of brain, skull and scalp surfaces. in Eleventh annual meeting of the organization for human brain mapping, Vol. 17 167 (Toronto., 2005).
7. Tustison, N.J., et al. N4ITK: Improved N3 Bias Correction. IEEE Transactions on Medical Imaging 29, 1310-1320 (2010).
8. Reinhold, J.C., Dewey, B.E., Carass, A. & Prince, J.L. Evaluating the Impact of Intensity Normalization on MR Image Synthesis. Proc SPIE Int Soc Opt Eng 10949, 109493H (2019).
9. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
10.Krizhevsky, A., Sutskever, I. & Hinton, G.E. Imagenet classification with deep convolutional neural networks. in Advances in neural information processing systems 1097-1105 (2012).
Figures
