1872
A robust transfer learning method to improve early diagnosis of autism spectrum disorder classification1AU MRI Research Center, Electrical and Computer Engineering, Auburn University, Auburn, AL, United States
Synopsis
Overfitting, the main issue that constrains the validity and generalizability of machine-learning in neuroimaging-based diagnostic-classification, is in part due to small sample-sizes in relation to what is required for generalization. Even with data aggregation (such as in ABIDE), the relatively smaller sample-sizes are a result of the fact that it is difficult/expensive to acquire data from clinical-populations. With healthy-controls, we have comparatively larger samples available. Therefore, we propose to address overfitting by using larger healthy-samples (HCP) to learn the neural signature of healthy-controls, with the aim of transferring that learning into the context of discriminating Autism from healthy-controls.
Introduction
Deep-learning models outperform traditional machine-learning methods in diagnostic classification1-3. With respect to autism spectrum disorders (ASD), open-source datasets such as ABIDE (Autism-Brain-Imaging-Data-Exchange) have accelerated the application of deep-learning to diagnostic classification4,5. However, overfitting is the main issue that constrains the validity and generalizability of deep learning, as well as traditional machine-learning1,6. However, the workings of deep-learning remain a black box, and hence it is difficult to understand the mechanistic and semantic basis for arriving at the decision, thus precluding an understanding of the neural mechanisms that may help distinguish between the groups. Here, we address these two issues. Overfitting is in part due to small sample-sizes in relation to what is required for generalization. Even with data aggregation (such as in ABIDE), the relatively smaller sample-sizes are a result of the fact that it is difficult and expensive to acquire data from clinical populations. With healthy-controls, we have comparatively larger samples available in the public domain. Therefore, we propose to address overfitting by using larger healthy samples to learn the neural signature of healthy-controls, with the aim of transferring that learning into the context of discriminating clinical populations from healthy-controls. The utility of such transfer-learning has been demonstrated in unrelated contexts7,8 and to a limited extent in disorder classification from neuroimaging data9-11. Here, we investigate the utility of transfer-learning from HCP (human-connectome-project) data for improving ASD classification from ABIDE data.Methods
In transfer-learning, the source-domain is defined as the space from which initial learning is derived. We utilized pre-processed HCP-data from healthy-controls for this purpose12. Likewise, the target-domain is defined as the space to which the classifier pre-trained in the source-domain is applied, in order to make the classification decision of interest. For this, we utilized data from subjects with ASD and healthy-controls available in the ABIDE database13, pre-processed with a pipeline identical to that used for HCP-data. We split the 895 subjects of ABIDE data into train (N=705) and independent test (N=190) datasets in the following ways: (i) random-matched-split: subjects drawn randomly so that training and testing datasets are matched on non-imaging measures, (ii) site-split: training data from 11 sites and testing data from 7 different sites. This was done to test the robustness of the model to different sources of training and testing datasets. Cross-validation tends to over-estimate the performance of a classifier6, and hence, we preferred independent test data to evaluate the model. During model-training in the source-domain, we used the convolutional VAE (variational-auto-encoder) on controls in HCP-data14,15. We applied an adaptive CNN model in the target-domain for classification between ASD and controls using ABIDE, with transferred parameters initializing the pre-trained VAE model from the source-domain (Fig.1). The classification performance was evaluated in terms of change in classification accuracy with increasing number of subjects from HCP-data used during pre-training. The inputs were functional connectivity matrices in each subject, extracted by estimating Pearson’s-correlation between pre-processed time-series from 200 brain-regions obtained from the Craddock atlas16. In order to provide mechanistic and semantic insights underlying the decision, a layer-wise relevant propagation (LRP) method17 was used18, producing heat-maps19, which can be interpreted in terms of the importance of functional connections towards making the decision.Results
The AUC (area-under-the-curve) in Fig.2 show that transfer-learning using combined VAE-CNN classification model can achieve 0.742 AUC on site-split and 0.754 AUC on matched-random-split, outperforming those obtained without transfer-learning (corresponding ROC curves in Fig.3). While a site-split deteriorated performance in the traditional model in comparison to matched-split (as expected6), transfer-learning helped site-split catch up with the matched-split with inclusion of more HCP-data in the transfer-learning model. The discriminative features from the heat map obtained from LRS show hyperconnectivity in fronto-temporal and ironto-insular pathways mainly in the right hemisphere, and under-connectivity in larger number of inter-hemisphere and inter-lobular connections in ASD (Fig.4).Discussion
In both matched and site-dependent splits, transfer-learning from HCP using a VAE-CNN classifier outperforms a traditional deep-learning CNN classifier without transfer-learning. While we show that performance of the transfer-learning model improves with the inclusion of more HCP-data, the curve by no means saturates. This means that inclusion of more healthy-control subjects in transfer-learning, possibly from a more diverse source such as UK biobank20 might further improve accuracy and generalizability. It is noteworthy that the accuracies we have obtained on independent test data are higher than those obtained from cross-validation in ABIDE21 (even though cross-validation over-estimates performance6), as well as on independent test data using traditional machine-learning6 and other transfer-learning approaches22. It is encouraging to note that transfer-learning seems to alleviate deterioration of performance due to site-dependent splits. This may help in multi-site studies. Widespread under-connectivity in inter-hemispheric and inter-lobar connections with sporadic over-connectivity in the right hemisphere in ASD is in line with previous results as well23-25.Conclusion
We have demonstrated the applicability of transfer-learning within a deep-learning framework for utilizing larger samples of available healthy-control data to improve generalizability and accuracy of diagnostic classification in ASD, as well as reduce the harmful effects of inter-site variability on classification.Acknowledgements
No acknowledgement found.References
1. Durstewitz, Daniel & Koppe, Georgia & Meyer-Lindenberg, Andreas. (2019). Deep neural networks in psychiatry. Molecular Psychiatry. 24. 1. 10.1038/s41380-019-0365-9.
2. Wen, D., Wei, Z., Zhou, Y., Li, G., Zhang, X., & Han, W. (2018). Deep Learning Methods to Process fMRI Data and Their Application in the Diagnosis of Cognitive Impairment: A Brief Overview and Our Opinion. Frontiers in neuroinformatics, 12, 23. doi:10.3389/fninf.2018.00023
3. Vieira, S., Pinaya, W.H., & Mechelli, A. (2017). Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications. Neuroscience & Biobehavioral Reviews, 74, 58-75.
4. Nielsen, J.A., et al., (2013). Multisite functional connectivity MRI classification of autism: ABIDE results. Front. Hum. Neurosci. 7 (September), 1–12.
5. Di Martino, A., O’Connor, D., Chen, B. et al. Enhancing studies of the connectome in autism using the autism brain imaging data exchange II. Sci Data 4, 170010 (2017) doi:10.1038/sdata.2017.10
6. P Lanka, D Rangaprakash, MN Dretsch, JS Katz, TS Denney, G Deshpande, "Supervised machine learning for diagnostic classification from large-scale neuroimaging datasets", Brain Imaging and Behavior, 2019 (in press).
7. Sarinnapakorn, Kanoksri & Kubat, M.. (2008). Combining Subclassifiers in Text Categorization: A DST-Based Solution and a Case Study. Knowledge and Data Engineering, IEEE Transactions on. 19. 1638-1651. 10.1109/TKDE.2007.190663.
8. Zhang, J., Li, W., Ogunbona, P., & Xu, D. (2019). Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition: A Problem-Oriented Perspective. ACM Comput. Surv., 52, 7:1-7:38.
9. Zou, L., Zheng, J., Miao, C., McKeown, M.J., & Wang, Z.J. (2017). 3D CNN Based Automatic Diagnosis of Attention Deficit Hyperactivity Disorder Using Functional and Structural MRI. IEEE Access, 5, 23626-23636.
10. Hosseini-Asl, E., Ghazal, M., Mahmoud, A., Aslantas, A., Shalaby, A.M., Casanova, M.F., Barnes, G., Gimel'farb, G.L., Keynton, R.S., & El-Baz, A.S. (2018). Alzheimer's disease diagnostics by a 3D deeply supervised adaptable convolutional network. Frontiers in bioscience, 23, 584-596 .
11. Cheng, B., Liu, M., Zhang, D. et al. Brain Imaging and Behavior (2019) 13: 138.
12. S.M.Smithetal. (2013).Resting-state fMRI in the Human Connectome Project, Neuroimage, vol. 80, pp. 144–168.
13. Cameron Craddock, et al., (2013). The Neuro Bureau Preprocessing Initiative: open sharing of preprocessed neuroimaging data and derivatives. In Neuroinformatics 2013, Stockholm, Sweden.
14. D. Kingma, M. Welling, Auto-Encoding Variational Bayes, ICLR, 2014
15. Doersch, C. (2016). Tutorial on Variational Autoencoders. ArXiv, abs/1606.05908.
16. Craddock, R.C., James, G.A., Holtzheimer, P.E., Hu, X.P., & Mayberg, H.S. (2012). A whole brain fMRI atlas generated via spatially constrained spectral clustering. Human brain mapping, 33 8, 1914-28 .
17. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K., Samek, W., & Suárez, O.D. (2015). On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PloS one.
18. Boehle, M.D., Eitel, F., Weygandt, M., & Ritter, K. (2019). Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification. Front. Aging Neurosci..
19. Simonyan, K., Vedaldi, A., & Zisserman, A. (2013). Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. CoRR, abs/1312.6034.
20. He, T., Kong, R. (2019) Deep Neural Networks and Kernel Regression Achieve Comparable Accuracies for Functional Connectivity Prediction of Behavior and Demographics. NeuroImage doi: 10.1016/j.neuroimage.2019.116276
21. He, L., Li, H., Holland, S. K., Yuan, W., Altaye, M., and Parikh, N. A. (2018). Early prediction of cognitive deficits in very preterm infants using functional connectome data in an artificial neural network framework. Neuroimage Clin. 18, 290–297. doi: 10.1016/j.nicl.2018.01.032
22. Li, H., Parikh, N.A., & He, L. (2018). A Novel Transfer Learning Approach to Enhance Deep Neural Network Classification of Brain Functional Connectomes. Front. Neurosci.
23. Cherkassky, V.L., Kana, R.K., Keller, T.A., Just, M.A., (2006). Functional connectivity in a baseline resting-state network in autism. NeuroReport 17 (16), 1687–1690.
24. Hagen, E.A., Stoyanova, R.S., Baron-Cohen, S., & Calder, A.J. (2013). Reduced functional connectivity within and between ‘social’ resting state networks in autism spectrum conditions. Social cognitive and affective neuroscience.
25. Lee, J.E., Bigler, E.D., Alexander, A.L., et al. (2007). Diffusion tensor imaging of white matter in the superior temporal gyrus and temporal stem in autism. Neuroscience Letters, 424, 127–32.
Figures
