2157
Pointwise analysis of tract microstructure using geometric deep learning for language performance prediction1Harvard Medical School, Boston, MA, United States, 2The University of Sydney, Sydney, Australia, 3The University of New South Wales, Sydney, Australia, 4University of Electronic Science and Technology of China, Chengdu, China
Synopsis
Keywords: Tractography, Tractography & Fibre Modelling, Point cloud, Deep learning
Motivation: The prediction of cognitive performance scores using diffusion MRI tractography enables the study of relationships between brain structure and function.
Goal(s): Our goal is to achieve accurate prediction of cognition and identify critical brain regions for prediction.
Approach: We propose a geometric deep-learning framework for language score prediction. It utilizes a point cloud representation of fiber tracts for detailed spatial and microstructure information and incorporates a novel regression loss to utilize the continuity of language scores.
Results: Our method outperforms comparison methods with state-of-the-art representations of fiber tracts and identifies predictive language-related brain regions.
Impact: Our proposed novel geometric deep learning framework using a point cloud representation of fiber tracts can be applied to various tractography-based prediction tasks to improve performance and provide a probe to explore relationships between brain structure and function.
Introduction
The brain’s white matter connections (fiber tracts) and their tissue microstructure can be quantitatively mapped using diffusion magnetic resonance imaging (dMRI) tractography1, enabling the study of the brain’s structural connectivity2. To better understand how brain structure relates to function, recent research explores the prediction of individual cognitive performance based on structural neuroimaging data (such as dMRI)3,4. A critical challenge is how to represent white matter tracts and their tissue microstructure. Traditional approaches usually involve averaging or binning data along the streamline (Figure 1), ignoring streamline-specific or pointwise information4–6. Another challenge is the improvement of performance in regression-based prediction. Previous methods often ignore the intrinsic continuity in regression scores, resulting in suboptimal performance7. In addition, the interpretation of predictive brain regions is a notable challenge drawing substantial attention8–11. Most existing studies identify entire white matter connections as important while ignoring subregions within them4,10. Therefore, we propose a novel geometric deep learning framework, which includes a point cloud representation of tracts to utilize pointwise information, a novel regression loss to capture the continuity of regression scores, and a critical region localization algorithm to identify predictive brain regions within tracts. Our proposed method is evaluated by predicting individual language performance scores based on individual white matter tracts.Methods
Our method was evaluated on dMRI data and two language-related assessment scores, the NIH Toolbox Picture Vocabulary Test (TPVT) and the Toolbox Oral Reading Recognition Test (TORRT)12, from 809 subjects of the Human Connectome Project. Whole brain tractography was generated from dMRI using a two-tensor unscented Kalman filter method13, followed by identification of white matter tracts14. The left arcuate fasciculus (AF), inferior longitudinal fasciculus (ILF), and superior longitudinal fasciculi II (SLF-II) were selected for prediction due to their relationship to language15.Each white matter tract was represented as a point cloud (Figure 1). Each streamline point was characterized by its three spatial coordinates and two additional measurements (tract-specific fractional anisotropy (FA) and number of streamlines (NoS)). During each training iteration, the input for the neural network was formed by randomly sampling a fixed number of points from the point cloud.
Our point-based neural network for performing regression is shown in Figure 2. We designed a Siamese network16 that contains two subnetworks with shared weights, where the subnetworks were adapted from the widely-used PointNet17. A pair of point clouds are input to the Siamese network during training, and a pair of language scores are predicted. We propose a new loss function Lps defined as follows:
$$L_{ps}=1/N_b \sum_{i} ((y_{i1}-y_{i2})-(\hat{y}_{i1}-\hat{y}_{i2}))^2 $$
where $$$y_{i1}$$$ and $$$y_{i2}$$$ are the labels of the input pair, and $$$\hat{y}_{i1}$$$ and $$$\hat{y}_{i2}$$$ are the predicted scores of the input pair. This loss enables the Siamese network to utilize information about the difference between the regression scores of the two inputs. The total loss then includes the proposed loss and a typical MSE regression loss. We trained a network for each tract and each regression task (TPVT/TORRT score prediction).
We propose a Critical Region Localization algorithm to identify critical regions within fiber tracts for language score prediction. First, the subject-wise contributing points are identified as point sets that contribute to the max-pooled features. Then group-wise analysis is performed to localize critical regions that are consistently important for prediction across testing subjects.
Results
The Pearson correlation coefficient (r) was adopted as the evaluation metric of prediction performance, following typical practice in the prediction of neurocognitive scores3,4,18. We compared our proposed method with several baseline methods that use different tract representations (mean value and AFQ6) and regression models (ElasticNet and 1D-CNN). As shown in Figure 3, our approach consistently outperformed baseline methods across all fiber tracts for both prediction tasks, as demonstrated by higher r values (Figure 3). Critical predictive regions, as shown in Figure 4, were distributed across the left hemisphere and all cerebral lobes for both assessments.Discussion
Our method effectively tackles challenges in predicting cognition with dMRI data. It adopts a point cloud representation of fiber tracts for detailed spatial and microstructure information and incorporates a novel regression loss to utilize information about the difference between continuous regression scores, leading to superior prediction performance. In addition, our method successfully identifies critical brain regions within white matter tracts for language score prediction.Conclusion
In this work, we propose a novel geometric deep learning framework for the prediction of language scores using white matter tracts represented as point clouds. Evaluated on a large-scale public dataset, our method showed superior prediction performance and successfully identified brain regions highly predictive of language scores.Acknowledgements
We acknowledge funding provided by the following National Institutes of Health (NIH) grants: R01MH125860, R01MH119222, and R01MH132610.References
1. Zhang, F. et al. Quantitative mapping of the brain’s structural connectivity using diffusion MRI tractography: A review. Neuroimage 249, 118870 (2022).
2. Suárez, L. E., Markello, R. D., Betzel, R. F. & Misic, B. Linking Structure and Function in Macroscale Brain Networks. Trends Cogn. Sci. 24, 302–315 (2020).
3. Gong, W., Beckmann, C. F. & Smith, S. M. Phenotype discovery from population brain imaging. Med. Image Anal. 71, 102050 (2021).
4. Feng, G. et al. Methodological evaluation of individual cognitive prediction based on the brain white matter structural connectome. Hum. Brain Mapp. 43, 3775–3791 (2022).
5. O’Donnell, L. J., Westin, C.-F. & Golby, A. J. Tract-based morphometry for white matter group analysis. Neuroimage 45, 832–844 (2009).
6. Yeatman, J. D., Dougherty, R. F., Myall, N. J., Wandell, B. A. & Feldman, H. M. Tract profiles of white matter properties: automating fiber-tract quantification. PLoS One 7, e49790 (2012).
7. Zha, K., Cao, P., Yang, Y. & Katabi, D. Supervised Contrastive Regression. arXiv [cs.LG] (2022).
8. Cui, H. et al. Interpretable Graph Neural Networks for Connectome-Based Brain Disorder Analysis. in Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 375–385 (Springer Nature Switzerland, 2022).
9. Li, X. et al. BrainGNN: Interpretable Brain Graph Neural Network for fMRI Analysis. Med. Image Anal. 74, 102233 (2021).
10. Chen, Y. et al. TractGraphCNN: anatomically informed graph CNN for classification using diffusion MRI tractography. arXiv [eess.IV] (2023).
11. Dan, T. et al. Neuro-RDM: An Explainable Neural Network Landscape of Reaction-Diffusion Model for Cognitive Task Recognition. in Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 365–374 (Springer Nature Switzerland, 2022).
12. Weintraub, S. et al. Cognition assessment using the NIH Toolbox. Neurology 80, S54–64 (2013).
13. Malcolm, J. G., Shenton, M. E. & Rathi, Y. Filtered multitensor tractography. IEEE Trans. Med. Imaging 29, 1664–1675 (2010).
14. Zhang, F. et al. An anatomically curated fiber clustering white matter atlas for consistent white matter tract parcellation across the lifespan. Neuroimage 179, 429–447 (2018).
15. Zekelman, L. R. et al. White matter association tracts underlying language and theory of mind: An investigation of 809 brains from the Human Connectome Project. Neuroimage 246, 118739 (2022).
16. Chopra, S., Hadsell, R. & LeCun, Y. Learning a Similarity Metric Discriminatively, with Application to Face Verification. in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) 539–546 (2005).
17. Qi, C. R., Su, H., Mo, K. & Guibas, L. J. Pointnet: Deep learning on point sets for 3d classification and segmentation. in Proceedings of the IEEE conference on computer vision and pattern recognition 652–660 (2017).
18. Tian, Y. & Zalesky, A. Machine learning prediction of cognition from functional connectivity: Are feature weights reliable? Neuroimage 245, 118648 (2021).
Figures
