1134
Estimating microscopy-informed fibre orientations from in-vivo dMRI using a domain adaptation adversarial network1Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom, 2Oxford Machine Learning in NeuroImaging Lab (OMNI), Department of Computer Science, University of Oxford, Oxford, United Kingdom
Synopsis
Keywords: Tractography, Tractography & Fibre Modelling, Multimodal, Microscopy, structural connectivity, diffusion, machine learning
Motivation: Joint modelling of diffusion MRI and microscopy can leverage their complementary strengths to improve the estimation of fibre orientations. Ideally, these benefits would extend beyond the few datasets where dMRI and microscopy are acquired in the same brain to improve orientation estimates in in-vivo data.
Goal(s): To translate the unique properties of joint dMRI-microscopy data modelling to benefit in-vivo dMRI datasets.
Approach: We construct a domain adaptation adversarial network that can estimate microscopy-informed FODs from single-shell in-vivo dMRI.
Results: Tractography performed using network-derived FODs show improved tracking in grey matter, bottleneck regions, superficial white matter fibres, and long-range structural connectivity.
Impact: Our microscopy-informed neural network improves fibre orientation estimation from in-vivo single-shell dMRI datasets. We demonstrate improvements in fibre tracking that may enable more precise and detailed detection of connectivity, with a broad range of applications in basic and clinical neuroscience.
Introduction
Previous work developed a method for deriving hybrid fibre orientations from datasets with both diffusion MRI (dMRI) and microscopy in the same brain1. Crucially, these hybrid orientations are in 3D, with resolution approaching the microscopy data, primarily informed by myelin staining, and are more axon-specific than those inferred by dMRI. Improved fibre tracking has been demonstrated in grey matter and through bottleneck regions that were poorly characterised by dMRI-only tractography1.This work aims to translate the benefits of the hybrid fibre orientation distributions (FODs) to in-vivo datasets with only dMRI. We leverage the hybrid outputs from a post-mortem macaque brain2 to train a microscopy-informed neural network for fibre orientation estimation that can be applied to the in-vivo dMRI dataset. By using a domain adaptation approach, a network can be trained to be agonistic to tissue state (in-vivo vs post-mortem). The network takes single-shell data as input and can be applied to both humans and macaques without requiring microscopy of the same brain. We validate the network through the generation of anatomically meaningful tractography and demonstrate its ability to recreate the previously observed benefits of post-mortem hybrid tractography in in-vivo scenarios.
Methods
Data: We trained the network using co-registered in-vivo dMRI (b=1 ms/μm2 at 1mm isotropic), post-mortem dMRI (b=4 ms/μm2 at 0.6mm and 1mm isotropic) and microscopy data (Gallyas silver stains for myelin, 0.28 μm/pixel) from a single macaque brain2. The myelin histology was processed using structure tensor analysis3 to estimate the primary fibre orientation per patch of pixels. The network was validated with the macaque dMRI data and Human Connectome Project (HCP) data4.Hybrid FODs: To estimate hybrid microscopy-dMRI orientations, we first derived multiple fibre populations within the dMRI voxels using a ball-and-stick model. For each microscopy pixel within that voxel, we matched its in-plane orientation to the closest dMRI fibre population. We merged the in-plane orientation from the microscopy data with the through-plane orientation of this best match to create a super-resolved 3D hybrid orientation1. Orientations can then be combined over local neighbourhoods to produce hybrid FODs at arbitrary resolutions (Fig1A).
Network-based FOD estimation: We used a supervised domain adaptation adversarial network (DAAN) architecture5 to simultaneously train on both in-vivo and post-mortem dMRI, leveraging the improved data quality post-mortem and ensuring generalisability to the in-vivo data. The DAAN-style architecture had three components5: a feature extractor, a predictor, and a domain classifier. The predictor produced microscopy-informed FODs from diffusion signals using a mean square error loss. The domain classifier was trained to distinguish between in-vivo and post-mortem data, and with the adversarial training, the feature extractor was invariant to the tissue state (Fig1B). The adversarial training was completed using a gradient reversal layer. We input 3x3x3 blocks extracted with sliding blocks of stride=1, to leverage neighbouring information. The input dMRI and output FODs were described using spherical harmonics (l=8) with a channel size of 45.
Evaluation: After training, the network was applied to unseen macaque dMRI datasets. The network FODs were compared to those from the constrained spherical deconvolution (CSD)6 and both were input into the tractography pipeline using MRtrix37. We studied a bottleneck region within the internal capsule (IC), using the trunk, arm and face regions in the motor cortex as seeds and observed the separation of streamlines through the IC8. We investigated the superficial white matter by placing seed/termination masks in adjacent gyri9. We compared our network's output to a tracer-based connectivity matrix from macaques (29 injection ROIs, 91 termination ROIs)10.
Results
With in-vivo macaque dMRI data, network estimated FODs followed anatomical expectations (Fig2) and compared to in-vivo CSD showed a more anatomically accurate pattern of fibres fanning into the cortex, and reduced dispersion.Our network FODs outperformed in-vivo CSD in several tractography tasks (Fig3). The network successfully 1) reconstructed the corticospinal tract showing less of the partial volume effect; 2) resolved the known anterior-posterior organisation of the trunk-arm-face regions passing through the IC bottleneck region; and 3) better delineated superficial (“u-fibre”) white matter tracts.
When compared to tracer data (Fig4) the network-based connectivity matrix showed a better correlation to the tracer connectivity and reduced false positives (regions with no tracer connectivity but tractography streamlines).
Fig5 showed network FOD estimation outputs from HCP data and tractography capturing fine anatomical structures, demonstrating the applicability of our method to in-vivo human datasets.
Future Work
Future work will apply the network to more typical-quality clinical human dMRI, where we hypothesise it will show specific benefits over current methods. The code will be made openly available.Acknowledgements
SZ was supported by the Chinese Government Scholarship. NKD was supported by the Bill and Melinda Gates Foundation. AFDH was supported by the EPSRC, MRC and Wellcome Trust (grants EP/L016052/1 and WT202788/Z/16/A). KLM and SJ were supported by the Wellcome Trust (grants WT202788/Z/16/A, WT215573/Z/19/Z and WT221933/Z/20/Z). The Wellcome Centre for Integrative Neuroimaging is supported by core funding from the Wellcome Trust (WT203139/Z/16/Z). This research was funded in whole, or in part, by the Wellcome Trust (WT202788/Z/16/A, WT215573/Z/19/Z). These authors contributed equally: Amy F.D. Howard, Karla L. Miller.
References
1. Zhu et al., Tackling the gyral bias and bottleneck problems with hybrid diffusion-microscopy tractography in the BigMac dataset. ISMRM 31st Annual Meeting2. Howard AF, et al. An open resource combining multi-contrast MRI and microscopy in the macaque brain, Nature Communication, 2023
3. Budde Md, Frank JA. Examining brain microstructure using structure tensor analysis of histological sections. NeuroImage, 2012
4. Van Essen et al., The WU-Minn Human Connectome Project: An overview. NeuroImage, 2013
5. Ganin et al., Domain-Adversarial Training of Neural Networks, Journal of Machine Learning Research, 2016
6. Tournier JD, et al. Robust determination of the fibre orientation distribution in diffusion MRI: Non-negativity constrained super-resolved spherical deconvolution. NeuroImage, 2007
7. Tournier JD, et al. MRtrix3: A fast, flexible and open software framework for medical image processing and visualisation. NeuroImage, 2019
8. Schilling KG, et al. Prevalence of white matter pathways coming into a single white matter voxel orientation: The bottleneck issue in tractography. Hum Brain Mapp, 2022
9. Reveley et al., Superficial white matter fiber systems impede detection of long-range cortical connections in diffusion MR tractography. PNAS, 2015
10. Markov et al., A Weighted and Directed Interareal Connectivity Matrix for Macaque Cerebral Cortex, Cereb Cortex, 2014
11. Dhollander et al., Fixel-based Analysis of Diffusion MRI: Methods, Applications, Challenges and Opportunities. NeuroImage, 2021
Figures
Figure 1: Overview of the hybrid fibre orientations and domain adaptation adversarial network (DAAN)-style structure. A) Hybrid fibre orientations are constructed by combining in-plane orientation from microscopy myelin staining with through-plane orientation from the dMRI ball and stick model. B) DAAN-style structure estimates hybrid microscopy-informed FODs from dMRI data acquired from post-mortem and in-vivo brains. There are three main components: 1) convolution-based feature extractor. 2) fully connected layer-based predictor 3) domain classifier.
Figure 2: FODs and dispersion comparison in the macaque dMRI dataset. A) The FODs near to grey matter are shown, revealing a clearer fanning pattern with higher specificity in the network FODs in comparison to those obtained using CSD. B) FOD dispersion is quantified with the fixel-based analysis, calculated as the average FOD volume divided by the peak amplitude across fibre populations11. The network FODs exhibit reduced dispersion when compared to the CSD FODs.
Figure 3: Tractography in the macaque dMRI dataset. A) Corticospinal tract reconstruction showing improved delineation in the network FODs. B) Bottleneck problem. ROIs corresponding to the functional control of trunk, arm and face in the motor cortex (left) are shown. Tractography was seeded from these ROIs passing through the internal capsule (IC). Tractography with the network FODs exhibits clear anterior-posterior separation. C) Visualization of superficial white matter fibres connecting neighbouring gyri, highlighting the detailed delineation with the network FODs.
Figure 5: Validation of the HCP data. A) The network FODs and CSD FODs are shown which align with our neuroanatomical expectations. B) Example tractography of the corticospinal tract (CST) and optic radiation (OR) capturing meaningful anatomical features demonstrates the applicability of the network to human data.