2158

ATM: Anatomy to Tract Mapping
Yee-Fan Tan1,2, Siyuan Liu3, Raphaël C.-W. Phan1, Chee-Ming Ting1, and Pew-Thian Yap2
1School of Information Technology, Monash University, Subang Jaya, Malaysia, 2Department of Radiology and Biomedical Research Imaging Center (BRIC), UNC Chapel Hill, Chapel Hill, NC, United States, 3Marine Engineering College, Dalian Maritime University, Dalian, China

Synopsis

Keywords: Tractography, Tractography & Fibre Modelling

Motivation: Conventional diffusion tractography relies on error-prone voxel-to-voxel tracing and typically demands diffusion MRI with high signal-to-noise ratio, spatial and angular resolution, which can be challenging to acquire.

Goal(s): To generate bundle-specific streamlines from anatomical MRI.

Approach: We present a deep learning framework for anatomy to tract mapping (ATM), allowing bundle-specific streamlines to be generated from anatomical MRI. ATM generates streamlines without resorting to voxel-to-voxel tracing, hence sidesteps challenges involved in tracing across complex configurations such as crossings, kissing, and bending and the bottlenecks where multiple bundles converge toward before re-emerging.

Results: ATM effectively captures bundle shapes and generates bundle-specific streamlines from T1-weighted MRI.

Impact: We demonstrate that tract streamlines can be estimated directly from anatomical MRI. This allows (1) tractography in the absence of diffusion MRI and (2) anatomy tractography to guide diffusion tractography.

Introduction

Diffusion tractography methods typically leverage local orientation information derived from diffusion MRI to direct streamline propagation from seed locations. However, the process relies on voxel-to-voxel tracing, which can be prone to errors, potentially leading to the generation of false-positive streamlines1,2. Moreover, the successful outcome of tractography heavily depends on the quality of the dMRI data, such as signal-to-noise ratio, spatio resolution and angular resolution. High-quality dMRI data can be challenging to acquire and can be unavailable to tractography3. Here, we present an anatomy to tract mapping (ATM) that is based on deep learning for reconstructing full streamlines without relying on voxel-to-voxel tracing. We demonstrate the effectiveness of the method in mapping anatomical information from T1-weighted (T1w) MRI data to bundle-specific streamlines.

Methods

ATM consists of two main components: (1) a segmentation module with a volumetric image encoder to extract anatomical features from T1w images and a decoder to predict the binary segmentation map of the tract of interest, (2) a streamline variational autoencoder (VAE) module that employs an encoder to embed streamline information into a low-dimensional space. The resulting latent samples, together with anatomical features conditioned using Feature-wise Linear Modulation4 (FiLM) layers, are input to a decoder to reconstruct the target streamline. The segmentation module of ATM comprises a total of n 3D convolutional downsampling blocks, followed by n-1 upsampling blocks. The entire module is complemented by a long-skip connection for precise tract segmentation. These convolutional blocks employ a kernel size of 3, and each block is equipped with filters specified as {32, 64, 128, 256}, followed by upsampling blocks with filter sizes of {128, 64, 32}. The final upsampling block is followed by a 3D convolutional output layer with a filter size of 1. The encoder-decoder component of the streamline VAE module is structured with n 1D Convolutional downsampling and upsampling blocks. This module utilizes a kernel size of 63, a latent space dimensionality of 32, and filter size for each downsampling and upsampling block are set as {32, 64, 128, 256} and {256, 128, 64, 32}, respectively. Each upsampling block is coupled with a FiLM layer for adaptive feature modulation except the output layer with a filter size of 3. These FiLM layers receive input from the FiLM shared-weight block, which consists of two layers of Multi-Layer Perceptron (MLP). Fig. 1 shows a comprehensive overview of the architectural details of ATM.
The training of ATM is supervised by 3D ground truth T1w MRI images along with their corresponding binary segmentations, which indicate the occupancy of the tract of interest on the voxel grid corresponding to the T1w MRI volumes, as well as the streamlines of the tract. The segmentation module is trained using the Dice loss function5. The training of the streamline VAE aligns with beta-VAE6, incorporating an adjustable hyperparameter, β, to control the emphasis on learning statistically independent latent factors. Mean Squared Error (MSE) is employed to minimize the streamline reconstruction loss. ATM is trained using the Adam7 optimizer with a learning rate of 1e-4 for a total of 2000 epochs.

Results

Data: We used the TractSeg dataset8, which comprises 72 manually validated bundles from 105 subjects in the Human Connectome Project (HCP) diffusion database9,10. We obtained the corresponding T1w MRI images from the HCP database and generated binary segmentation maps of the target tract using DSI Studio11. ATM was trained using 80 subjects from the dataset. The experiments focused on the CST_left, CC2, and CC7 tracts.
ATM takes latent samples drawn from a Gaussian distribution and anatomical features extracted from T1w images to generate bundle-specific streamlines. The quality of the generated tracts is evaluated based on the shape similarity (SM) metric from Bundle Analytics12 (BUAN). The SM values for the generated CST_left, CC2, and CC7 are 0.7695, 0.6265, and 0.7195, respectively. Higher SM values indicate closer resemblance to the ground truth. Fig. 2. illustrates the comparison between the ground truth and the generated tracts for subject 677968. Fig. 3. shows the tracts in T1w space.

Discussion

Our results demonstrate the potential of ATM in generating tracts solely from anatomical features. However, it is noteworthy that the SM scores of CC2 and CC7 show lower-quality results, likely due to its inherent complexity. Further work entails (1) optimization of network architecture and (2) validation with more tract bundles (3) Streamlines filtering.

Conclusion

We proposed a novel deep learning approach for anatomy to tract mapping. Experimental results demonstrate the feasibility of generating bundle-specific streamlines guided by anatomical information. We expect further performance improvement with refinement of network architecture.

Acknowledgements

This work was supported in part by National Institutes of Health (NIH) grants EB008374 and MH125479.

References

  1. P. Poulin, D. Jörgens, P.-M. Jodoin, and M. Descoteaux, “Tractography and machine learning: Current state and open challenges,” Magnetic Resonance Imaging, vol. 64, pp. 37–48, 2019.
  2. J. H. Legarreta, L. Petit, P.-M. Jodoin, and M. Descoteaux, “Generative sampling in bundle tractography using autoencoders (GESTA),” Medical Image Analysis, vol. 85, p. 102761, 2023.
  3. L. Y. Cai, H. H. Lee, N. R. Newlin, C. I. Kerley, P. Kanakaraj, Q. Yang, G. W. Johnson, D. Moyer, K. G. Schilling, F. C. Rheault, et al., “Convolutional-recurrent neural networks approximate diffusion tractography from T1-weighted MRI and associated anatomical context,” bioRxiv 2023.
  4. E. Perez, F. Strub, H. D. Vries, V. Dumoulin, and A. Courville, “FiLM: Visual reasoning with a general conditioning layer,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, 2018.
  5. F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation,” Nature Methods, vol. 18, no. 2, pp. 203–211, 2021.
  6. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “beta-VAE: Learning basic visual concepts with a constrained variational framework,” in International Conference on Learning Representations, 2016.
  7. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  8. J. Wasserthal, P. Neher, and K. H. Maier-Hein, “TractSeg - fast and accurate white matter tract segmentation,” NeuroImage, vol. 183, pp. 239–253, 2018.
  9. D. C. Van Essen, S. M. Smith, D. M. Barch, T. E. Behrens, E. Yacoub, K. Ugurbil, W.-M. H. Consortium, et al., “The Wu-Minn human connectome project: an overview,” NeuroImage, vol. 80, pp. 62–79, 2013.
  10. M. F. Glasser, S. N. Sotiropoulos, J. A. Wilson, T. S. Coalson, B. Fischl, J. L. Andersson, J. Xu, S. Jbabdi, M. Webster, J. R. Polimeni, et al., “The minimal preprocessing pipelines for the human connectome project,” NeuroImage, vol. 80, pp. 105–124, 2013.
  11. F.-C. Yeh, T. D. Verstynen, Y. Wang, J. C. Fernández-Miranda, and W.-Y. I. Tseng, “Deterministic diffusion fiber tracking improved by quantitative anisotropy,” PloS one, vol. 8, no. 11, p. e80713, 2013.
  12. B. Q. Chandio, S. L. Risacher, F. Pestilli, D. Bullock, F.-C. Yeh, S. Koudoro, A. Rokem, J. Harezlak, and E. Garyfallidis, “Bundle analytics, a computational framework for investigating the shapes and profiles of brain pathways across populations,” Scientific Reports, vol. 10, no. 1, p. 17149, 2020.

Figures

Figure 1. Overview of ATM.

Figure 2. Ground truth and generated tracts for CST_left, CC2, and CC7 using ATM.

Figure 3. Sagittal view of ground truth and ATM tracts for CST_left, CC2, and CC7 in T1W space.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
2158
DOI: https://doi.org/10.58530/2024/2158