3497

Mapping transient coactivity patterns of brain in latent space with variational autoencoder neural network
Kaiming Li1 and Xiaoping Hu1
1Department of Bioengineering, UC Riverside, Riverside, CA, United States

Synopsis

The brain is a complex dynamic system that constantly evolves. Characterization of the spatiotemporal dynamics of brain activity is fundamental to understand how brain works. Current studies with functional connectivity and linear models are limited by sacrificed temporal resolution and insufficient model capacity. With a generative variational auto encoder (VAE), the present study mapped the high-dimensional transient co-activity patterns (CAPs) of large datasets in a low-dimensional latent space. We demonstrated with multiple datasets that VAE can effectively represent the transient CAPs in latent space, paving the way for frame-wise modeling of the complex spatiotemporal dynamics in future.

INTRODUCTION

The brain is a complex dynamic system with multiple spontaneous neural processes interacting with each other in a dynamic and coordinated manner 1,2. Characterization of the complex spatiotemporal dynamics is fundamental to advance our knowledge of how the brain works. With resting-state fMRI (rs-fMRI), recent studies have begun to investigate temporal patterns by time-varying functional connectivity (FC) analysis 3-6. However, most efforts with FC 7-10 and linear models 11-13 have been limited by sacrificed temporal resolution and insufficient model capability. Inspired by the concept of transient co-activation maps (CAPs) 14, the present study mapped the high-dimensional transient CAPs of large datasets in a low-dimensional latent space with a generative variational autoencoder (VAE) 15 and reconstructed the latent processes for future frame-wise temporal modeling.

METHODS

Datasets and preprocessing. Three public brain imaging datasets (ABIDE-1 16, HCP 17, and B-SNIP 18 ) were used in the present study. Standard preprocessing 19 was performed on the ABIDE-1 and B-SNIP datasets. For HCP, we used the preprocessed dataset directly. The time series of each voxel was z-scored and thresholded with the 85th percentile of the entire time series 14, resulting in a transient CAPs map at each time frame. The resultant transient CAPs were then spatially downsampled to 1cm isotropic for subsequent analysis.

The VAE model. The model contains an encoder and a decoder, as shown in Figure 1(A). The encoder maps an overt high-dimensional transient CAP image $$$x$$$ in the image space to a low-dimensional attribute $$$z$$$ in the latent space. Considering the high individual variability of subjects in brain anatomy, the encoder represents a statistical distribution of individual transient CAP $$$x:q_{\theta}(z|x)$$$. Given an attribute $$$z$$$ in the latent space, the VAE model will generate a sample $$$\widehat{x}$$$ with the decoder $$$p_{\phi}(x|z)$$$, which is a statistical model. Note that both $$$x$$$ and $$$z$$$ are multivariate vectors. In the present study, the state attribute $$$z$$$ followed a multivariate normal distribution. The dimension of latent space was set to 30 for low-resolution datasets (ABIDE-1 and B-SNIP) and to 50 for high-resolution datasets (HCP) or for comparison across datasets.

Representation of latent dimension in image space. After training, we reconstructed the representation of each latent dimension in the image space by generating an image sample for the unit latent vector along that dimension, using the trained decoder. The representations of all latent dimensions were clustered with hierarchical clustering for visual inspection. The distance amongst the representations was measured in the Euclidean space.

Reproducibility and robustness. Models were separately trained on four HCP rs-fMRI sessions, with weights initialized from a model trained on a random session. Model weights were visualized, and their distributions were compared. We also tested whether the VAE model trained on ABIDE-1 can be used on B-SNIP.

RESULTS

Latent space mapping and CAPs reconstruction. Figure 1 demonstrates the scheme of the VAE model. In Panel 1B, the model was trained on ABIDE-1. The original transient CAPs, the encoded latent state attribute $$$z$$$ and the reconstructed transient CAPs are depicted, respectively, in each row. As seen, the VAE model was able to reconstruct the main bodies of the original CAPs and denoise them.
In particular, with the VAE model trained on ABIDE-1, we were able to reconstruct the transient CAPs in B-SNIP dataset (Figure 1C), suggesting that this model is robust across datasets with similar image resolutions. Losses of training and validation (Figure 2) indicate the convergence of VAE in our experiment.

Representation of latent dimensions in the image space. To interpret the latent space and the latent attributes, we generated an image sample for the unit attribute vector along each dimension using the trained decoder and showed them in Figure 3B. The images were arranged according to their similarity in the image space (Figure 3A). Note the similar but gradually changing patterns in the image space. Latent attributes connect frame-by-frame and form processes. The processes can represent the temporal dynamics of transient CAPs compactly.

Reproducibility and robustness. In Figure 4, we visualized the first (FC1) and the last layer (FC4) of the VAE models trained on low-resolution datasets (ABIDE-1 and B-SNIP) and high-resolution ones (four HCP rsfMRI sessions). For the four HCP sessions, the trained models were very similar for both layers, as seen in both the weight matrices (upper row) and the weight histogram (lower row). The model from low-resolution datasets was generally similar to those from HCP, but lost details as revealed by the sparser weight matrix, particularly in layer FC4. We further examined the differences in FC4 between low- and high-resolution models and found that they mainly resided in the outer space of the brain, the temporal lobe, and places close to the sinus (Figure 5). These regions usually suffer signal loss and image distortion in the acquisition.

DISCUSSION & CONCLUSION

Our study investigated the representation of transient CAPs in a low-dimension latent space using a generative variational auto-encoder. We found that VAE can represent the transient CAPs of large brain imaging datasets in latent space effectively and compactly, paving the way for modeling the complex spatiotemporal dynamics of brain frame-wisely in future.

Acknowledgements

No acknowledgement found.

References

  1. Shine JM, Breakspear M, Bell PT, et al. Human cognition involves the dynamic integration of neural activity and neuromodulatory systems. Nat Neurosci. 2019;22(2):289-296.2.
  2. Spadone S, Della Penna S, Sestieri C, et al. Dynamic reorganization of human resting-state networks during visuospatial attention. Proc Natl Acad Sci U S A. 2015;112(26):8112-8117.3.
  3. Sakoglu U, Pearlson GD, Kiehl KA, Wang YM, Michael AM, Calhoun VD. A method for evaluating dynamic functional network connectivity and task-modulation: application to schizophrenia. MAGMA. 2010;23(5-6):351-366.4.
  4. Chang C, Glover GH. Time-frequency dynamics of resting-state brain connectivity measured with fMRI. Neuroimage. 2010;50(1):81-98.5.
  5. Hutchison RM, Womelsdorf T, Allen EA, et al. Dynamic functional connectivity: Promise, issues, and interpretations. NeuroImage. 2013;80(Hum. Brain Mapp. 31 2010):360--378.6.
  6. Calhoun VD, Miller R, Pearlson G, Adalı T. The chronnectome: time-varying connectivity networks as the next frontier in fMRI data discovery. Neuron. 2014;84(2):262-274.7.
  7. Vanni MP, Chan AW, Balbi M, Silasi G, Murphy TH. Mesoscale Mapping of Mouse Cortex Reveals Frequency-Dependent Cycling between Distinct Macroscale Functional Modules. J Neurosci. 2017;37(31):7513-7533.8.
  8. Aruljothi K, Marrero K, Zhang Z, Zareian B, Zagha E. Functional Localization of an Attenuating Filter within Cortex for a Selective Detection Task in Mice. J Neurosci. 2020;40(28):5443-5454.9.
  9. Allen EA, Damaraju E, Plis SM, Erhardt EB, Eichele T, Calhoun VD. Tracking whole-brain connectivity dynamics in the resting state. Cereb Cortex. 2014;24(3):663-676.10.
  10. Zalesky A, Breakspear M. Towards a statistical test for functional connectivity dynamics. Neuroimage. 2015;114:466-470.11.
  11. Beckmann CF, DeLuca M, Devlin JT, Smith SM. Investigations into resting-state connectivity using independent component analysis. Philos Trans R Soc Lond B Biol Sci. 2005;360(1457):1001-1013.12.
  12. Vidaurre D, Smith SM, Woolrich MW. Brain network dynamics are hierarchically organized in time. Proceedings of the National Academy of Sciences. 2017;114(48):12827-12832.13.
  13. Taghia J, Cai W, Ryali S, et al. Uncovering hidden brain state dynamics that regulate performance and decision-making during cognition. Nat Commun. 2018;9(1):2505.14.
  14. Liu X, Duyn JH. Time-varying functional network information extracted from brief instances of spontaneous brain activity. Proc Natl Acad Sci U S A. 2013;110(11):4392-4397.15.
  15. Kingma DP, Welling M. Auto-Encoding Variational Bayes. arXiv e-prints. 2013:arXiv:1312.6114. https://ui.adsabs.harvard.edu/abs/2013arXiv1312.6114K. Accessed December 01, 2013.16.
  16. Di Martino A, Yan CG, Li Q, et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol Psychiatry. 2014;19(6):659-667.17.
  17. Van Essen DC, Smith SM, Barch DM, et al. The WU-Minn Human Connectome Project: an overview. Neuroimage. 2013;80:62-79.18.
  18. Tamminga CA, Pearlson G, Keshavan M, Sweeney J, Clementz B, Thaker G. Bipolar and schizophrenia network for intermediate phenotypes: outcomes across the psychosis continuum. Schizophr Bull. 2014;40 Suppl 2:S131-137.19.
  19. Li K, Sweeney JA, Hu XP. Context-dependent dynamic functional connectivity alteration of lateral occipital cortex in schizophrenia. Schizophr Res. 2020.

Figures

Figure 1. (A) a scheme of the VAE model. (B) transient CAPs and latent attribute reconstruction with ABIDE-1 data. Each time series describes a latent attribute $$$z$$$ (dim=50). The five slices on its left show a raw transient CAP or $$$x$$$ while the five slices on its right depict a reconstructed sample $$$\widehat{x}$$$. For each slice, the top is the left hemisphere, and the bottom is the right hemisphere. The tested images are from the ABIDE-1 dataset. (C). Reconstruction of B-SNIP images with the model learned on ABIDE-1.

Figure 2. Losses of the VAE model during training and validation with the ABIDE-1 dataset. The ratio of training and validation subjects was 7:3.

Figure 3. Left: Hierarchical clustering of the representations of latent dimensions (dim=30) in the image space. Right: the corresponding image representations of the latent dimensions. The model was trained on ABIDE-1.

Figure 4. Comparison of VAE models trained on different datasets. ABIDE+BS is the low-resolution ABIDE-1 and B-SNIP dataset, while REST1-LR, REST1-RL, REST2-LR, and REST2-RL are high-resolution HCP resting-state fMRI datasets. The first layer FC1 and the last layer FC4 were depicted. Each showed the weight matrix (upper row) and the weight distribution (lower row). Note that all models were initialized with the model trained on REST2_RL.

Figure 5. The differences in layer FC4 between low- and high- resolution models. Warm color regions affect the low-resolution model, while cold color regions affect the high-resolution model.

Proc. Intl. Soc. Mag. Reson. Med. 29 (2021)
3497