4014
Brain-to-brain communication during eye contact: A data-driven approach based on the dfMRI1Zuckerman Institute, Columbia University, New York, NY, United States, 2Psychology, Columbia University, New York, NY, United States, 3Biomedical engineering, Columbia University, New York, NY, United States
Synopsis
A comprehensive picture of brain-to-brain interaction during eye contact is presented at the brain network level. It consists of five dyadic brain networks (two exogenous, two endogenous, and one exogenous-to-endogenous network), which are derived from a novel data-driven approach based on the dyadic fMRI data. Our experimental results not only explicitly map out the intricate relations between imitation, empathy, mentalization, and face cognization network during eye contact, but also provide a quantitative measure on the information transmission between dyads. Thus, it effectively established the foundation for quantifying human social communication with dyadic fMRI.
Introduction
Although there have been many fMRI studies on the brain responses to eye contact1,2, most of them cannot explicitly distinguish whether the dyadic brain synchrony is coming from mutual information transmission or individual social cognitive and affective responses. This is largely because the conventional fMRI methodology cannot capture sufficient live brain-to-brain interactions, even with MRI hyperscan3. However, such granular knowledge is essential to understand human social interactions, as well as psychiatric social behavior disorders such as Autism. This study provides an unprecedented dyadic fMRI (dfMRI) approach4,which is capable of quantitatively determining whether the synchrony between two brains is the result of dyadic information transmission or monadic social responses. For the first time, the intricate relations of eye movement, emotion, and mind interactions during eye contact began to reveal at the brain network level.Methods
The dfMRI experiment was conducted on a Siemens Skyra scanner with a custom-made dual-head coil system. A total of 17 dyads participated in the experiment. Each trial consists of three functional tasks: (i) dyads open and close their eyes simultaneously; (ii) dyads open and close their eyes alternatively; (iii) dyads have their eyes closed entirely. The dfMRI protocol and its data post-processing are detailed in Ref. 4. The highlights are: (a) the exogenous and endogenous dual systems were derived based on the data from both the task 1 and the task 2; (b) five independent components from ICA of the dyadic data of eye contact (task 1) were selected as five dyadic reciprocal brain networks (Fig. 1); (c) all the nodes in each network were projected to either exogenous or endogenous systems (Tab. 1).Information transmissions in the five bipartite networks were measured by Mutual Information (MI). For each bipartite network, the time series of all parcellates from each brain were orthogonalized by PCA, then the distance correlation between the two brains’ parcellates’ time series were calculated and form the probability density matrix ρAB for this bipartite network. The ρAB can be used to derive each brain’s density matrix, ρA =TrB(ρAB) and ρB =TrA(ρAB). The MI can be calculated from the entropies (H) of the bipartite system and its subsystems, MI = H(ρA) + H(ρB) - H(ρAB). The MI of all five bipartite networks for both eye contact (task 1) and eye closed (task 3) are listed in Tab. 2.
Results
Among the five dyadic networks, there are two exogenous networks (exo-net), two endogenous networks (endo-net), and one exogenous-to-endogenous network (exo-endo-net). By definition, the exo-nets handle exteroceptive stimuli between dyads; the exo-endo-net subserves interoceptive stimuli from the exogenous to the endogenous system within each individual; the endo-nets carry out reflective mental processes. The networks’ labels and coordinates are shown in Tab. 1.The MI of both eye contact and eye closed are listed in Tab. 2. The results suggest that only exo-nets have significant MI, and noticeably the transmission between dyadic exo-net 1 only occurs during eye contact, while the transmission between dyadic exo-net 2 happens when eyes close. There is no communication between dyads on their endo-nets and exo-endo-net. These are fully consistent with the exogenous-endogenous dual system model4.
Discussions
During eye contact, social information transmissions are carried out by the exogenous system. The exo-net 1 overlaps with the empathy network; the exo-net 2 overlaps with the imitation network (mirror neurons)5. Both are well-known to represent two major social communication channels. Tab. 2 suggests that the imitation process only occurs during eye contact, but the empathy process can be carried out without the need of eye contact.The exogenous and endogenous dual systems are mediated by the sensorimotor cortex (PRG and POG) and precuneus (PCN) in the exo-endo-net. The endo-net 1 overlaps with the mentalization (theory of mind) network; the endo-net 2 overlaps with the familiar face network. Tab. 2 suggests that even though these three networks are in synchrony between dyads during eye contact, they are largely individual activities that have no direct information exchange between dyads.
Given the limitations of the first generation dfMRI – limited FOV due to B0 inhomogeneity and large voxel size (4x4x4mm3), the network nodes identified here may not be the complete list for each network. However, they are significant enough to indicate the basic function of social networks. Meanwhile, there might be more networks to be discovered in the second generation dfMRI, which is currently under development.
Conclusions
A comprehensive picture of brain-to-brain interaction during eye contact is derived at the brain network level with a novel data-driven and deductive approach. It not only explicitly maps out the intricate relations between imitation, empathy, mentalization, and face cognization network, but also quantitatively measures the information transmission between dyads in all bipartite networks. Thus, it effectively established the foundation for quantifying human social communication with dfMRI.Acknowledgements
The authors thank for the funding support from NSF grant NCS-1926789.References
1. Hooker, C. I. et al. Brain networks for analyzing eye gaze. Cognitive Brain Res 17, 406-418, doi:Doi 10.1016/S0926-6410(03)00143-5 (2003).
2. Ethofer, T., Gschwind, M. & Vuilleumier, P. Processing social aspects of human gaze: A combined fMRI-DTI study. Neuroimage 55, 411-419, doi:Doi 10.1016/J.Neuroimage.2010.11.033 (2011).
3. Koike, T., Sumiya, M., Nakagawa, E., Okazaki, S. & Sadato, N. What Makes Eye Contact Special? Neural Substrates of On-Line Mutual Eye-Gaze: A Hyperscanning fMRI Study. eNeuro 6, doi:10.1523/ENEURO.0284-18.2019 (2019).
4. Lee, R. F. Dual Logic and Cerebral Coordinates for Reciprocal Interaction in Eye Contact. PLoS One 10, e0121791, doi:10.1371/journal.pone.0121791 (2015).
5. Iacoboni, M. Imitation, empathy, and mirror neurons. Annu Rev Psychol 60, 653-670, doi:10.1146/annurev.psych.60.110707.163604 (2009).
Figures
