4550

Hybrid EEG and fMRI platform for multi-modal neurofeedback
Marsel Mano1,2, Elise Bannier2,3, Lorraine Perronnet1,2, Anatole Lécuyer1, and Christian Barillot2,4,5,6

1Hybrid Team, Inria, Rennes, France, 2VisAGeS Project-Team, Inria, Rennes, France, 3Service de Radiologie, CHU Pontchaillou, Rennes, France, 4VisAGeS U746, Inserm, Rennes, France, 5UMR CNRS 6074, IRISA, Rennes, France, 6University of Rennes, Rennes, France

Synopsis

Neurofeedback (NFB) relies on neurosignals for the estimation of brain activity. There exist a wide variety of NFB applications that use one type of neurosignals like fMRI or electroencephalography (EEG). Recently, the combination of two or more neurosignals has been receiving a lot of attention in the research community, but still very few multi-modal NFB applications exist. This is primarily because of the lack of commercial multi-modal NFB systems and the associated technical difficulties in building them.

Here we are going to describe a bi-modal EEG and fMRI NFB platform that we have build in our lab. Our platform is designed to maximize modularity and parallel processing in order to be able to provide real-time NFB with high level of synchronization and minimal delays. We have successfully used our platform to conduct over 100 uni-modal and bi-modal NFB experiments with more than 30 healthy subjects.

Purpose

Uni-modal neurofeedback (NFB) applications are based on the brain activity estimated by only one neurosignals measurement technology (i.e. electroencephalography (EEG) or fMRI), whereas multi-modal applications are based on information acquired with more than one measurement technology. There exist a variety of uni-modal NFB research applications1,2, but very few multi-modal ones3. This is primarily because of the lack of commercial systems that perform multi-modal NFB and their associated technical burdens4,5,6.

In this abstract we describe the architecture and the components of our bi-modal (EEG and fMRI) NFB platform, at Neurinfo*. The platform has a modular parallel processing oriented architecture that promotes high real-time performance and allows for easy future addition and/or replacement of its processing modules. Currently, the platform is able to provide NFB based on EEG and/or fMRI, but its architecture and design principles are valid for any combination of two or more real-time brain activity measurement technologies.

*. http://incr.fr/recherche/57-unite-visages-u746-inserm-irisa/98-la-plateforme-neurinfo

Platform description

Our platform is based on the integration and the synchronization of an MR-compatible EEG and an fMRI acquisition subsystems (Figure 1).

The EEG signals, acquired with a 64-channel solution from Brain Products, are send via USB to Recorder and then forwarded to RecView where the MR gradient artifacts7 and the balistocardiogram artifacts are removed8. The EEG subsystem uses a synchronization module to achieve EEG-fMRI phase synchronization, necessary for the removal of MR artifacts and an electrocardiogram (ECG) channel for recording the heart activity. Next, the EEG data is sent via TCP/IP to the EEG object. The EEG object has an EEG real-time processing pipeline (Figure 2a) that can pre-process, filter, extract features and estimate EEG-NFB in real-time.

Our fMRI subsystem is a Nordic-Neurolab fMRI solution under a Siemens 3T MR scanner. The imaging on the MR scanner (Magnetom 3T Verio, Siemens Healthineers, Erlangen, Germany, VB17) is performed with a 12-ch head coil allowing secure installation of the EEG cap and connection of the bundle to the amplifiers. The fMRI data is sent to the fMRI object in the NFB Unit using a TCP/IP client/server buffer solution. Similar to the EEG object, the fMRI object has also a real-time fMRI pipeline (Figure 2b) that can pre-process, filter, extract features and estimate fMRI-NFB.

The outputs of both pipelines are sent to the Joint NFB. Joint NFB does the fusion (i.e. averaging or joint modeling) of both NFBs, and then forwards its results to Visualize (Figure 3). Visualize controls the display that communicates with the subject. It has a collection of visual objects used for explaining the NFB tasks (i.e. texts) and for animating the NF representation (i.e. 2D/3D objects). The stimulation and the NFB are displayed on the LCD display (Figure 1).

The NFB Control is responsible for the full synchronization of both modalities. Furthermore, it contains the experiment protocol (i.e. types of tasks, durations, repetitions), it starts/stops the experiment, controls all the other objects' behavior throughout the experiments, and stores the data at the end of each session.

NFB experiments and real-time performance

Our platform has been successfully used to conduct several uni-modal and bi-modal NFB studies. The goal of the studied NFB protocols has been to maximize the brain activity measured by EEG and fMRI while performing motor imagery related mental tasks. All experiments were preceded by a preparation phase needed to install the EEG cap and prepare the subjects for the NFB experiments (Figure 4a, b & c). Next, a calibration session (without NFB) was recorded and the data was processed offline to extract initialization information and define the NFB targets. After calibration, the subjects conducted several NFB sessions, where they showed to be able to modify their brain activity by performing the tasks and controlling the feedback presented on the screen (Figure 5).

Our platform has shown very good real-time performance with various pre-processing, filtering, NFB calculation and visualization methods. The entire fMRI process from acquisition to NFB calculation takes around 150ms, well below the TR of regular EPI sequences (1000ms or 2000ms). The NFB visualization is very fast (1-2ms) and it is done within one screen refresh (i.e. 16,7ms for a screen with 60Hz refresh rate).

Conclusion

The platform introduced here offers a reliable environment for bi-modal EEG-fMRI NFB experiments. Its modular architecture is generic enough to be adapted to other similar neurosignals acquisition subsystems, and also easily adaptable to different experimental environments and/or NFB protocols. The parallel real-time processing pipelines have shown to be an optimal solution for real-time NFB applications.

Acknowledgements

No acknowledgement found.

References

1. Weiskopf N, Scharnowski F, Veit R, et al. Self-regulation of local brain activity using real-time fMRI. Journal of Physiology-Paris. 2004;98(4):357-373.

2. Zich C, Debener S, Kranczioch C, et al. Real-time EEG feedback during simultaneous EEG–fMRI identifies the cortical signature of motor imagery. NeuroImage. 2015;114:438-447.

3. Zotev V, Phillips R, Yuan H, et al. Self-regulation of human brain activity using simultaneous real-time fMRI and EEG neurofeedback. NeuroImage. 2014;85:985-995.

4. Sulzer J, Haller S, Scharnowski F, et al. Real-time fMRI neurofeedback: progress and challenges. Neuroimage. 2013;76:386-399.

5. Birbaumer N, Ruiz S and Sitar R. Learned regulation of brain. Trends in cognitive sciences. 2013:17(6):295-302.

6. Stoeckel L E, Garrison K A, Ghosh S S, et al. Optimizing real time fMRI neurofeedback for therapeutic discovery and development. NeuroImage: Clinical 2014;5:245-255.

7. Allen P J, Josephs O and Turner R. A method for removing imaging artifact from continuous EEG recorded during functional MRI. Neuroimage 2000;12(2):230-239.

8. Debener S, Mullinger K J, Niazy R K and Bowtell R W. Properties of the ballistocardiogram artefact as revealed by EEG recordings at 1.5, 3 and 7 T static magnetic field strength. Int. Journal of Psychophysiology 2008;67(3):189-199

Figures

Figure 1. The detailed diagram of the platform. The EEG (purple) and fMRI (orange) signal flow including their respective acquisition subsystems and the different modules inside the NFB Unit. Both signals are first merged into the Joint NFB to then later terminate to Visualize. Recorder and RecView are the only commercial software in the NFB Unit. The NFB Control exchanges synchronization and control information with the rest of the hardware and software components.

Figure 2. The real-time processing pipelines for: a) EEG and b) fMRI. For each pipeline there exist inputs that can be obtained offline (calibration or a priori) or online (updates). These inputs are then used throughout the different processing steps to assist the NFB estimation process. Some examples of methods that can be applied at each step of the respective pipelines are shown inside the purple boxes.

Figure 3. The hybrid platform is able to provide both uni-modal and bi-modal NFB and the visualization is flexible following the results of the calculation. In this visualization example, the NFB goal is to put the disk inside the small 'green' square. The disk position inside the big square is determined by the normalized NFB values coming from the brain activity measured with EEG and fMRI.

Figure 4. Different phases of the system preparation: a) Installation and preparation of EEG subsystem, b) ECG electrode mounting, c) Re-testing the EEG acquisition after fMRI coil installation and before the acquisition sessions start.

Figure 5. Snapshot of a subject during an experiment. The subject is trying to move the orange disk toward the darkest blue colored area using its brain activity generated by performing motor imagery tasks and measured in real time using EEG and fMRI.

Proc. Intl. Soc. Mag. Reson. Med. 25 (2017)
4550