Introduction

DSI is a useful magnetic resonance imaging (MRI) technique that can be used to characterize the three-dimensional (3D) diffusional displacement of water molecules in a model-free approach.^1,2 3D fiber tracts may be generated using DSI or other diffusion MRI methods. The resulting fiber tracts are typically presented on a two-dimensional (2D) display that cannot effectively show depth information. Augmented Reality (AR) is an exciting technology that may help to overcome these limitations by allowing the user to visualize 3D data in the real world. The purpose of this study was 1) to create a workflow to visualize fiber tracts derived from DSI in the Microsoft HoloLens, an AR headset capable of rendering 3D holograms in the real world and 2) to perform an initial evaluation of the added value of visualizing the 3D fiber tracts in AR to the clinical workflow.

Methods

DSI was performed in one healthy brain on a clinical 3T scanner (Skyra, Siemens, Erlangen) with a 32-channel head coil using a radially symmetric q-space sampling scheme (RDSI).³ A single-shot twice refocused spin echo sequence⁴ was used with the following sequence parameters: b_max -4000s/mm², TE/TR=114/2600ms, 2.2mm³ resolution, field of view 220 x200mm, 60 slices, parallel imaging (2x GRAPPA) and multiband acceleration of two⁵, one b₀ acquisition, one average, 10:49min. Fiber tracts were generated with a modified streamline tracking algorithm as implemented in DSI Studio (http://dsi-studio.labsolver.org, 1000 tracts, uniformly distributed seed points). A 3D surface mesh with embedded color maps was generated using a custom made script (Python) in Blender (http://www.blender.org). The mesh was then exported in .obj format so that it could be imported directly into the Unity game engine, version 5.4 (Unity Technologies, San Francisco, CA, USA). The model was finally deployed into the Microsoft HoloLens headset for viewing in AR. An outline of the process used to produce the virtual fiber tracts is shown in Figure 1. Fiber tracts were evaluated by two experienced radiologists and three magnetic resonance physicists with expertise in diffusion imaging. Questions regarding which method of viewing was preferred and whether the HoloLens model would be helpful for surgical planning were asked.

Results

Our developments have permitted stereoscopic 3D exploration of white matter fiber tracts in AR using the Microsoft HoloLens (Figure 2). The AR model permitted a good visual match to the original 3D surface mesh (Figure 3). A short video of the AR fiber tracts is shown in Figure 4. For the model evaluation, four out of five (80%) of the reviewers preferred the AR model over the computer model. In addition, all believed that the AR model would be helpful for surgical planning due to the advanced 3D spatial visualization and ability to view from different angles.

Discussion

We have developed a method to visualize DSI tractography in AR using the Microsoft HoloLens. Our results suggest that visualization of white matter fiber tracts in AR facilitates navigation through the complex fiber tract models, thus helping to gain better insight in the connectivity in the brain.

Conclusion

This study demonstrates that it is feasible to create 3D models of fiber tracts that can be implemented and viewed in AR. Visualization in AR provides physicians and surgeons with a more realistic depiction of the anatomy and may have great impact in surgical planning and navigation.

Acknowledgements

This work was supported by the Center for Advanced Imaging Innovation and Research (www.cai2r.net), a NIBIB Biomedical Technology Resource Center (NIH P41 EB017183).