0834

SuperSurfer: Cortical surface reconstruction using super-resolution anatomical MR images synthesized by deep learning
Qiyuan Tian1, Berkin Bilgic1,2, Qiuyun Fan1, Chanon Ngamsombat1, Akshay S Chaudhari3, Ned A Ohringer1, Yuxin Hu3, Thomas Witzel1, Kawin Setsompop1,2, Jonathan R Polimeni1,2, and Susie Y Huang1,2

1Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, United States, 2Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, United States, 3Radiological Sciences Laboratory, Department of Radiology, Stanford University, Stanford, CA, United States

Synopsis

Recent studies have shown that anatomical MR images with sub-millimeter resolution can improve the accuracy of cortical surface and thickness estimation compared to the standard 1-millimeter isotropic resolution. Here we propose a new method, entitled SuperSurfer, that synthesizes sub-millimeter anatomical MR images from standard 1-mm isotropic anatomical images using a convolutional neural network-based super-resolution approach intended for improved cortical surface reconstruction. We quantified the displacement of the reconstructed surfaces and difference in cortical thickness derived from the super-resolution and standard-resolution data and demonstrated that SuperSurfer provides improved cortical surfaces that are similar to those obtained from native sub-millimeter resolution data.

Target Audience

Neuroscientists, neuroradiologists, and neuroimaging researchers interested in using cortical surface reconstruction to analyze brain MRI data.

Introduction

Accurate and automated cortical surface reconstruction of the human cerebral cortex from anatomical MRI data is an invaluable tool for a variety of applications, including cortical thickness estimation, regional cortical area and volume quantification1, gray-white boundary-based image co-registration2, and surface-based analysis of functional and diffusion MRI signals3,4. Reconstruction methods used in tools such as FreeSurfer conventionally require 1-mm isotropic voxel size input images and generate cortical surfaces with sub-voxel precision and accuracy by accounting for partial volume effects at the cortical boundaries. Recent studies have shown that sub-mm voxel sizes can further improve the accuracy of the cortical surface and thickness estimation by reducing partial volume effects5,6. Despite the value of sub-mm cortical surface reconstruction, many existing brain anatomical datasets have been acquired at a standard 1-mm isotropic spatial resolution. The ability to leverage existing datasets for a more detailed understanding of human brain structure and function, while avoiding the long acquisition times required to obtain sub-mm anatomical MRI data, motivates the current work. We propose a new technique entitled SuperSurfer that reconstructs cortical surfaces using super-resolution sub-mm isotropic resolution anatomical MR images. In this work, we synthesize 0.7 mm isotropic T1-weighted (T1w) images from 1-mm isotropic T1w images using a convolutional neural network (CNN) and demonstrate that the super-resolution, sub-mm T1w images provide improved cortical surface reconstruction similar to results obtained from native 0.7-mm isotropic data.

Methods

Data. High-resolution (0.7-mm isotropic nominal) and B1 bias-field-corrected T1w images of 30 unrelated subjects (10 for evaluation, 20 for training) from the Human Connectome Project WU-Minn-Ox Consortium were used (https://www.humanconnectome.org/). The acquisition was performed on a customized 3T scanner (Siemens Skyra) using a T1w 3D MPRAGE sequence5. Low-resolution (1-mm isotropic) images were obtained by blurring/anti-aliasing then down-sampling the native high-resolution images with trilinear interpolation.

Data Formatting. The low-resolution T1w images were up-sampled to nominally-high-resolution using cubic spline interpolation, which served as the input of CNN. The native high-resolution images served as the output of CNN. Blocks of 32×32×32 voxel size were extracted (stride of 16) from all subjects (~40,000 blocks from 20 training subjects).

Network. A 3D very-deep CNN for super-resolution (VDSR)7,8 was used to learn the transformation from the up-sampled nominally-high-resolution block to the native high-resolution block on the same 0.7-mm isotropic voxel grid. VDSR was implemented using the Keras API (https://keras.io/) with a Tensorflow (https://www.tensorflow.org/) backend, L2 loss, ReLU activation, 3×3×3 kernels, 15 layers, 64 kernels at each layers. The training was performed on 15 subjects and validated on 5 using an NVidia V100 GPU for 20 epochs (~7 hours).

Evaluation. The learned VDSR parameters were applied to 10 new subjects not included in the training process. FreeSurfer reconstruction was performed on the native, nominal and synthesized high-resolution T1w images with the “hires” option6. The vertex-wise displacement of the estimated GM-WM and GM-CSF surfaces and the cortical thickness compared to the ground truth were computed. The displacement maps of the 10 subjects were co-registered to FreeSurfer’s “fsaverage” space and averaged.

Results

The synthesized high-resolution T1w images (Fig.1f) were visually comparable to the native high-resolution images (Fig.1a) and contained more textural detail compared to the low-resolution (Fig.1b) and up-sampled nominally-high-resolution images (Fig.1c) (enlarged regions in the primary visual cortex in Fig.2a). The synthesized T1w images had substantially lower difference (Figs.1d,g) and higher structural similarity indices than the up-sampled nominally-high-resolution images compared to ground truth.

Figures 2b,c demonstrate that the estimated GM-WM and GM-CSF surfaces reconstructed from the synthesized high-resolution T1w images (green contour) were highly similar to those reconstructed from ground truth (red contour) and markedly improved upon the low-resolution and up-sampled nominally-high-resolution images, especially the GM-WM surface.

The vertex-wise displacement of the estimated GM-WM and GM-CSF surfaces and cortical thickness between the synthesized high-resolution images and ground truth were substantially smaller than those between the up-sampled nominally-high-resolution images and ground truth (Fig.5, first row, absolute difference in mm: 0.032±0.059 vs. 0.13±0.12, 0.04±0.059 vs. 0.17±0.12, 0.036±0.035 vs. 0.19±0.11, respectively) at the single-subject level (Fig.3) and when averaged across 10 subjects (Fig.4).

Discussion

SuperSurfer is a simple approach to obtain more accurate cortical surface estimation from existing standard 1-mm isotropic T1w images using CNN-based super-resolution for GM-WM and GM-CSF contrast enhancement. SuperSurfer only uses the super-resolution images as an intermediate step for cortical surface reconstruction rather than as the ultimate result. Therefore, it does not suffer from issues regarding the interpretation of results directly from the CNN, with success gauged instead on the improvements seen in the surface positioning.

Acknowledgements

This work was supported by the NIH (grants P41-EB015896, S10-RR019307, K23-NS096056, R01-MH111419, U01EB026996) and an MGH Claflin Distinguished Scholar Award.

References

[1] Fischl B, Dale AM. Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proceedings of the National Academy of Sciences. 2000;97(20):11050-11055.

[2] Greve DN, Fischl B. Accurate and robust brain image alignment using boundary-based registration. NeuroImage. 2009;48(1):63-72.

[3] Polimeni JR, Fischl B, Greve DN, Wald LL. Laminar analysis of 7T BOLD using an imposed spatial activation pattern in human V1. NeuroImage. 2010;52(4):1334-1346.

[4] McNab JA, Polimeni JR, Wang R, et al. Surface based analysis of diffusion orientation for identifying architectonic domains in the in vivo human cortex. NeuroImage. 2013;69:87-100.

[5] Glasser MF, Sotiropoulos SN, Wilson JA, et al. The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage. 2013;80:105-124.

[6] Zaretskaya N, Fischl B, Reuter M, Renvall V, Polimeni JR. Advantages of cortical surface reconstruction using submillimeter 7 T MEMPRAGE. NeuroImage. 2018;165:11-26.

[7] Kim J, Kwon Lee J, Mu Lee K. Accurate image super-resolution using very deep convolutional networks. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016.

[8] Chaudhari AS, Fang Z, Kogan F, et al. Super‐resolution musculoskeletal MRI using deep learning. Magnetic resonance in medicine. 2018.

Figures

Figure 1. A representative axial slice from (a) ground-truth 0.7-mm isotropic T1w images, (b) down-sampled 1-mm isotropic T1w images, (c) up-sampled nominally 0.7-mm isotropic T1w images from down-sampled 1-mm isotropic images using spline interpolation and (f) synthesized 0.7-mm isotropic T1w images using a convolutional neural network (CNN). Maps of the (d, g) absolute difference and the (e, h) structural similarity index (SSIM) between the up-sampled and synthesized images versus the ground-truth images are displayed. A window of [0, 1100] is used for displaying T1w images and [0, 1] for the difference and SSIM maps.

Figure 2. (a) Enlarged views of axial slices from the ground-truth, up-sampled/nominal and convolutional neural network (CNN) synthesized 0.7-mm isotropic and down-sampled 1-mm isotropic T1w images in the primary visual cortex, with reconstructed gray matter-white matter surfaces and gray matter-cerebrospinal fluid (CSF) surfaces visualized as colored contours. (b, c) Reconstructed surfaces displayed in (a) are overlaid on the ground-truth 0.7-mm isotropic T1w images. Magenta arrowheads highlight locations where CNN synthesized high-resolution images provided improved cortical surface estimates.

Figure 3. Right-hemisphere vertex-wise displacement/difference of the gray matter (GM)-white matter (WM) surfaces (first row), GM-cerebrospinal fluid (CSF) surfaces (second row), and cortical thickness (third row) estimated from the synthesized and native high-resolution images and the up-sampled nominally-high-resolution and native high-resolution images of a representative subject. The arrows highlight the primary sensory cortex, where improvements in the synthesized high-resolution images matched improvements in high-resolution data as reported previously6.

Figure 4. Right-hemisphere vertex-wise displacement/difference of the gray matter (GM)-white matter (WM) surfaces (first row), GM-cerebrospinal fluid (CSF) surfaces (second row), and cortical thickness (third row) estimated from the synthesized and native high-resolution images and the up-sampled nominally-high-resolution and native high-resolution images averaged across 10 evaluation subjects in FreeSurfer’s fsaverage space. The arrows highlight the primary sensory cortex, where improvements in the synthesized high-resolution images match improvements in high-resolution data reported previously6.

Figure 5. Histograms of the vertex-wise displacement/difference of the gray matter (GM)-white matter (WM) surfaces, GM-cerebrospinal fluid (CSF) surfaces, and cortical thickness estimated from the synthesized and native high-resolution images (red curves) and the up-sampled nominally-high-resolution and native high-resolution images (blue curves) from across the whole hemisphere (first row) and in the primary visual cortex (second row).

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)
0834