Yao Wu1, Kushal Kapse1, Christina Elizabeth Mastracchio1, Hironori Teramoto1, Stephanie Araki1, Patricia Saulino1, Merrick Lynne Kasper1, Nickie Niforatos Andescavage1, Gilbert Vezina1, and Catherine Limperopoulos1
1Children's National Hospital, Washington, DC, United States
Synopsis
Keywords: Analysis/Processing, Segmentation, Fetal hippocampus; Convolution neural networks
Motivation: The ability to accurately segment the fetal hippocampus is critical to advancing our understanding of the origins of prenatal memory and emotional processing difficulties. Current manual methods are laborious and subjective.
Goal(s): We aim to automate left and right fetal hippocampal segmentation in 3D MR images.
Approach: We applied a 3D U-Net based model to automatically segment the left and right fetal hippocampus in 3D MR images.
Results: Our dataset comprised 131 fetuses with 191 MRI scans. The results demonstrated high accuracy and efficiency, particularly for this challenging-to-segment structure, illuminating the potential of deep convolutional neural networks in this application.
Impact: This study's automatic
fetal hippocampal segmentation with deep learning has the potential to advance
in utero brain development research and biomarker studies. The potential impact
includes improving early diagnostics, in-utero neuro-surveillance, and future
targeted therapeutics.
INTRODUCTION
The
hippocampus is a crucial structure in the brain associated with memory,
emotion, and spatial navigation. Understanding the role of the hippocampus in
early brain development and later neuropsychiatric disorders is vital. Fetal
MRI provides a unique opportunity to investigate this structure in utero.
However, accurately segmenting the in utero fetal hippocampus in MR images
poses a significant challenge due to its small size, similarity in intensity to
adjacent structures (e.g., amygdala, parahippocampal gyrus, choroid plexus), fetal
motion, and maternal respiration. Manual segmentation, the current standard of
evaluation, is labor-intensive, time-consuming, and subjective, making the
development of automated segmentation methods essential.
Currently,
no fully automatic method exists for fetal hippocampus segmentation in MR
images. Advanced deep learning models have demonstrated promise in medical
image analysis. Particularly, U-Net has showed its potential in yielding
promising results in medical image segmentation [1]. In this study, we evaluated
a U-Net-based model for the automated segmentation of the left and right fetal
hippocampus in 3D MR images.METHODS
Fetal brain
T2-weighted MRI was performed using a GE 1.5T scanner and an 8-channel receiver
coil. The scanning protocol included multiplanar multiphase, single-shot
fast-spin echo acquisitions (repetition time: 1100 ms; echo time: 160 ms; flip
angle: 90°; field-of-view: 32 cm; matrix: 256×192; in-plane resolution:
1.25×1.66 mm2; slice thickness: 2 mm). Participants were
free-breathing during the MRI scanning, and the acquisition time was 2 to 3
minutes for each of the axial, sagittal, and coronal planes. All brains were structurally
normal on conventional MRI.
After image
acquisition, motion-corrupted stacks of 2-dimensional slices from all 3 planes
(i.e., coronal, sagittal, and axial) were reconstructed into a high-resolution
3-dimensional image (Figure 1). In this step, the fetal brain region in MRI slices
was automatically detected using You Only Look Once (YOLO) [2], a deep convolutional
neural network designed for object detection. After fetal brain detection, a
parallel slice-to-volume reconstruction method with evaluated point-spread functions
was applied to remove motion and reconstruct the fetal brain slices into a 3D image
[3]. The reconstructed fetal brain was rigidly registered to an in-house developed
fetal brain atlas from 18 to 37 gestational weeks for reorientation using FSL FLIRT
(FMRIB's Linear Image Registration Tool) in FSL [4]. After fetal brain reconstruction
and reorientation, the image with 0.86×0.86×0.86 mm3 resolution was
used in the following process.
Manual
segmentation of the left and right fetal hippocampus was used as the ground
truth for model training and evaluation. Automatic segmentation of the fetal hippocampus
(Figure 2) was performed with a modified model based on 3D U-Net [5], which contained
3 encoding layers and 3 decoding layers, with the repeated application of 3×3×3
convolutions, batch normalization, and parametric rectified linear unit (PReLU),
followed by 2×2×2 max pooling/transposed convolution operations with stride 2 for
downsampling/upsampling feature maps. The Adam optimizer was used with a
learning rate of 1e-4. Cross entropy was used as the loss function. The image of
size 117×159×126 was split into patches of size 64×64×64 with stride 2×2×2 and fed
into the model. The image patches were fed to 96 filters in the first layer of
the U-Net structure. The model was trained for 2000 epochs, 50 steps per epoch
with a batch size of 4.
RESULTS
The
training dataset consisted of 79 subjects with 111 scans at 23-40 weeks of
gestational age, with 20% selected for model validation. The testing data
consisted of 52 subjects with 80 scans ranging from 26 to 39 gestational weeks.
The running time for one testing image ranged from 10-15 seconds. The Dice coefficients
between the automatic segmentation and manual segmentation were 83.7% and 85.2%
for left and right hippocampus, respectively.DISCUSSION
To the best of our knowledge, this is the first fully automatic method
for in utero fetal hippocampal segmentation in MR images. The use of deep convolution
neural networks, comprehensive dataset, high accuracy, and rapid processing
time provide valuable insights. Future investigations will involve comparative
assessments of the 3D U-Net with alternative automatic segmentation methods.CONCLUSION
This study presents the validation of an automatic fetal hippocampal
segmentation method, showcasing high precision and efficiency in analyzing in
utero hippocampal structure. Our results highlight the potential of deep
learning techniques for the advanced understanding of early brain development,
and the potential to identify prenatal imaging biomarkers for later childhood
disorders.Acknowledgements
This work was partly supported by National Institutes of Health-R01HL116585, A. James & Alice B. Clark Foundation, Brain & Behavior Research Foundation-28218, and Thrasher Research Fund-14764.References
1. Ronneberger
O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image
segmentation. Medical Image Computing and Computer-Assisted Intervention. 2015;pp.
234-241.
2. Redmon
J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time
object detection. Proceedings of the IEEE conference on computer vision and
pattern recognition 2016;pp. 779-788.
3. Kainz
B, Steinberger M, Wein W, et al. Fast volume reconstruction from motion
corrupted stacks of 2D slices. IEEE Trans Med Imaging. 2015;34(9):1901-1913.
4. Jenkinson
M, Bannister P, Brady M, Smith S. Improved optimization for the robust and
accurate linear registration and motion correction of brain images. Neuroimage.
2002;17(2):825-41.
5. Zhao
L, Asis-Cruz JD, Feng X, Wu Y, Kapse K, Largent A, Quistorff J, Lopez C, Wu D,
Qing K, Meyer C. Automated 3D fetal brain segmentation using an optimized deep
learning approach. American Journal of Neuroradiology. 2022;43(3):448-54.