4570
3D Fetal Brain Segmentation using an Optimized Deep Learning Approach1Childrens National Hospital, Washington, DC, United States, 2University of Virginia, Charlottesville, VA, United States, 3Zhejiang University, Hanzhou, China
Synopsis
An essential step to accurately quantify fetal brain development is to reliably segment brain regions and perform volumetric measurements. However, this task mainly relies on labor intensive manually contouring. In this work, a 3D U-Net method was optimized and evaluated for fetal brain segmentation. 3D U-Net and 4D atlas-based segmentation methods were compared on 46 fetal brain MRI scans with gestational age 26.4 to 39.1 weeks. The proposed method resulted in (1) higher consistency with the manual segmentation, (2) shorter processing time, and (3) more consistent results across gestational ages, compared to a 4D atlas-based method.
INTRODUCTION
Accurate assessment of fetal brain anatomy in the second and third trimester of gestation is important for identifying early impairments in fetal brain growth (1). However, because of the active myelination process and water content changes (2), fetal brain images contain high signal variation within each tissue and show low image contrast (3). In addition, maternal respiration and irregular fetal movements often result in serious motion artifacts which corrupt MR images. Although temporal and spatial atlases have been developed for preterm infants and fetal brains (4), manual segmentation and correction are often still required.In recent years, deep convolutional neural networks have shown promising performance in the automatic processing of fetal MR images. Studies have focused on locating the interesting regions (e.g., fetal abdomen(5), the whole fetal envelope (6), and the body(7)) and on whole fetal brain extraction (8–10). Previously, we developed a 3D U-Net model for fetal brain MRI segmentation. Here, we optimized the method and compared its performance to a 4D atlas-based segmentation method on 46 healthy fetuses.
METHODS
46 MR scans were acquired on healthy fetuses from low-risk pregnancies. The mean gestational age (GA) at MRI was 33.3±4.0 weeks with a range of 26.4 - 39.1 weeks.T2-weighted images were acquired prospectively on a 1.5T GE scanner. High-resolution 2D images were acquired in coronal, sagittal and axial planes using single-shot fast-spin-echo sequences. The reconstruction matrix size was 256x256. TE was 157-163ms and the TR was about 1100ms. The three-plane images were reconstructed to a high-resolution 3D volume using the slice-to-volume registration method (11).
Ground truth was provided by the manually corrected segmentation. Fetal brain regions were segmented automatically by registering to a 4D atlas (4) using ANTS. Six brain regions were defined, including the cortical grey matter, the white matter, the cerebrospinal fluid the cerebellum, the deep gray matter and the brain stem. Each region was manually corrected by an experienced researcher (5 years’ experience) using ITK-SNAP.
A modified 3D U-Net was optimized. First, to avoid overfitting, early termination was considered by recording the model performance during 40 epochs of training and validation. Second, because fetal brains develop rapidly with GA, the fetal brain was resized to 80x110x90 according to the edge of the brain. Third, to generate realistic augmentation, the patches of fetal brain were` flipped in the left-right direction. The optimized U-Net was evaluated using 10-fold cross-validation on two P100 GPUs.
As a reference method, the images were segmented using Draw-EM (33) with 28-CPU parallel computing.
To provide quantitative evaluation, Dice score, 95% Hausdorff distance, sensitivity and specificity of each brain region were calculated. Wilcoxon signed-rank test was used to evaluate the performance statistically.
RESULTS
The proposed method provided the best performance with 20 epochs, left-right flip augmentation and resized image. First, the number of epochs was chosen by noting that the performance of the training continuously improved, but the cross-entropy and the dice score of the validation approached a stable stage after 20 epochs. Second, because the brain was left-right symmetric, the left-right flip generated the best validation performance. three-direction flip generated unrealistic data, which brought noise into the training process and in turn reduced the performance, as shown in Table 1. Third, because fetal brain size varies across GA, the image resize step provided a simple normalization which could reduce the variance between subjects and improve the performance.The proposed method (2.5 minutes) was about 9 times faster than the atlas-based method (22 minutes) to segment each fetal brain.
Fig. 1 and 2 show the results of the fetal brain at 28+2/7 and 35+2/7 weeks GA. The arrows on the axial/coronal images highlight the mislabeled deep grey matter/CSF and cortical grey matter regions in the atlas-based method. In contrast, the proposed method provided high consistency with the manual segmentation and resulted in more smooth and continuous segmentation in the cortical grey matter.
In the cross-validation, the proposed method provides more accurate segmentation than the atlas-based method, with an average Dice score of 0.901 vs 0.820 across the six brain regions (0.903 vs 0.821 in CSF, 0.84 vs 0.736 in the cortical grey matter, 0.906 vs 0,849 in the white matter, 0.9 vs 0.776 in the deep grey matter, 0.942 vs 0.885 in the cerebellum, and 0.913 vs 0.851 in the brain stem), Fig. 3.
The proposed method showed significant improvements compared to the atlas-based method in each brain region. The Dice score and 95% Hausdorff distance showed significant improvement (p<0.001). Improved specificity and sensitivity scores were found in cortical grey matter and white matter regions (p<0.05).
The proposed method showed consistent performance across GA. As shown in Fig. 4, the Dice score of each interested region maintained its superior performance across GA in the proposed method. In the cortical grey matter, the atlas-based method resulted in reduced accuracy around 35 weeks GA during which the secondary sulci develop.
DISCUSSION
In summary, our study demonstrated the superior performance of the 3D U-Net method for fetal brain segmentation, compared with an atlas-based segmentation method. Its high accuracy, efficiency, and consistency will provide useful quantification of in-utero regional fetal brain growth metrics to aid clinical diagnosis and large cohort studies.Acknowledgements
This work was partly supported by R01HL116585 from the NIH National Heart, Lung and Blood Institute, NIH R21EB022309 from the National Institute of Biomedical Imaging and Bioengineering, UL1TR001876 and KL2TR001877 from NIH National Center for Advancing Translation Sciences.References
1. Clouchoux C, Guizard N, Evans AC, Du Plessis AJ, Limperopoulos C. Normative fetal brain growth by quantitative in vivo magnetic resonance imaging. Am J Obstet Gynecol. Elsevier Inc.; 2012;206(2):173.e1-173.e8.
2. Xue H, Srinivasan L, Jiang S, et al. Automatic segmentation and reconstruction of the cortex from neonatal MRI. Neuroimage. 2007;38(3):461–477.
3. Despotović I, Goossens B, Philips W. MRI Segmentation of the Human Brain: Challenges, Methods, and Applications. Comput Math Methods Med. 2015;2015:1–23.
4. Gholipour A, Rollins CK, Velasco-Annis C, et al. A normative spatiotemporal MRI atlas of the fetal brain for automatic segmentation and analysis of early brain growth. Sci Rep. Springer US; 2017;7(1):1–13http://dx.doi.org/10.1038/s41598-017-00525-w.
5. Wu L, Xin Y, Li S, Wang T, Heng PA, Ni D. Cascaded Fully Convolutional Networks for automatic prenatal ultrasound image segmentation. Proc - Int Symp Biomed Imaging. 2017;663–666.
6. Joskowicz L, Event A, Leo AJ. Interactive fetal envelope segmentation in MRI scans by real-time fine-tuning of a Fully Convolutional Network. Isr Radiol Assoc Annu Meet. 2019.
7. Li Y, Xu R, Ohya J, Iwata H. Automatic fetal body and amniotic fluid segmentation from fetal ultrasound images by encoder-decoder network with inner layers. Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS. IEEE; 2017;1485–1488.
8. Rueckert D, Rajchl M, Kamnitsas K, et al. DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks. IEEE Trans Med Imaging. 2016;36(2):674–683.
9. B ME, B GW, Li W, et al. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013. 2013;8149:313–320
10. Ong YIR, Iang DEX, Hu WEZ, et al. Deriving external forces via convolutional neural networks for biomedical image segmentation. 2019;10(8):3800–3814.
11. Kainz B, Steinberger M, Wein W, et al. Fast Volume Reconstruction from Motion Corrupted Stacks of 2D Slices. IEEE Trans Med Imaging. 2015;34(9):1901–1913.
Figures
