4810
Real-time Ultrafast Fetal Brain Localization using Convolutional Neural Networks1Division of Diagnostic Imaging and Radiology, Children's National Health System, Washington, DC, United States
Synopsis
The advent of fetal magnetic resonance imaging has provided innovative approaches to study in-vivo brain development in the womb. One of the major challenges in the quantification of fetal brain growth and development is regional and tissue-specific segmentation. An automated brain localization algorithm can reduce the time and facilitate accurate segmentation. In this study, we propose an ultra-fast and robust method for fetal brain localization from SSFSE anatomical images using a minimally modified object localization algorithm called You Only Look Once (YOLO). YOLO provides not only the enhanced accuracy of brain localization by differentiating brain from maternal tissues but also fast computation time for brain detection compared to the other algorithms.
Introduction
Fetal magnetic resonance imaging (MRI) has given us an opportunity to quantify fetal brain growth precisely. Despite drastic technological advances, quantification of fetal MRI remains a labor-intensive and time-consuming process. Unlike the adult brain, fetal brain extraction is a unique and challenging task due to severe motion artifacts and other intra-uterine physiological artifacts. Despite the recent advent of convolutional neural network (CNN) models including U-Net4 and many template-based segmentation2, their performance is limited in differentiating maternal tissues from fetal brain tissues for a clinical use5. Estimating the rough location of the fetal brain before accurate segmentation can be beneficial not only to improve the segmentation accuracy but also to reduce the computation time. In this study, we aimed to test the feasibility of a CNN-based object detection algorithm called You Only Look Once1 (YOLO) version 2 to localize the fetal brain in real time.Methods
Single-shot fast spin echo (SSFSE) volumes of the fetal brain were acquired in three (axial, coronal and sagittal) acquisition directions on GE 1.5T MR scanner for analysis with TE/TR=1100/160ms, matrix size 256x256, and voxel size 1.25x1.25x2 mm3. The fetal brain masks were manually delineated on each slice using ITK-SNAP and automatically mapped into a bounding box surrounding the brain region. Each slice was used as an individual input to CNN, and divided into 8x8 grid cells to predict which grid the center of the fetal brain falls into. The object was described as a bounding box. Each grid cell was described with 6 elements consisting of coordinates (x, y), width, height and confidence score (C), and class probability (Pc). The network was organized as 22 convolutional layers with rectified linear unit (ReLU) activation and batch normalization, including five max-pooling layers. Especially, the 13th layer was concatenated to the 21st layer to improve detecting a small object. The last convolutional layer of 8x8x1024 was fed into a fully connected layer (see Figure 1). The network was trained using predefined ground truth to maximize the accuracy of bounding box and minimize the losses related to confidence score and class probability. The grid cells with C<0.65 were discarded, and the predicted box with maximum C was finally considered to detect the fetal brain when the intersection over union (IoU) between two predicted boxes was greater than 0.3. The finally chosen bounding box was converted to a binary mask of elliptical shape which is more suitable to cover the brain region, and this mask was dilated by 2 pixels. The performance of YOLO v2 was evaluated by comparing with the other advanced object detection algorithm called the R-CNN3.Results
The network was trained with 82 (82 x avg. 60 slice = 4980 images) fetal MR scans and validated with 28 scans (28 x avg. 60 slices = 1680 images). The network training took about 3 hours on an NVIDIA Tesla P100 12 GB GPU. As summarized in Table 1, the mean average precision was higher with YOLO compared to R-CNN in the testing dataset (76.7% vs 69.3%) and the overall mask accuracy on all slices was also with in YOLO (98% vs 87%). The computation speed was much faster with YOLO compared to R-CNN (0.06 vs 0.33 second/slice). Figure 2 illustrates an example of fetal brain localization on a test image.Discussion
Our results demonstrate that YOLO can be reliably used to localize the fetal brain from anatomical MR images with higher accuracy and faster computation compared to R-CNN. This brain localization algorithm needs to be validated with a larger cohort.Conclusion
We are the first to propose the YOLO-based method for fetal brain localization on in utero fetal brain MR images. The proposed method can be used to enhance super-resolution reconstruction of fetal brain volume by providing better resolution of fetal brain regions as a prior to a fully automated fetal brain segmentation pipeline. This method of localization could also be employed as the prior step in any segmentation pipeline for different fetal organs like heart, liver, and placenta.Acknowledgements
I would like to express my sincere gratitude to our Fetal & Neonatal MRI machine learning team members Yao Wu and Li Zhao. I also would like to thank my family and friends for their continuous support.References
[1] Redmon, Joseph, and Ali Farhadi. “YOLO9000: Better, Faster, Stronger.” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, doi:10.1109/cvpr.2017.690.
[2] Makropoulos, A., Counsell, S. J., & Rueckert, D. (2018). A review on automatic fetal and neonatal brain MRI segmentation. NeuroImage,170, 231-248. doi:10.1016/j.neuroimage.2017.06.074
[3] Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition. doi:10.1109/cvpr.2014.81
[4] Ronneberger, O. (2017). U-Net: Convolutional Networks for Biomedical Image Segmentation. Informatik Aktuell Bildverarbeitung Für Die Medizin 2017,3-3. doi:10.1007/978-3-662-54345-0_3
[5] Gholipour, A, et al. “Robust Super-Resolution Volume Reconstruction From Slice Acquisitions: Application to Fetal Brain MRI.” IEEE Transactions on Medical Imaging, vol. 29, no. 10, 2010, pp. 1739–1758., doi:10.1109/tmi.2010.2051680.
Figures
