1023
Brain Tissue Segmentation robust to motion artifacts using Deformation-Aware Network1Department of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seoul, Korea, Republic of, 2Department of Artificial Intelligence, College of Software & Convergence Technology, Daeyang AI Center, Sejong University, Seoul, Korea, Republic of, 3Cognitive Science Research Group, Korea Brain Research Institute, Daegu, Korea, Republic of
Synopsis
Keywords: Segmentation, Segmentation, Brain tissue
Motivation: Motion artifacts in MRI scans present challenges by causing blurred images with tissue-like appearances, significantly complicating the tissue segmentation process.
Goal(s): Our goal is to achieve accurate brain tissue segmentation even in the presence of motion artifacts.
Approach: We propose a brain tissue segmentation method robust to motion artifacts, that generates a motion deformation map and a prediction mask for brain tissue segmentation. The motion deformation map serves as an indicator within the segmentation network, aiding in the identification of regions impacted by motion artifacts.
Results: Our method demonstrates superior performance compared to other segmentation models, especially when dealing with motion-corrupted data.
Impact: We propose a motion-robust segmentation network that incorporates prior motion knowledge via a motion estimation network. By employing a multi-task learning approach involving joint motion estimation and segmentation networks, we improve brain tissue segmentation by recovering incorrectly segmented structures.
Introduction
Accurate segmentation of brain tissue is essential for early infant brain development studies and quantitative tissue assessments.1 Deep learning-based segmentation networks perform well with clean input data but struggle with motion-corrupted data, commonly observed in clinical MRI scans due to their lengthy data acquisition duration (see Figure 1).2,3 Dealing with such artifacts in the volumetric data requires additional MRI scans, which incurs additional time, cost, and inconvenience for subjects. Even with slight motion, segmentation results can be erroneous. Therefore, enhancing segmentation accuracy on motion-corrupted input data is difficult yet crucial. In this regard, we propose a robust segmentation network for motion artifacts by leveraging the prior knowledge of motion obtained through a jointly stitched motion estimation network.Method
Data acquisitionWe utilized 3D T1-weighted brain MRI scans obtained from the Korea Brain Research Institute (KBRI) and publicly open dataset from OpenNeuro.4 KBRI dataset consists of 38 motion free and 3 motion-corrupted scans. OpenNeuro dataset includes three sets of data from 138 subjects: motion-free scans and two levels of motion-corrupted scans. Among these 163 subjects were used for training and validation, while an additional 16 subjects were used for evaluation.
To create motion corrupted datasets, we employed a motion simulator5, which enabled us to introduce different levels of 3D rigid motion artifacts. Motion level 1 and 2 were generated through sudden rotational movements within the ranges of [-5°, +5°] and [-9°, +9°] respectively, paired with corresponding translational movements between -5 and +5mm, -9 and +9mm. According to Blumenthal's motion rating6, motion level 1 can be classified as mild motion, while level 2 can be categorized as moderate motion.
Proposed Network
Our proposed network has been designed to provide tissue segmentation for brain MR data affected by motion artifacts. It consists of joint motion estimation and segmentation networks, as illustrated in Figure 2.
We introduced the motion estimation network as an additional network to identify and distinguish motion artifacts during the segmentation process. By employing the concept of warping, we obtain motion deformation maps that help in correcting motion-corrupted images. This approach not only provides an understanding of the magnitude of motion but also guides attention towards the affected areas.
To improve segmentation performance by leveraging information from the motion estimation network, we employed a Multi-Task Learning (MTL) approach. This allows the networks to share valuable information such as motion artifact magnitude values and relevant regions. During training, the segmentation network learns to generate the segmentation mask by utilizing the motion information, while the motion estimation network provides a motion deformation map. To enhance the effectiveness of MTL, we incorporate cross-stitch units. These units dynamically adjust the weights between feature maps from each network during training, determining their importance and enhancing the interrelation between the networks.7
To regularize the output of the motion estimation, we incorporate a Spatial Transformer Network (STN). The STN is cascaded with the motion estimation network to perform motion reduction.
Result
We evaluated the performance of our proposed joint motion estimation and segmentation network for brain tissue segmentation, comparing it to two other models: nnUNet8 and UNETR9. In Figure 3 and 4, we illustrate qualitative comparisons of the brain tissue segmentation results on the OpenNeuro and KBRI datasets, respectively. Figure 3 presents that our proposed network excels in accurately predicting the details of Cerebrospinal Fluid (CSF). It demonstrates promising results in brain tissue segmentation, achieving Dice scores of 68.83%, 78.47%, and 83.03% for CSF, Gray Matter (GM), and White Matter (WM), respectively. As illustrated in Figure 5, these values demonstrate an improvement of approximately 5% for each class compared to nnUNet. When compared to UNETR, we observed an enhancement of around 7% in the CSF and approximately 5% in GM and WM.Discussion & Conclusion
In this study, we proposed a motion-robust brain tissue segmentation network that effectively overcome the challenges posed by motion artifacts in the data. The proposed network leverages prior knowledge of motion artifact, resulting in more accurate and reliable segmentation results.The limitation of our approach is the requirement for a 2-channel concatenated input for the motion estimation network, which can be challenging to obtain in large quantities. Our proposed network resulted in accurate segmentation even for subjects affected by motion artifacts, particularly in pediatric datasets.
The in vivo experiments and quantitative scores have demonstrated the robustness of our algorithm on motion corrupted data. Our motion-robust brain tissue segmentation network has shown promising performance. By leveraging prior knowledge of motion, our network offers improved segmentation outcomes with motion corrupted data.
Acknowledgements
No acknowledgement found.References
1. Akkus, Zeynettin, et al. "Deep learning for brain MRI segmentation: state of the art and future directions." Journal of digital imaging 30 (2017): 449-459.
2. Chen, Zhaolin, et al. "Deep learning for image enhancement and correction in magnetic resonance imaging—state-of-the-art and challenges." Journal of Digital Imaging 36.1 (2023): 204-230.
3. Choi, Yoonseok, et al. "A Single Stage Knowledge Distillation Network for Brain Tumor Segmentation on Limited MR Image Modalities." Computer Methods and Programs in Biomedicine (2023): 107644.
4. Ádám Nárai and Petra Hermann and Tibor Auer and Péter Kemenczky and János Szalma and István Homolya and Eszter Somogyi and Pál Vakli and Béla Weiss and Zoltán Vidnyánszky (2022). Movement-related artefacts (MR-ART) dataset. OpenNeuro. [Dataset] doi: doi:10.18112/openneuro.ds004173.v1.0.2
5. Lee, Seul, et al. "Deep learning in MR motion correction: a brief review and a new motion simulation tool (view2Dmotion)." Investigative Magnetic Resonance Imaging 24.4 (2020): 196-206.
6. Blumenthal, Jonathan D., et al. "Motion artifact in magnetic resonance imaging: implications for automated analysis." Neuroimage 16.1 (2002): 89-92.
7. Elmahdy, Mohamed S., et al. "Joint registration and segmentation via multi-task learning for adaptive radiotherapy of prostate cancer." IEEE Access 9 (2021): 95551-95568.
8. Isensee, Fabian, et al. "nnu-net: Self-adapting framework for u-net-based medical image segmentation." arXiv preprint arXiv:1809.10486 (2018).
9. Hatamizadeh, Ali, et al. "Unetr: Transformers for 3d medical image segmentation." Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2022.
Figures
Figure 1. Results of FreeSurfer masks on motion-clean and motion-corrupted data at two levels. Motion artifacts induce blurring and misstructured tissues, resulting in inaccurate segmentation mask results.
Figure 3. Qualitative comparison of the proposed motion estimation and segmentation networks against the nnUNet and UNETR methods. All images depict the identical slice for each subject. Tested on the OpenNeuro dataset, which includes in-vivo motion artifacts. The numbers below each segmentation prediction mask represent the average Dice scores of Cerebrospinal Fluid (CSF), Gray Matter (GM), White Matter (WM) for each subject.
Figure 4. Qualitative comparison of the proposed motion estimation and segmentation networks against the nnUNet and UNETR methods. Each row of the image, from top to bottom, sequentially represents the coronal view of subject1, the sagittal view of subject2, and the axial view of subject3. Tested on the KBRI dataset, which consists of pediatric data with simulated motion artifacts. The numbers below each segmentation prediction mask represent the average Dice scores of CSF, gray matter, and white matter for each subject.
