2967
Pseudo-Label Assisted nnU-Net (PLAn) Enables Brain Segmentation at 7T1NINDS, National institutes of health, Bethesda, MD, United States
Synopsis
Keywords: Multiple Sclerosis, High-Field MRI, Transfer learning; Brain Segmentation; Lesion detection
Motivation: Brain segmentation is more challenging at 7T compared to 3T, primarily due to increased bias fields and other artifacts. Generating training data for 7T brain segmentation is tedious, making transfer learning based models a more feasible option.
Goal(s): Brain and lesion segmentation algorithm for use with 7T images in multiple sclerosis.
Approach: A 3T to-7T transfer learning algorithm (called PLAn) for skull stripping, lesion, and brain segmentation was trained and tested on participants clinically diagnosed with multiple sclerosis.
Results: In both quantitative and qualitative analysis, PLAn significantly outperformed other segmentation methods including nnU-Net in lesion and brain segmentation.
Impact: Brain volume is a commonly used marker of disease progression in various neurological and neuropsychiatric diseases; however it is more difficult to implement on 7T images. PLAn, a deep-learning algorithm, can produce fast and reliable whole-brain segmentations.
Introduction
Volumetric segmentation of brain magnetic resonance images (MRI) is now a commonly used postprocessing step enabling noninvasive quantitative analysis of disease progression in various neurological and neuropsychiatric diseases such as multiple sclerosis (MS), Alzheimer’s disease, human immunodeficiency viral infection, and depression.1-4 Compared to 3T, 7T MRI offers imaging at higher signal-to-noise ratio, enabling sub-millimeter resolution, and allowing for more sensitive analyses than were previously possible. However, automatic whole brain and lesion segmentation at 7T presents challenges, primarily from bias fields, susceptibility artifacts and distortions. Probabilistic atlas-based segmentation or multi-contrast segmentation methods developed for lower field strengths, such as FreeSurfer5 and Classification using DErivative-based Features (C-DEF),6 may be degraded in their effectiveness when applied to 7T MRI data.7 Therefore, there is a dearth of brain segmentation algorithms from 7T images compared to 3T and lower fields, especially ones that can efficiently segment tissue classes including any pathology in the whole brain. Here, we sought to use a novel 3T-to-7T transfer learning algorithm to do both skull stripping and whole brain and lesion segmentation performed on multiple imaging contrasts generated in a single MP2RAGE acquisition on participants clinically diagnosed with multiple sclerosis (MS).Methods
Retrospective analysis was performed on participants recruited into a natural history of MS study (NCT00001248) after written informed consent. 3T images, acquired on a Skyra system with a 32-channel head coil, included 1-mm isotropic resolution MP2RAGE, 3D FLAIR, and PD/T2 (at resolution of 0.7x0.7x3 mm). 7T images were acquired on a Magnetom system with 32-channel (Nova) head coil included MP2RAGE (two inversion time images and T1-maps at 0.7-mm isotropic resolution). Cohort 1 (25 participants) underwent 3T and 7T scans within 9 months of each other (no clinical disease progression or new lesions between scans) and were randomly split into a training/validation group (n=5) and a testing group (n=20). The C-DEF segmentation from 3T was used to create model labels6 (referred as pseudo-labels), and used as reference gold-standard for quantitative assessments because of its performance.8, 9 3T MP2RAGE scans were used to pre-train an nnU-Net model using the 3D full-resolution U-Net configuration. Transfer learning was achieved by loading the pre-trained model weights and preprocessing settings (except for the final softmax layer), then fine-tuning the network with the manually edited 7T labels (created by C.D, H.D, and validated by E.S.B or M.I.G) and 7T MP2RAGE scans from training group. Fine-tuning was conducted for 125 epochs with initial learning rate of 1 x 10-4 (default = 1 x 10-2). The fine-tuned model was then applied to the 7T MP2RAGE scans of the 8 unseen MS participants in Cohort 2 to obtain cross-validation results. For Cohort 2, manual lesion segmentations were drawn on the MP2RAGE T1w image using ITK-Snap (C.D) and validated by a neurologist (E.S.B, M.I.G). Segmentation results from FreeSurfer, C-DEF, nnU-Net, and PLAn were rated visually by experts in a blinded fashion and quantitatively using Dice Similarity Coefficient (DSC).Results
The nnU-Net and PLAn methods produced excellent skull stripping results throughout the brain (Fig. 1), with expert ratings for skull stripping having scores of 4.42 (CI: 4.33, 4.51) for nnU-Net and 4.4 (CI: 4.3, 4.5) for PLAn, respectively, compared to 4.03 (CI: 3.7, 4.2) for AFNI skull stripping. The D/L methods (nnU-Net and PLAn) significantly outperformed both C-DEF and FreeSurfer for all tissue segmentation classes in both qualitative (Fig. 2) and quantitative (Fig. 3) assessments. Despite its substantial advantages over other 7T segmentation methods explored herein, nnU-Net 7T still produced noticeable deficiencies in lesion detection (Fig. 2B, C). To address these deficiencies, we implemented the PLAn 7T method and PLAn's lesion DSC improved by 16% compared to nnU-Net. The significant improvement in PLAn 7 T's lesion sensitivity compared to nnU-Net 7T was validated in an unseen cohort (Cohort 2, Fig 4).Discussion and Conclusion
In both quantitative and qualitative analysis, PLAn and nnU-Net outperformed C-DEF and FreeSurfer in whole brain segmentation, and PLAn outperformed nnU-Net (and other methods) in lesion detection. Importantly, PLAn-7T only requires a 3T data for training, but not in routine application of the segmentation model. Limited availability of labeled data makes transfer learning an attractive option, and pre-training a nnU-Net model using readily obtained 3T pseudo-labels was shown to boost lesion detection capabilities at 7T. We are currently testing the performance of PLAn on MP2RAGE images acquired on the Philips 7T and evaluating the need for additional training or imaging contrasts (such as FLAIR performed on the 7T) to boost lesion detection capabilities.Acknowledgements
Study was funded by the Intramural Research Program at the NINDS.References
1. Traboulsee A, Simon JH, Stone L, et al. Revised Recommendations of the Consortium of MS Centers Task Force for a Standardized MRI Protocol and Clinical Guidelines for the Diagnosis and Follow-Up of Multiple Sclerosis. AJNR Am J Neuroradiol 2016;37:394-401.
2. Mina Y, Wu T, Hsieh HC, et al. Association of White Matter Hyperintensities With HIV Status and Vascular Risk Factors. Neurology 2021;96:e1823-e1834.
3. de Leeuw FE, Barkhof F, Scheltens P. Progression of cerebral white matter lesions in Alzheimer's disease: a new window for therapy? J Neurol Neurosurg Psychiatry 2005;76:1286-1288.
4. Espinoza Oyarce DA, Shaw ME, Alateeq K, Cherbuin N. Volumetric brain differences in clinical depression in association with anxiety: a systematic review with meta-analysis. J Psychiatry Neurosci 2020;45:406-429.
5. Fischl B, Salat DH, Busa E, et al. Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron 2002;33:341-355.
6. Selvaganesan K, Whitehead E, DeAlwis PM, et al. Robust, atlas-free, automatic segmentation of brain MRI in health and disease. Heliyon 2019;5.
7. Spini M, Choi S, Harrison DM. 7T MPFLAIR versus MP2RAGE for Quantifying Lesion Volume in Multiple Sclerosis. J Neuroimaging 2020;30:531-536.
8. Selvaganesan K, Whitehead E, DeAlwis PM, et al. Robust, atlas-free, automatic segmentation of brain MRI in health and disease. Heliyon 2019;5:e01226.
9. Dieckhaus H, Meijboom R, Okar S, et al. Logistic Regression-Based Model Is More Efficient Than U-Net Model for Reliable Whole Brain Magnetic Resonance Imaging Segmentation. Top Magn Reson Imaging 2022;31:31-39.
Figures
