1209
SegFormer for Precise Quantification of Lung Ventilation Defects in Hyperpolarized Gas Lung MRI1School of Biomedical Engineering, Faculty of Engineering, The University of Western Ontario, London, ON, Canada, 2Department of Physics and Astronomy, The University of Western Ontario, London, ON, Canada, 3Lawson Health Research Institute, London, ON, Canada
Synopsis
Keywords: Analysis/Processing, Hyperpolarized MR (Gas), Deep Learning; Magnetic Resonance Imaging (MRI); Hyperpolarized Gas MRI; Segmentation; Ventilation Defect; Chronic Obstructive Pulmonary Disease (COPD); Lung Imaging
Motivation: Current methods for quantifying lung ventilation defects using hyperpolarized gas MRI are effective but time-consuming. Deep Learning offers potential enhancements in image segmentation, with Vision Transformers (ViTs) emerging as notable alternatives to traditional CNNs.
Goal(s): The study aims to assess SegFormer's capability for automating the segmentation and quantification of ventilation defects in hyperpolarized gas MRI, comparing its efficiency and accuracy against traditional methods.
Approach: Utilizing a dataset from 56 study participants, the study adopted the SegFormer architecture for segmenting MRI slices.
Results: SegFormer, especially with ImageNet pretraining, surpassed CNN-based techniques in segmentation. Specifically, the MiT-B2 configuration of SegFormer showcased exceptional efficacy and efficiency.
Impact: SegFormer's efficiency in hyperpolarized gas MRI enhances future clinical decision-making with swift and precise segmentation. Its superiority may inspire broader adoption and further exploration into Vision Transformers' potential in medical imaging.
Introduction
Ventilation defects in the lungs (1), characterized by impaired airflow and reduced gas exchange, can arise from various factors such as small airway obstructions, mucus accumulation, and tissue damage(2). Hyperpolarized 129Xe/3He lung MRI is an efficient technique used to investigate and assess pulmonary diseases and ventilation defects (3). Current methods for quantifying these defects rely on semi-automated-techniques (4), involving hierarchical K-means clustering (5) for 3He MR images and seeded region-growing algorithms (6) for 1H MR images. Despite their effectiveness, these methods are time-consuming. Deep Learning (DL) has revolutionized medical imaging, particularly in image segmentation (7). While Convolutional Neural Networks (CNNs) like UNet (8) are currently the standard, Vision Transformers (ViTs) (9, 10) have emerged as a compelling alternative. ViTs have excelled in various computer vision tasks (11), owing to their multi-head self-attention mechanisms that capture long-range dependencies with less inductive bias. SegFormer (12), a specific ViT architecture, addresses some of the limitations of CNNs, such as low-resolution output and inability to capture long-range dependencies. It also introduces a Positional-Encoding-Free design, making it more robust.The purpose of this study is to explore the efficacy of SegFormer in the automatic segmentation and quantification of ventilation defects in hyperpolarized gas MRI. We aim to demonstrate that SegFormer not only outperforms semi-automated-techniques and CNN-based methods in accuracy but also significantly reduces the training time.
Methods
We collected data from 56 study participants, comprising 9 healthy individuals, 28 with COPD, 9 with asthma, and 10 with COVID-19. This resulted in 1456 2D slices segmented using MATLAB R2021b and the hierarchical K-means clustering method. The dataset was balanced, with an even distribution of data from each participant group across the training (80%), validation (10%), and testing (10%) sets. The code was implemented in PyTorch and executed on two parallel NVIDIA GA102, GeForce RTX 3090 GPUs. Proton and hyperpolarized slices were registered using a landmark-based image affine registration approach.In our research, we utilized the SegFormer architecture that incorporates hierarchical decoding for enriched feature representation, employs overlapping patches to enhance boundary recognition, uses an MLP-head for pixel-wise segmentation mask creation, and integrates a Bottleneck Transformer for reduced computational demands. Segformer utilizes both coarse and fine-grained features in lung MRI. While coarse features distinguish lung from non-lung tissues, fine-grained ones enable precise boundary identification and early disease feature detection, enhancing overall MRI interpretation.
Results
In this study, the efficacy of SegFormer was assessed through various Mix Transformer encoders (MiT), with MiT-B0 offering rapid inference and MiT-B2 targeting peak performance. Without pretraining, SegFormer registered a Dice Similarity Coefficient (DSC) of 0.96 generally, and 0.94 for hyperpolarized gas MRI within the training dataset. Remarkably, with ImageNet (13) pretraining, SegFormer surpassed CNN-based counterparts while requiring fewer computational resources.After ImageNet pretraining, MiT-B2 achieved a training DSC of 0.980 for Proton MRI and 0.974 for hyperpolarized gas MRI, with testing scores of 0.975 and 0.965, respectively, utilizing 24 million parameters. MiT-B0 recorded training DSCs of 0.973 (proton MRI) and 0.969 (hyperpolarized gas MRI), with test scores of 0.969 and 0.951. In contrast, the pretrained Unet++ with VGG 16 (14) backbone reported training DSCs of 0.964 and 0.953, and testing DSCs of 0.955 and 0.942, using 14 million parameters. The pretrained UNet with a ResNet 50 (15) backbone yielded training DSCs of 0.971 and 0.962, and test DSCs of 0.960 and 0.951, utilizing 23 million parameters.
These findings underscore SegFormer's excellence, especially the MiT-B2 configuration, in segmenting and quantifying ventilation defects in hyperpolarized gas MRI. The SegFormer's fewer learnable parameters also led to a reduced training time in contrast to the CNN-based models, without compromising on performance. DSC results are tabulated in Table 1. Case studies for proton MRI segmentation can be seen in Figure 1, with hyperpolarized gas MRI cases in Figure 2, and a VDP value comparison across select cases presented in Figure 3.
Discussion and Conclusion
Our study underscores the effectiveness of SegFormer in hyperpolarized gas MRI for segmenting and quantifying ventilation defects. SegFormer not only outperformed UNet and Unet ++ with various backbones in DSC but also excelled in training time efficiency. SegFormer's implicit understanding of spatial context, without traditional positional encodings, is particularly promising for medical imaging. However, our study is limited to a specific patient cohort, warranting further validation for broader applicability. In conclusion, SegFormer presents a transformative approach for efficient and precise quantification of ventilation defects in hyperpolarized gas MRI. Its superior performance in both accuracy and computational efficiency positions it as a promising tool for broader clinical applications in in hyperpolarized gas MRI.Acknowledgements
We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada, R5942A04.
References
References:
1. Altes TA, Powers PL, Knight-Scott J, et al.: Hyperpolarized 3He MR lung ventilation imaging in asthmatics: preliminary findings. J Magn Reson Imaging 2001; 13:378–384.
2. Harris RS, Fujii-Rios H, Winkler T, Musch G, Melo MFV, Venegas JG: Ventilation Defect Formation in Healthy and Asthma Subjects Is Determined by Lung Inflation. PLOS ONE 2012; 7:e53216.
3. Perron S, Ouriadov A: Hyperpolarized 129Xe MRI at low field: Current status and future directions. Journal of Magnetic Resonance 2023; 348:107387.
4. Kirby M, Heydarian M, Svenningsen S, et al.: Hyperpolarized 3He Magnetic Resonance Functional Imaging Semiautomated Segmentation. Academic Radiology 2012; 19:141–152.
5. MacQueen J: Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics. Volume 5.1. University of California Press; 1967:281–298.
6. Adams R, Bischof L: Seeded region growing. IEEE Transactions on Pattern Analysis and Machine Intelligence 1994; 16:641–647.
7. Malhotra P, Gupta S, Koundal D, Zaguia A, Enbeyle W: Deep Neural Networks for Medical Image Segmentation. Journal of Healthcare Engineering 2022; 2022:1–15.
8. Ronneberger O, Fischer P, Brox T: U-Net: Convolutional Networks for Biomedical Image Segmentation. 2015.
9. Dosovitskiy A, Beyer L, Kolesnikov A, et al.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. 2020.
10. Al-hammuri K, Gebali F, Kanan A, Chelvan IT: Vision transformer architecture and applications in digital health: a tutorial and survey. Vis Comput Ind Biomed Art 2023; 6:14.
11. Thisanke H, Deshan C, Chamith K, Seneviratne S, Vidanaarachchi R, Herath D: Semantic segmentation using Vision Transformers: A survey. Engineering Applications of Artificial Intelligence 2023; 126:106669.
12. Xie E, Wang W, Yu Z, Anandkumar A, Alvarez JM, Luo P: SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers. 2021.
13. Deng J, Dong W, Socher R, Li L-J, Kai Li, Li Fei-Fei: ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL: IEEE; 2009:248–255.
14. Simonyan K, Zisserman A: Very Deep Convolutional Networks for Large-Scale Image Recognition. 2015.
15. He K, Zhang X, Ren S, Sun J: Deep Residual Learning for Image Recognition. 2015.
Figures
Figure 1: Comparative visualization of lung segmentation in proton MRI using various methodologies. Each row showcases a different case, with corresponding Dice Similarity Coefficient (DSC) values provided. Ground truth masks and their overlap with predicted masks illustrate the segmentation accuracy of each method. The red areas in the overlapping masks indicate discrepancies between the predicted segmentation and the ground truth.
Figure 2: A side-by-side evaluation of lung segmentation methodologies applied to hyperpolarized (HP) gas MRI. Each row displays a distinct case, complemented by the Dice Similarity Coefficient (DSC) values. The presented ground truth and overlay of segmentation masks highlight the accuracy of each technique. Regions in red within the overlapping masks spotlight the areas where the predicted segmentations diverge from the ground truth.
Figure 3: Comparative analysis of the ventilated defect percentage (VDP) values derived from the ground truth (GT_VDP) and various segmentation methodologies. Proton MRI and hyperpolarized MRI columns provide the context, while the percentage values represent the VDP for each case. Regions in red in the segmentation masks highlight areas contributing to the VDP. The results show the difference in VDP calculations between the segmentation methods and the ground truth.
Table 1: Overview of segmentation models, their pretraining status, number of parameters, and performance metrics. The table displays the median Dice Similarity Coefficient (DSC) values for both training and testing phases on Proton MRI and hyperpolarized MRI datasets. The DSC values serve as a measure of segmentation accuracy for each model.