2095

Deep Learning Myocardial Segmentation in 3D Whole-Heart Joint T1/T2 mapping: Comparison of nnU-Net and MA-SAM

Carlota Gladys Rivera¹, Carlos Velasco ², Alina Hua², René M. Botnar^1,2,3,4,5, and Claudia Prieto^1,2,4
¹IMPACT, Center of Interventional Medicine for Precision and Advanced Cellular Therapy, Santiago, Chile, ²School of Biomedical Engineering, King’s College London, London, United Kingdom, ³Millennium Institute iHEALTH, Santiago, Chile, ⁴School of Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile, ⁵Institute for Biological and Medical Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile

Synopsis

Keywords: Analysis/Processing, Segmentation, 3D mapping, joint T1/T2

Motivation: The significant amount of data collected from a single 3D whole-heart joint T1/T2 mapping sequence substantially increases the time required for segmenting and analyzing the quantitative maps, therefore, automating the segmentation process could result in a significant reduction.

Goal(s): To automate the segmentation of myocardium using state-of-the-art segmentation networks.

Approach: Two segmentation networks, nnUNet and MA-SAM, are trained and compared for myocardial segmentation of whole-heart joint T1/T2 mapping in healthy subjects and patients.

Results: nnUNET and MA-SAM achieved good quality segmentations with DICE score higher than ~0.856 with smoothed masks. nnU-Net achieved better results in term DICE and required the shortest training time.

Impact: State-of-the-art nnUNET and MA-SAM networks achieve accurate automatic myocardial segmentation of whole-heart joint T1/T2 mapping. This can significantly reduce the laborious task of manual segmentation and could help to accelerate the analysis and therefore the diagnosis of myocardium-related disease.

Introduction

Myocardial tissue characterization plays a crucial role in the detection of a wide range of cardiovascular (CVD) diseases^1,2. Traditionally, 2D quantitative maps of a single parameter (such as T1 and T2) are acquired under separate breath-holds for different short-axis slices, requiring the acquisition of multiple images³. Recently, the feasibility of producing co-registered 3D whole-heart simultaneous joint T1 and T2 from a single free acquisition has been demonstrated⁴. This approach promises a comprehensive assessment of myocardial tissue, simplifying the scan planning. Nonetheless, the analysis requires manual volumetric myocardial segmentation of both T1 and T2 maps, which is time-consuming and prone to errors. Recently, several studies have shown the good performance 2D automatic deep learning segmentations for 2D cardiac cine and 2D single-parametric mapping, however evaluation of automatic segmentation for 3D multiparametric joint T1/T2 mapping is scarce. In this work, we propose to compare two state-of-the-art segmentation networks, nnUNET⁵ and Model Agnostic Segment Anything Model (MA-SAM)⁶, for left ventricle (LV) myocardial segmentation of whole-heart joint T1/T2 mapping. Comparison was performed in terms of Dice score in data from healthy subjects and patients.

Methods

Segmentation networks: two state-of-the-art segmentation networks, nnUNet and MA-SAM, are trained and compared for LV myocardial segmentation of whole-heart joint T1/T2 mapping. nnUNet is a UNet based segmentation method that automatically configures itself, from preprocessing to predictions⁵. MA-SAM is a foundation model for general image segmentation based on transformers⁶ . A schematic representation of the proposed approach is shown in Figure 1.
3D joint T1/T2 mapping acquisition: 3D whole-heart joint T1/T2 maps (1x1x2 mm3) were acquired with a recently proposed image-navigator based approach⁴, which consists of the simultaneous acquisition of 4 interleaves volumes (the 4th with highest myocardial-blood contrast, T2 weighted). Data was acquired on a 1.5T scanner (MAGNETOM Aera, Siemens Healthcare, Erlangen, Germany) on N = 60 (10 healthy subjects and 50 patients with suspected CVD). Manual segmentations were performed slice-by-slice in the short-axis orientation by an experienced clinical reader on the 4th interleaved volume. Since the 3D T1 and T2 maps are co-registered a single mask is needed. The annotated masks were morphologically smoothed by a series of 3D morphological operations to ensure connectivity between adjacent slices. Examples of the manual segmentations and its corresponding smooth version are shown in Figure 3.
Training: Data from 54 subjects was used for training and 6 for testing. nnUNet was trained on 3D and MA-SAM on 2D slice-by-slice due to network implementation limitations. Both networks were trained with the 4th interleaved volume (with highest myocardial-blood contrast) data for 100 epochs. Training was performed both with smoothed and unsmoothed masks. Training took 4 hours on 1x NVIDIA Quadro RTX 4000 and 11 hours on 2x NVIDIA RTX A6000 for nnUNET and MA-SAM respectively.
Evaluation: Both networks were compared in terms of Dice metric for the 6 subjects in testing. Networks were also compared in terms of computational requirements and training and inference times.

Results

Segmentations mask results for nnUNET and MA-SAM are shown overlaid on the T1 and T2 maps for a representative subject, in Figures 3 and 4. Results are shown at the short-axis mid-ventricular level, as well as in the long-axis 2-chamber and 4-chamber views for all four training sets (nnUNET, nnUNET smooth mask, MA-SAM and MA-SAM smooth mask). Good visual segmentation results are observed in all cases, with improved results for the smooth versions. DICE metric results for the test data are shown in Figure 5. nnUNet outperforms MA-SAM in terms of DICE scores. It is also noticeable that the smoothed masks yield better results in terms of DICE score for both networks. Segmentation time was ~60s per slice for manual segmentation vs. ~6s per slice for the proposed nnU-Net and ~16.5s MA-SAM, corresponding to 40, 4 and 11 min per volume.

Discussion and conclusion

In this study, we demonstrate the feasibility of nnU-Net and MA-SAM for automatic myocardium segmentation in 3D whole-heart joint T1/T2 mapping. Both approaches achieved good quality segmentations (Dice score higher than 0.856 with smoothed masks). Segmentation time is reduced with both networks, with nnU-Net being 10 times faster and MA-SAM being 4 times faster than manual segmentation, considering an average of 40 slices per patient. nnU-Net with smoothed masks achieved the best results in terms of visual assessment and DICE score and required the shortest processing time at inference. Nevertheless, difference in average Dice score with respect to MA-SAM did not exceed 2%, so further comparison in a larger testing cohort is needed.

Acknowledgements

The authors acknowledge financial support from: (1) BHF RG/20/1/34802 (2) EPSRC EP/V044087/1 (3) ANID Millennium Institute iHEALTH, ICN2021_004; Fondecyt 1210637 and 1210638; Basal Funding, IMPACT, FB210024 and (6) the Technical University of Munich – Institute for Advanced Study.

References

^[1] Captur G, Manisty C, Moon JC. Cardiac MRI evaluation of myocardial disease. Heart. 2016;102(18):1429-1435. doi:https://doi.org/10.1136/heartjnl-2015-30907

^[2] Salerno M, Kramer CM. Advances in Parametric Mapping With CMR Imaging. JACC: Cardiovascular Imaging. 2013;6(7):806-822. doi:https://doi.org/10.1016/j.jcmg.2013.05.005

^[3] Messroghli DR, Moon JC, Ferreira VM, et al. Clinical recommendations for cardiovascular magnetic resonance mapping of T1, T2, T2* and extracellular volume: A consensus statement by the Society for Cardiovascular Magnetic Resonance (SCMR) endorsed by the European Association for Cardiovascular Imaging (EACVI). Journal of Cardiovascular Magnetic Resonance. 2017;19(1). doi:https://doi.org/10.1186/s12968-017-0389-8

^[4]Milotta G, Bustin A, Jaubert O, Radhouène Neji, Prieto C, Botnar RM. 3D whole‐heart isotropic‐resolution motion‐compensated joint T1/T2 mapping and water/fat imaging. Magnetic Resonance in Medicine. 2020;84(6):3009-3026. doi:https://doi.org/10.1002/mrm.28330

^[5] Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods. 2020;18(2):203-211. doi: https://doi.org/10.1038/s41592-020-01008-z

^[6] Chen C, Miao J, Wu D, et al. MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation. arXiv (Cornell University). Published online September 15, 2023. doi:https://doi.org/10.48550/arxiv.2309.08842

Figures

Figure 1: Schematic representation of the proposed framework. A) A 3D volume is used as input for an nnU-net, which extracts a 'dataset fingerprint,' i.e., a set of dataset-specific properties such as image size, voxel information, intensities, etc. After the network prediction, corresponding predicted segmentation masks are provided. B) 2D inputs are used from preprocessing a 3D volume for the MA-SAM network, obtaining the predicted masks.

Figure 2: Fourth T2 weighted contrast of a subject with suspected cardiovascular disease, alongside its respective manual segmentation and the smoothed version.

Figure 3: Representative views (top: short axis, middle: two chamber, bottom: four chamber) of T1 maps from a subject with suspected cardiovascular disease. Pink contours depict the segmentation masks. Segmentations results are shown for a ) manual segmentation drawn slice by slice by an experienced clinical reader, b) nnUNet, c) nnUnet with smoothed masks, d) MA-SAM and e) MA-SAM with smoothed masks.

Figure 4: Representative views (top: short axis, middle: two chamber, bottom: four chamber) of T2 maps from a subject with suspected cardiovascular disease. White contours depict the segmentation masks. Segmentations results are shown for a) manual segmentation drawn slice by slice by an experienced clinical reader, b) nnUNet, c) nnUnet with smoothed masks, d) MA-SAM and e) MA-SAM with smoothed masks.

Figure 5: Blox plot of Dice score for the 4 testing: nnUNet, MA-SAM, "nnUNet_s" and "MA-SAM_s" of which the last two correspond to the test results using the smoothed data input. The respective average of each test is displayed beside each box.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)

2095

DOI: https://doi.org/10.58530/2024/2095