1398
Enhancing Deep Learning-Based Liver Vessel Segmentation on MRI with Image Translation Techniques1Siemens Healthineers, Princeton, NJ, United States, 2Department of Electrical and Computer Engineering, University of Arizona, Tucson, AZ, United States, 3Department of Medical Imaging, University of Arizona, Tucson, AZ, United States, 4Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, United States, 55Department of Bioengineering, University of California Los Angeles, Los Angeles, CA, United States, 6Siemens Healthineers, Erlangen, Germany, 7Siemens Healthineers, Malvern, PA, United States
Synopsis
Keywords: AI Diffusion Models, Segmentation, Liver Vessel Segmentation
Motivation: To improve liver vessel segmentation on MRI under annotation constraints.
Goal(s): Apply an advanced unpaired image translation technique, SynDiff, to create synthetic MR images from CT data.
Approach: By incorporating vessel masks in the translation process, the optimized SynDiff models generated synthetic images that facilitated more effective pretraining of segmentation models.
Results: Validated across multiple pretraining settings, the refined SynDiff approach surpassed the standard nnU-Net and other pretraining-based methods, substantially improving liver vessel segmentation performance.
Impact: This study remarkably advances liver vessel segmentation on MRI, demonstrating that synthetic data can effectively augment limited datasets, leading to improved model performance. It has great potential for broader applications in medical image analysis.
Introduction
Liver vessel segmentation on MRI is important for data processing in diagnostic tasks [1]. To boost deep learning-based segmentation with limited data, pretraining and synthetic data adoption are effective strategies. Recent image translation techniques take advantage of available data in different domains (e.g., translating CT to MRI) [2], and an adversarial diffusion modeling-based method, SynDiff, has achieved state-of-the-art performance using unpaired data [3].. In this work, we adapted the SynDiff approach to generate a substantial volume of synthetic MR images from a CT dataset with vessel segmentation and employed this synthetic data to pretrain a neural network model for liver vessel segmentation on MRI.Method
Datasets: We developed and evaluated our segmentation models using an in-house MRI dataset (40 cases) with liver vessels annotated by an expert. Each case had a 3D volume of T1-weighted (T1w) Dixon water images. The dataset comprises 52 to 88 slices per case, with matrices ranging from 256x240 to 576x416, in-plane resolution from 0.69mm to 1.40mm, and slice thickness of 3.0mm to 4.0mm. For CT-to-MR image translation and segmentation pretraining, we used the hepatic vessel subset (with segmentation masks) from the Medical Segmentation Decathlon Dataset [4], comprising 303 3D CT volumes.CT-to-MR Translation: Figure 1 depicts the workflow of our proposed method. A total of 12,521 CT slices and 1,689 MR slices were obtained to train the image translation model. A straightforward approach is only using MR/CT images as training data to generate synthetic MR images. However, as illustrated in Figure 2, the vessels in MR and CT images exhibit different contrast against the background liver tissue, and an unpaired image translation model struggles to capture this subtle distinction. Therefore, we enhanced SynDiff by integrating vessel masks as a second channel to focus more on the vessel regions. We denoted these two CT-to-MR translation methods as SynDiff-1 and SynDiff-2, respectively. Subsequently, we used the trained SynDiff-1 and SynDiff-2 models to generate 12,521 synthetic MR images from CT slices. For the corresponding liver vessel masks, the original CT annotations were assumed as ground truth for SynDiff-1 synthetic data, while the binarized second channel of a SynDiff-2 generated image was assumed as ground truth for SynDiff-2 synthetic data. Figure 2 presents an example of real MR and CT data alongside the synthetic MR.
Segmentation Models: We trained the nnUNet (2D) [5] to segment liver vessels in T1w Dixon water MR images. The segmentation model trained solely with real MR data served as the baseline. In addition, we pretrained the liver vessel segmentation model using SynDiff-1/SynDiff-2 data, followed by fine-tuning with real MR data. For a thorough comparison, we also directly used real CT data for pretraining. Moreover, we compared pretraining with 30, 100, and all 303 CT-to-MR synthetic cases.
Evaluation: The 40 real MRI cases were randomly divided into five subsets for a five-fold cross-validation. Vessel segmentation results were characterized by Dice scores and Average Symmetric Surface Distance (ASSD) [6].
Results
Representative vessel segmentation results are shown in Figure 3. Tables 1 and 2 quantitatively compare the segmentation models across various training settings. The baseline segmentation model achieved a Dice score of 0.693 ± 0.061, while pretraining with CT cases yielded comparable Dice scores. Pretraining with SynDiff-1 data marginally improved the Dice scores, and pretraining with 303 SynDiff-2 cases achieved the highest Dice scores of 0.717 ± 0.045. Regarding ASSD, the baseline segmentation reached 1.74 ± 0.72 mm; pretraining with either CT data or SynDiff-1 showed similar performance. In contrast, pretraining with 303 SynDiff-2 cases resulted in the best ASSD of 1.56 ± 0.63 mm.Discussion
These results show that the efficacy of pretraining is contingent upon the similarity between the synthetic and real images. Since SynDiff is tailored for 2D image translation, we limited our pretraining and fine-tuning to 2D segmentation models. Future endeavors may involve 3D image translation for 3D segmentation models. Additionally, while the nnU-Net method served as the baseline for experiments, the proposed method is designed to be universally adaptable to various segmentation approaches. Furthermore, while liver vessel segmentation was conducted on the T1w Dixon water MR images for this study, our proposed image translation method for vessel segmentation training is potentially applicable to other MRI sequences or imaging modalities.Conclusion
This work showed that pretraining strategies using synthetic MR images generated by the enhanced SynDiff method markedly boosts deep learning-based liver vessel segmentation on MRI. This approach offers a promising solution to the challenge of limited annotated data, demonstrating the potential of synthetic data in medical image analysis.Acknowledgements
This work was supported by the National Institutes of Health under award number NIH/NIBIB U01 EB031894.References
[1] Moccia, S., De Momi, E., El Hadji, S., & Mattos, L. S. (2018). Blood vessel segmentation algorithms—review of methods, datasets and evaluation metrics. Computer methods and programs in biomedicine, 158, 71-91.
[2] Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2223-2232).
[3] Özbey, M., Dalmaz, O., Dar, S. U., Bedel, H. A., Özturk, Ş., Güngör, A., & Çukur, T. (2023). Unsupervised medical image translation with adversarial diffusion models. IEEE Transactions on Medical Imaging.
[4] Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, et al (2022). The medical segmentation decathlon. Nature communications, 13(1), 4128.
[5] Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2), 203-211.
[6] Heimann, T., Van Ginneken, B., Styner, M. A., Arzhaeva, Y., Aurich, V., Bauer, C., ... & Wolf, I. (2009). Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE transactions on medical imaging, 28(8), 1251-1265.
Figures
