2245
CDGAN：Cross Datasets Generative Adversarial Network for MR multi-contrast Image Synthesis1Xiamen University, Xiamen, China, 2Clinical & Technical Support, Philips Healthcare, China, Shengzhen, China
Synopsis
Keywords: Other AI/ML, Brain, Synthesis
Motivation: Multi-contrast MR images usually take a long time to scan, resulting in only a part of the valuable contrasts being obtained. Current deep learning methods face challenges when applied to domain adaptation across datasets or when tasked with generating high-quality images of various contrast.
Goal(s): Our purpose is to synthesize diverse contrast MRIs across different datasets.
Approach: we propose a cross-dataset generative adversarial network (CDGAN).The synthesized MR modalities of one specific object not only conform to the characteristics of the modalities themselves, but also have the same structure.
Results: This method effectively addresses the issue of synthesizing across datasets.
Impact: The method demonstrates a significant improvement in the quality of generated images when tested on different datasets.
Summary of Main Findings
The CDGAN we proposed successfully enables T1 to generate MRI with flexible contrasts across different datasets, and helps to obtain ultra-large and high-quality multi-contrast MRI training samples for various purposes.Introduction
Magnetic resonance imaging (MRI) allows the capture of anatomical structures under various contrast conditions, enabling the collection of additional diagnostic information during the examination process. However, due to the limitation of clinical imaging time, only a few contrast images can usually be collected. In the field of deep learning-based lesion segmentation or MRI reconstruction driven by synthetic data training1-2, it is often necessary to perform mass synthesis of missing contrast images. Generative adversarial networks (GANs) have emerged as the first choice for MRI synthesis. In supervised learning, improving the discriminator and generator is achieved by augmenting the loss function and modifying the generator (e.g., pGAN3 and edaGAN4), resulting in the synthesis of more realistic images within the same dataset. However, the application of these frameworks to domain adaptation problems yields poor results, and the generated contrast types are limited. The UNIT5 method has been verified to be able to solve the problem of cross-dataset generation of natural images. However, when we use this method to generate medical images, the quality of the generated images is not high. In addressing this, we propose a cross-dataset adversarial generative network capable of generating high-quality images with flexible contrast across different datasets.Methods
Datasets: IXI - This dataset comprises T1-weighted, T2-weighted, and PD-weighted images. HCP6 - This dataset includes T1-weighted images and diffusion-weighted imaging (DWI) with two b-values and 90 diffusion gradient directions. ATAG - This dataset consists of T1-weighted and T2*-weighted images.Framework: Figure 1 illustrates the specific framework. Utilizing T1 modality as a bridge, we designed the CDGAN network to enable T1 to generate other contrasts such as T2, PD, DWI, and T2*. CDGAN possesses cross-dataset generalization capabilities. In the first training phase, we address domain shift issues through a cycle-consistency structure. Subsequently, in the second training phase, a supervised learning approach is employed to generate high-quality target modalities. In the first phase, adversarial and cycle losses are utilized, while in the second phase, L1 loss, VGG loss, and adversarial loss are employed. This two-phase training approach ensures the CDGAN ability to generalize across datasets and generate high-quality images in various contrasts.
Results
Figure 2-4 present partial results. We utilized three datasets—IXI, ATAG, and HCP—to generate the T2* modality for IXI, the T2 and PD modalities for ATAG, and the T2 and PD modalities for HCP. Judging from the generated images, our method has more accurate texture details and image quality than UNIT, and has better contrast in the same dataset than pGAN. When tested within the same dataset, our method demonstrated comparable performance to pGAN. In cross-dataset testing, our approach outperformed pGAN and UNIT.Discussion
The generated results indicate that our approach effectively addresses the domain adaptation challenges across diverse datasets. Moreover, our framework demonstrates the capability to generate images with flexible contrast, unrestricted by the modalities present in the target datasets.Conclusion
In this study, our proposed CDGAN addresses the limitations of category scarcity and domain adaptation in conditional generation, resulting in the generation of higher-quality MRI images.Acknowledgements
This work was supported by the National Natural Science Foundation of China under grant numbers 82071913, 12375291 and 22161142024.References
1. Haq MM, Huang J. Adversarial domain adaptation for cell segmentation. In Proceedings of the Third Conference on Medical Imaging with Deep Learning 2020.
2. Lin X, Dai LX, Yang QQ, et al. Free-breathing and instantaneous abdominal T2 mapping via single-shot multiple overlapping-echo acquisition and deep learning reconstruction. European Radiology 2023.
3. Salman UH, Mahmut Y, et al. Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks. IEEE Transactions on Medical Imaging 2019.
4. Dalmaz O, Saglam B, Gonc K, et al. edaGAN: Encoder-Decoder Attention Generative Adversarial Networks for Multi-contrast MR Image Synthesis. International Conference on Electrical and Electronics Engineering 2022.
5. Liu MY, Breuel T, Kautz J. DIFFnet: Unsupervised Image-to-Image Translation Networks. Neurios 2017.
6. Essen DC, Ugurbil K, Auerbach E, et al. The Human Connectome Project: a data acquisition perspective. Neuro Image 2012.
Figures
