Introduction

Multiple Sclerosis (MS) is an inflammatory demyelinating and neurodegenerative disease of the central nervous system with severe neurological consequences. Effective management requires the development of new disease characterization strategies, which would benefit from advanced analysis of clinical MRI¹. However, clinical imaging is not always comprehensive or consistent. Cycle-Consistent Generative Adversarial Network (CycleGAN) is a deep -learning model that may overcome this challenge through non-paired image translation^2–4. CycleGAN contains two generators and two discriminators, which target creating synthetic images and differentiating authentic from generated images, respectively. Our goal is to investigate the utility of CycleGAN for mutually translating brain MRI sequences acquired from people with MS, with or without spectral normalization that is hypothesized to improve the training of GANs^5,6.

Methods

Brain MRI scans of 104 MS people (aged 20-40 years; ~2/3 being female) participating in an ongoing clinical study were examined. Imaging protocols included (1.5 and 3T) T1-weighted (T1w) MRI, acquired with slice thickness of 1-3 mm, matrix size of 192x256 to 512x512, TE of 2.08 to 33.96 ms, TR of 0.0043 and 2.49 s; and T2-weighted (T2w) MRI with slice thickness of 3mm, matrix size of 256x256 to 512x512, TE of 83-122.90 ms, and TR of 2.97-11.47 s. All images were preprocessed for consistency through these steps: brain extraction with HD-BET⁷, bias field correction with ANTs N4⁸, and nonlinear co-registration to MNI-152 with ANTs SyN^9,10.
The CycleGAN model was implemented following published specifications². It worked by bi-directionally translating T1w and T2w images (Fig.1). Two equivalent models were created: one with and another without spectral normalization in the discriminators. The models were trained for 200 epochs, with a linear learning rate decay from the 100^th epoch. Data were split at 0.8:0.1:0.1 for train, validation, and test. Model performance was assessed using 5 metrics: Mean Absolute Error (MAE), Normalized Root Mean Squared Error (NRMSE), Peak Signal-to-Noise-Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Multi-Scale SSIM (MS-SSIM). Statistical analyzes used the bootstrapped Welch’s t-test method with effect size calculated with Cohen’s d. Significance value was p < 0.05.

Results

Without normalization, the CycleGAN model demonstrated robustness in virtually all performance metrics. Specifically, for T1w to T2w translation, the MAE was 0.039±0.018, NRMSE was 0.11±0.028, PSNR was 21.16±4.03, SSIM was 0.75±0.13, and MS-SSIM was 0.81±0.093. For T2w to T1w translation, the metrics were slightly better: MAE = 0.036±0.028, NRMSE = 0.13±0.077, PSNR = 24.44±5.87, SSIM = 0.80±091, and MS-SSIM = 0.83±0.068. By adding spectral normalization, in T1w to T2w translation we had MAE = 0.031±0.014, NRMSE = 0.092±0.023, PSNR = 22.81±3.37, SSIM = 0.75±0.11, MS-SSIM = 0.81±0.080, and in T2w to T1w they were MAE = 0.038±0.030, NRMSE = 0.14±0.030, PSNR = , SSIM = 0.80±0.094, MS-SSIM = 0.83±0.073.
Fig. 3 shows sample images generated by the CycleGAN-based models. Incorporating spectral normalization, the CycleGAN converged to a lower loss than that without (Fig. 4), suggesting a better or similar performance. For T1w to T2w translation, spectral normalization resulted in lower MAE and NRMSE, and higher PSNR, SSIM, and MS-SSIM (p < 0.0001). However, in T2w to T1w translation, the non-normalized model showed a marginally lower MAE (p = 0.017, d = 0.061) and higher SSIM (p < 0.0001, d = 0.066), and similar NRMSE, PSNR, and MS-SSIM (Fig.5).

Discussion

This study demonstrated the possibility of CycleGAN-based models for overcoming the missing data problem as commonly seen in clinical practice based on brain MRI of MS individuals. The image quality metrics (SSIM and PSNR) achieved by the non-normalized CycleGAN model are similar to values achieved in previous studies performing a paired image analysis^11,12. Adding spectral normalization to the model improved model performance, especially in T1w to T2w translation, with moderate effect sizes achieved. However, the results were inconsistent in T2w to T1 translation, with almost opposite findings on MAE and SSIM. One possible explanation is that the resolution is higher in our T2w than T1w. In T1w to T2w translation, the generator is challenged to enhance the quality of generated T2w to that of source T2w. In this direction, spectral normalization has helped regulate the discriminators by preventing them from easily converging, and that improved generator’s performance. Our primary limitation stems from a relatively small sample size. Future studies are needed to verify the current findings using different samples, preferably with larger sample sizes, and test the usability of the generated images for secondary analysis.

Conclusion

Translation of brain MRI sequences for MS people is possible using CycleGAN -based deep learning models. Spectral normalization can help especially when the source domain has a lower quality than the destination domain.

Acknowledgements

No acknowledgement found.

References

1. Zeng, C., Gu, L., Liu, Z. & Zhao, S. Review of deep learning approaches for the segmentation of multiple sclerosis lesions on brain MRI. Front. Neuroinform. 14, 610967 (2020).
2. Zhu, J.-Y., Park, T., Isola, P. & Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. in 2017 IEEE International Conference on Computer Vision (ICCV) (IEEE, 2017). doi:10.1109/iccv.2017.244.
3. Takamiya, K., Iwamoto, Y., Nonaka, M. & Chen, Y.-W. CT brain image synthesization from MRI brain images using CycleGAN. in 2023 IEEE International Conference on Consumer Electronics (ICCE) 1–4 (IEEE, 2023).
4. Van Nguyen, H. & Trong, T. H. Brain MRI images generating method based on CycleGan. J. Sci. Technol. Issue Inf. Commun. Technol. 13–18 (2022).
5. Lan, H., Toga, A. W., Sepehrband, F. & the Alzheimer Disease Neuroimaging Initiative. Three‐dimensional self‐attention conditional GAN with spectral normalization for multimodal neuroimaging synthesis. Magn. Reson. Med. 86, 1718–1733 (2021).
6. Miyato, T., Kataoka, T., Koyama, M. & Yoshida, Y. Spectral normalization for generative adversarial networks. arXiv [cs.LG] (2018).
7. Isensee, F. et al. Automated brain extraction of multisequence MRI using artificial neural networks. Hum. Brain Mapp. 40, 4952–4964 (2019).