Synopsis

We present a multi-site diffusion MRI (dMRI) data harmonization method using CycleGAN network with segmentation loss (CycleGAN_S). This method aims to learn an efficient mapping of dMRI signal using rotation invariant spherical harmonics features from the same set of subjects across sites. At the same time, it has potential to learn the tumor pathology (if exists) during harmonization. We compare our CycleGAN_S network with the CycleGAN network. We show that our CycleGAN_S network has better multi-site diffusion MRI data harmonization accuracy. Moreover, our method shows up to 60% improvement on the prediction of tumor pathology.

Introduction

Harmonization of multi-site diffusion MRI (dMRI) datasets can dramatically increase the statistical power of neuroimaging studies and enable comparative studies pertaining to several brain disorders^1,2. Given the importance of the problem, initially, several data pooling methods^1,2,3,4 have been published which are based on removing statistical differences from diffusion tensor imaging derived metrics. Recently, model-free dMRI harmonization methods have been proposed which can be used to harmonize the raw dMRI signal across sites. Based on the dMRI data on hand, there have been successful attempts to harmonize raw dMRI signal^5-10. However, none of the methods have been tested on harmonizing of diffusion MRI data with gross pathologies. In this work, we propose a multi-site dMRI data harmonization using CycleGAN with segmentation loss (CycleGAN_S) which has two goals: (i) learning an efficient mapping of dMRI signal from the same set of subjects across sites; (ii) preserving existing tissue abnormalities or pathologies during harmonization.

Methods

In our previous work^5-8,10, rotation invariant spherical harmonics (RISH) have been shown to successfully represent different micro-structural tissue properties of the dMRI signal for each spherical harmonics (SH) order. They can be appropriately scaled to modify the dMRI signal without changing the principal directions of the fibers⁵. In this work, using HCP Prisma RISH-features (up to 6th order: {L0, L2, L4, L6}) as input, our first goal is to learn a nonlinear mapping of Prisma to Connectom scanner using the paired RISH-features. To this end, we first match the resolutions, then spatially align Connectom and Prisma RISH-features for each subject. Later, we segment the white matter, gray matter and CSF from the L0 RISH-feature using Geodesic Information Flow (GIF) software¹⁷. Next, we construct CycleGAN^14-16 network with aligned Prisma RISH-features as input and Connectom RISH-features as output with their corresponding segmentation maps (Figure 1). The cycle consistency and adversarial loss were shown¹⁶ to only constrain the network to learn a global mapping that matches the marginal distribution but not the conditional distribution pertaining to the tissue variabilities. Therefore, a model trained using these losses does not guarantee to preserve the pathologies (e.g. tumor). In addition to adversarial and cycle-consistency loss, “segmentation loss”¹⁵ can be included as a multi-class cross entropy loss: $$$L_{segm}(S_P,S_C,G_P,G_C)=-Y_P log(S_P(G_P(C)))+-Y_C log(S_C(G_C(P)))$$$, where $$$Y$$$ denotes the ground truth segmentation map. The segmentation loss can regularize the generators and preserve the variability of the tissues (white matter, gray matter or CSF). Segmentation map can be extended to pathologies by supervising the pathology as a semantic label.

Experiments and Results

Harmonization. We used 16 HCP healthy subjects with dMRI scans obtained from both Siemens Prisma and Skyra Connectom scanners¹¹, where we learned the mapping for $$$b=3000$$$ shell from Prisma to Connectom scanner using RISH-features. The final signal was estimated by multiplying Prisma SH coefficients with the voxel-wise scale between Connectom and harmonized Prisma RISH-features. We generated Fractional-Anisotropy (FA), Mean-Diffusivity (MD), mean Diffusion-Weighted-Imaging (DWI) and RISH-features L0, L2 using the Connectom data and our harmonized results. Training was done using leave-one-out cross validation and Normalized Mean Squared Error (NMSE) was computed on all measures (See Table 1 for the performance of CycleGAN and CycleGAN_S networks).

Validation on pathology. To test the performance of the harmonization on data with pathology, we simulated tumor on Prisma and Connectom dMRI data. Independent dataset that includes a low-grade tumor subject (scanned on a Siemens scanner at Brigham and Women’s Hospital) was used for simulation, where its T1, T2 and dMRI data were co-registered. Then, the steps in Figure 2 were applied to simulate tumor for Prisma and Connectom dMRI data (See Figure 3 for RISH-features of tumor data).

Next, we ran two experiments: (1) we trained CycleGAN and CycleGAN_S networks with 16 healthy subjects and tested on data with simulated tumor; (2) we trained CycleGAN and CycleGAN_S with simulated tumor as input. For CycleGAN_S, the tumor was included as a semantic label in the segmentation map. Figure 4 depicts the qualitative comparison of the methods for RISH-Feature L0. Our CycleGAN_S network is observed to perform better in prediction of tumor when compared to CycleGAN. Quantitatively, CycleGAN_S showed up to 60% improvement when compared to CycleGAN on the prediction of tumor for Connectom data.

Discussion and Conclusion

Acknowledgements

References

1. Jahanshad, N. et al. Multi-site genetic analysis of diffusion images and voxelwise heritability analysis: A pilot project of the enigma-dti working group. NeuroImage 81, 455-469 (2013).

2. Kochunov, P. Multi-site study of additive genetic effects on fractional anisotropy of cerebral white matter: Comparing meta and mega analytical approaches for data pooling. NeuroImage 95, 136-150.

3. Pohl, K.M. et al. Harmonizing dti measurements across scanners to examine the development of white matter microstructure in 803adolescents of the ncanda study. Neuroimage 130, 194-213 (2016).

4. Fortin, J.P. et al. Harmonization of multi-site diffusion tensor imaging data. Neuroimage 161, 149-170 (2017).

5. Mirzaalian H. et al. Inter-site and inter-scanner diffusion mri data harmonization. NeuroImage 135, 311-323 (2016).

6. Mirzaalian H. et al. Multi-site harmonization of diffusion mri data in a registration framework. Brain Imaging and Behavior, (2017).

7. Mirzaalian, H. et al. Harmonizing Diffusion MRI Data Across Multiple Sites and Scanners. Springer. p. 12-19, (2015).

8. Cetin Karayumak, S. et al. Retrospective harmonization of multi-site diffusion MRI data acquired with different acquisition parameters. Neuroimage 184, 180–200 (2018).

9. Tax C. et al. Cross-vendor and Cross-protocol harmonisation of diffusion MRI data: a comparative study. ISMRM 2018.

10. Cetin Karayumak, S. et al. Harmonizing diffusion MRI data across magnetic field strengths. Medical Image Computing and Computer Assisted Intervention- MICCAI 2018 pp 116-124 and Lecture Notes in Computer Science book series (LNCS, volume 11072).

11. Van Essen DC, Smith SM, Barch DM, et al. The WU-Minn Human Connectome Project: An Overview. NeuroImage. 2013;80:62-79. doi:10.1016/j.neuroimage.2013.05.041.

12. Avants, B.B., Tustison, N.J., Song, G., Cook, P.A., Klein, A., Gee, J.C.A reproducible evaluation of ANTs similarity metric performance in brain image 0 registration. NeuroImage 54, 2033-2044, 2011.

13. Fedorov A. et al. 3D Slicer as an Image Computing Platform for the Quantitative Imaging Network. Magn Reson Imaging. 2012 Nov;30(9):1323-41.

14. Jun-Yan Zhu et al., Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, ICCV, 2017.

15. Zhang, Z. et al. Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency, CVPR, 2018.

16. Jue Jiang et al., Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation, MICCAI, 2018.

17. Cardoso, M.J. et al., Geodesic information flows: Spatially-variant graphs and their application to segmentation and fusion. IEEE Transactions on Medical Imaging 34(9) (2015) 1976–1988