Introduction

Recent advances in image processing and computer vision have led to the emergence of various clinical decision support tools for medical imaging, and particularly MRI. The deployment of such tools across multiple institutions is, however, hindered by data heterogeneity that can arise from differences in MR scanner hardware or acquisition protocols. To overcome this limitation, post-hoc harmonization techniques can be implemented to reduce inter-site variability and improve the performance of clinical decision support tools^1–3.
In this work, we propose to use a conditional Generative Adversarial Network⁴ (cGAN) to perform cross-protocol harmonization of T₁-weighted MP-RAGE brain images in a dementia patient cohort. Differences between brain images both prior and after harmonization are quantified in terms of both image similarity and volumetric estimation of brain regions using MorphoBox⁵, an automated brain segmentation prototype software.

Methods

MR dataset and preprocessing
Seventy-two patients suffering from dementia (36$$$\,$$$females, age 70.2$$$\,$$$±$$$\,$$$11.9$$$\,$$$y/o) underwent an MR examination at 3T (MAGNETOM Prisma, Siemens Healthcare, Erlangen, Germany). Two distinct 3D T₁-weighted MP-RAGE sequences were acquired within the same session, resulting in paired acquisitions. The protocol parameters of the two sequences are reported in Table$$$\,$$$1.
Images acquired with protocol 1 (P1) were considered as reference since MorphoBox was optimized for this contrast. Images acquired with protocol 2 (P2) were corrected for intensity non-uniformities using N4 bias field correction and spatially rigidly registered to P1 images.

Harmonization using a GAN
Eighty central slices were extracted in each orientation (i.e., sagittal, axial and coronal) for both protocols from sixty-five randomly selected training patients, resulting in approximatively 16k training images. A 2.5D pix2pix model⁴ was implemented using a three channels input composed by stacks of three consecutive slices in the same orientation, within the same minibatch. The model was trained to learn a mapping between co-registered N4-corrected P2 images and the reference P1 images. The cGAN architecture and relevant training parameters are reported in Figure$$$\,$$$1.
During testing, three volumes were reconstructed independently for each orientation, and voxel-wise averaging was performed to obtain the final harmonized image for seven test datasets.

Statistical analysis
To study the added value of using a cGAN for cross-protocol harmonization, similarity metrics were computed between protocols, both before and after harmonization, in the testing cohort. Particularly, we computed the mean squared error (MSE), the peak-signal to noise ratio (PSNR) and the structural similarity index measure (SSIM⁶).
Subsequently, the effects of harmonization on brain morphometry were assessed by estimating brain volumes using MorphoBox in original images (P1 and P2), preprocessed images (N4 bias field correction, co-registration) and harmonized images using the cGAN. For each image, we computed the relative error in volumetric estimation of brain regions with respect to the reference P1. A paired, two-sided Student t-test was performed to identify significant differences in the distribution of relative errors between original images acquired with P2 and harmonized images. P-values were corrected for multiple comparison using Benjamini-Hochberg false discovery rate method.

Results

Figure$$$\,$$$2 shows representative examples of the images acquired with the two protocols, after preprocessing and after harmonization, together with the resulting segmentations. Compared to the reference segmentation, the boundaries between brain lobes (highlighted with arrows) showed substantial differences when computed on P2 images. Visually, the segmentation of the occipital lobe improved after correcting for N4 bias field (orange arrow), whereas the boundary between the frontal and parietal lobe (yellow arrow) improved in cGAN-harmonized images.
The distribution of image similarity metrics computed in the testing cohort is shown in Figure$$$\,$$$3 for the original contrast (in orange), the N4-corrected image (in dark red) and the cGAN-harmonized result (in blue). In terms of MSE and PSNR, the harmonized image was the closest to the reference image, whereas the highest SSIM was achieved for the preprocessed contrast.
When comparing the relative errors in volumetric estimation, represented as mean$$$\,$$$±$$$\,$$$sd in Figure$$$\,$$$4, one can notice that overall, applying N4 bias field correction (in dark red) reduces the relative error compared to the original images (in orange). However, the greatest improvement was observed after harmonizing the contrasts using the cGAN, which significantly improved the segmentation for most brain regions, highlighted in green. The greatest improvements were observed for the segmentation of white matter in brain lobes. On the other hand, the segmentation accuracy of some regions (highlighted in red) was also decreased with a smaller effect size.

Discussion and Conclusions

This work shows that post-hoc harmonization using a cGAN is a promising approach to improve performance and generalizability of automated clinical decision support tools for radiology. Overall, the similarity between contrasts was improved. The lower value in SSIM is likely to be caused by a checkboard pattern in the synthetic image. Nevertheless, when segmented with MorphoBox, the volumetric estimation of most brain regions was significantly improved by the harmonization.
While we only tested on one segmentation tool, we believe that this conclusion is generalizable to similar software. Despite its relevance and utility to overcome the limitation of data heterogeneity between different institutions, this method was implemented only for two protocols and therefore suffers from poor scalability to multiple sites. Further, it requires paired acquisitions of travelling subjects which limits its applicability to clinical frameworks. Thus, future work should investigate new unsupervised methods.

Acknowledgements

No acknowledgement found.