Introduction

The variations in MR images across different hardware, sequences, or scan parameters create a domain gap that needs to be bridged by a step called image harmonization. This is essential to facilitate the effective utilization of conventional or deep learning-based image analysis techniques, such as segmentation. Multiple methods, including a deep learning-based approach, have been proposed for achieving this image harmonization.^1-4 Nevertheless, these approaches frequently require multiple datasets for deep learning training and may still encounter challenges when applied to images from previously unseen domains. To mitigate this limitation, we introduced an approach known as "Blind Harmonization".⁵ This approach relies solely on the target domain data for training but retains the ability to harmonize images from previously unseen domains. As an implementation of this concept, we have developed BlindHarmony, leveraging an unconditional flow model.⁶

Methods

[BlindHarmony] The main idea of blind harmonization is to generate an image that is structurally close to the source domain image but has a high probability in the target domain. Firstly, we define image distance for quantifying structural closeness. Concerning the source domain image and corresponding harmonized image, we can see the two relationships: two images are highly correlated and the edges of the two images coincide. Based on this fact, we can define a distance between the source domain image ($$$x_s$$$) and its harmonized image ($$$x_h$$$):$$D\left(x_h,x_s\right)=\beta_1\{1 - NCC\left(x_h,x_s\right)\}+\beta_2 \| M G x_h \|_1,\qquad[Eq.1]$$ where $$$NCC$$$ denotes normalized crosscorrelation and $$$\| \|$$$ denotes the L1 norm. $$$M$$$ is a mask obtained by thresholding the gradient value of $$$x_s$$$, which retains the non-edge regions, and $$$G$$$ represents the gradient operator. $$$\beta$$$s are hyperparameters. Secondly, in order to measure the probability of the target domain, we utilize an unconditional flow model. If a flow model $$$f_\theta$$$ is trained only with the target domain images, the probability of the target domain image can be parameterized by a simple Gaussian latent vector ($$$z=f_\theta(x)$$$). Using these settings, we can formulate an optimization equation for blind harmonization as follows: $$\widehat{x_h}= \arg\min_x{D\left(x,x_s\right) + \alpha |f_\theta(x)|^2}.\qquad[Eq.2]$$ Direct optimization of Eq. 2 requires the calculation of the gradient of $$$f_\theta$$$ which is computationally demanding. Instead, we employ iterative optimization both in image and latent vector spaces. In a latent vector space, $$$z$$$ is updated so that it does not deviate from the center of the Gaussian. In an image space, the gradient descent of $$$D(x,x_s)$$$ is calculated and updated. After an N iteration, the resultant image would be a harmonized image. [Details and evaluation] T1-weighted images in the OASIS3⁷ dataset were used for train and evaluation. OASIS3 contains multisite data and images acquired with the Siemens TrioTim scanner were used as the target domain. The neural spline flow (NSF) model⁸ was used for flow model training. To evaluate the performance of the BlindHarmony, twenty traveling subjects with four source domains were utilized. We applied the BlindHarmony to the source domain images and calculated the PSNR and SSIM between the harmonized image and target domain image. Moreover, to measure the functional utility of the proposed method, we tackle the downstream task of white matter segmentation. The white matter segmentation network is trained on the target domain data, so it can describe the effectiveness of harmonization from source to target domain. As a comparison, histogram matching (HM), low-frequency replacing (SSIMH),⁹ CycleGAN-based style transfer, and supervised training with U-net are adopted. It should be noted that the CycleGAN and U-net are trained on each source-target pair.

Results

Figure 3 displays the results of harmonizing source domain images (column 1) to target domain images (column 2) using BlindHarmony (column 3). BlindHarmony effectively reduced inter-scanner variability and aligned the images with the target domain. Quantitative evaluation (Fig. 3b) via PSNR and SSIM metrics demonstrated improved values (averaged PSNR: 21.5 dB to 22.2 dB) compared to the source images. It is worth noting that U-net outperformed BlindHarmony, but they were trained separately for each source domain. Figure 4 presents white matter segmentation results for each harmonization method. Remarkably, BlindHarmony enhanced segmentation performance, harmonizing source domain images with the target domain effectively. The IoU values (Fig. 4b) were also higher for BlindHarmony, further confirming its superior harmonization performance.

Conclusion and Discussion

In this study, we propose BlindHarmony, a flow-based blind harmonization method for MR images. Unlike other existing harmonization methods, our network is trained exclusively on the target domain dataset and can be applied to previously unseen domain images. Our study demonstrates the feasibility of blind harmonization, providing an advantage in scenarios where access to source domain data is limited or unavailable.

Acknowledgements

This work was supported by the Korea Agency for Infrastructure Technology Advancement(KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant 21NPSS-C163415-01), Samsung Electronics Co., Ltd (IO201216-08215-01), and Institute of New Media and Communications (INMC), SNU.