2415

Correcting MRI-specific biases introduced when Bland-Altman plots are used to compare the performance of reconstruction algorithms

Michael Smith¹, Ishani DasGupta¹, and Elise Fear¹

¹University of Calgary, Calgary, AB, Canada

Synopsis

The Bland-Altman plot is a commonly-used graphical method to compare two measurement techniques and look for systematic biases or outliers. We have identified that the Bland-Altman approach of plotting the differences between the two techniques against their average will introduce false biases in an MRI context. These biases are introduced by the magnitude operation necessary to display or analyze MRI images. We demonstrate a modified Bland-Altman approach that corrects these biases.

Introduction

MRI-appropriate metrics are needed to identify potential intensity or noise biases introduced by algorithms: e.g. when compressed sensing [1, 2] or data truncation [3] parameters are modified. Using mean-square error measures or approaches that mimic the human visual system [4] have limitations, including requiring comparison between an image and a gold standard that often differ in size because of MRI protocols. The Bland-Altman plot [5] is a well-established, alternative approach to identify inconsistency between two measurement techniques by graphing the difference between their results against their average.. We have identified that standard Bland-Altman plots introduce false biases in an MRI context, and propose correction approaches.

Theory

Bland-Altman plots can be applied in an MRI context by graphing the differences between images, $$$(I1[x,y]- I2[x,y])$$$, generated by two reconstruction protocols, against their local average intensity, $$$(I1[x,y]+I2[x,y])/2$$$. Outliers and systematic biases can by identified by computing the mean and standard deviation of the differences for each average intensity. In Fig. 1, the changes in the standard deviation of image differences at a given average intensity appear to indicate dissimilarities between the two imaging protocols in high and low intensity levels impacting the contrast of those regions of interest (ROI). However, the reconstructed images were generated differing only by a minor factor in their initial k-space SNR. We hypothesize these false biases are introduced by the magnitude operation needed to prepare the complex-valued MRI images for display or analysis, similar to the MRI SNR biases described by McGibney and Smith [6]. We propose a modified plot generated prior to the magnitude operation in order to remove these MRI-specific Bland-Altman biases.

Method

Gaussian white noise with zero mean was added to the high-resolution 512 x 512 k-space of a G.E. phantom, Fig. 2, prior to DFT reconstruction in order to generate two images with signal-to-noise ratios (SNR) of 16 and 20. Additional images were generated with the centre of k-space shifted by 4 points in both $$$kx$$$ and $$$ky$$$ directions to ensure strong real and imaginary components present in their final, complex-valued images. A standard Bland-Altman analysis was generated with differences between the magnitude images plotted against the average intensity of these images. In our proposed modified Bland-Altman approach, the differences between the real-components of the image are plotted against the real-component average intensity with additional plots comparing the imaginary components of the image differences and local average intensities.

Results

Standard Bland-Altman plots obtained with magnitude data are shown for images that are identical except for SNRs of 16 and 20 in Fig. 1, and with the addition of mis-centred k-space data in Fig. 3. Both the plots from the centred and mis-centred k-space data sets show standard deviations changing with image intensity, unexpected for essentially identical images differing in only their SNR. The distortion in the plots at low intensity amplitude can be explained with an extension of ideas from McGibney and Smith [6]. Specifically, Gaussian white noise (GWN) superimposed upon a large intensity ROI will remain with GWN characteristics after the magnitude operation. This is in contrast with GWN on low intensity ROI, where noise rectification will occur during the magnitude operation. This causes a Rician noise distribution which has no negative value, leading to a non-zero mean and an effective standard deviation which both change with ROI intensity level.

Figs. 4 and 5 show the results from the proposed modified Bland-Altman plot that has been adjusted for the MRI context. Although shifting the center of k-space drastically changes the appearance of the complex valued plots between Figs. 4 and 5, neither of the modified plots show distortions in the standard deviation of the noise, which correctly remains constant with changes in the average intensity.

Discussion and Conclusion

The Bland-Altman plot is an established graphical approach to identify biases between two measurement techniques. In an MR context, these plots can be used to compare images from different reconstruction algorithms to identify potential changes in intensity or noise characteristics as new algorithm parameters are introduced or optimized. We have identified that Bland-Altman plots can introduce false biases or hide true biases when applied in an MRI context. We have identified that these biases are introduced by the magnitude operation necessary to display or analyze MRI images. This rectification operation modifies the noise characteristics of images in an intensity dependent manner. We have proposed a modified Bland-Altman approach that corrects these biases. We are exploring the impact of potentially equivalent biases present when Total Variation techniques are used during compressed sensing reconstruction to suppress noise introduced due to under-sampling k-space data

Acknowledgements

Thanks to Dr. E. MacDonald, University of Calgary, for supplying the G.E. Phantom data.

References

[1] M. Lustig, D. Donoho and J. M. Pauly, “Sparse MRI: The Application of Compressed Sensing for Rapid MR Imaging”, Magnetic Resonance in Medicine, 58, 1182 – 95, 2007.

[2] S. Ravishankar and Y. Bresler, "MR image reconstruction from highly undersampled k-space data by dictionary learning," IEEE transactions on medical imaging, 30(5), 1028 – 1041, 2011.

[3] Z. P. Liang et al., “Constrained Reconstruction Methods in MR Imaging”, Reviews of Magnetic Resonance in Medicine, 4, 67 – 185, 1992.

[4] P. Adibpour and M. Smith, "Total Variation Assisted Fourier Shift Manipulation to Remove Gibbs' Artifacts in Compressive Sensing Techniques", IEEE Nuclear Science Symposium / Medical Imaging Conference, Strasbourg, France, 2016.

[5] J. M. Bland. D. G. Altman, “Statistical methods for assessing agreement between two methods of clinical measurement”, Lancet, 307 - 310, 1986.

[6] G. McGibney and M. Smith, "An unbiased signal-to-noise ratio measure for magnetic resonance images", Medical Physics, 20 (4), 1077 - 1078, 1993.

Figures

Fig. 1: Standard Bland-Altman plot reveals reduced standard deviations at low intensities suggesting biases between the reconstruction approaches used to reconstruct images. These biases are false as the images were reconstructed from identical k-spaces except for $$$SNR = 20$$$ and $$$SNR = 16$$$.

Fig. 2: High resolution IDFT reconstruction from a GE phantom’s 512 x 512 k-space

Fig. 3: Standard Bland-Altman plot for two images that are identical except for $$$SNR = 20$$$ and $$$SNR = 16$$$ with the k-space data miss-centered by 4 samples in both $$$kx$$$ and $$$ky$$$ directions. This apparently indicates the same biases between the two reconstruction techniques at low intensities as shown in Fig. 1.

Fig. 4: The proposed modified Bland-Altman plot for two images that are identical except for $$$SNR = 20$$$ and $$$SNR = 16$$$. The real and imaginary components of the modified plots are offset for easier interpretation.

The standard deviations of the differences between the two images are no longer distorted by the magnitude operator; remaining constant with intensity level. Our modified plot has revealed a previously undetected intensity bias at low, essentially background, intensities. Further investigation showed this bias to be real, and associated with image distortions when the original low-noise k-space was unintentionally overwritten by its magnitude image's k-space.

Fig. 5: Modified Bland-Altman plot for two images that are identical except for $$$SNR = 20$$$ and $$$SNR = 16$$$, with equivalent miss-centring of k-space data by 4 samples in both $$$kx$$$ and $$$ky$$$ directions. The real and imaginary components of the modified plots are offset for easier interpretation.

As expected, the miss-centring of k-space data has introduced significant imaginary components into the complex-valued MRI image. However, the standard deviation of the differences correctly remains constant at all intensity levels . The plot still shows the intensity biases unintentionally introduced by over-writing the original low-noise k-space by its magnitude image's k-space.

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)

2415