0876

Multi-Atlas Segmentation of MR Brain Images with Lesions Using Subspace-Assisted-GAN Based Image Recovery
Yi Ding1, Huixiang Zhuang1, Yue Guan1, Yunpeng Zhang1, Ziyu Meng1, Zhi-Pei Liang2,3, and Yao Li1
1School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China, 2Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, United States, 3Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, United States

Synopsis

Keywords: Analysis/Processing, Machine Learning/Artificial Intelligence, Multi-Atlas Segmentation

Motivation: Multi-atlas segmentation (MAS) of MR brain images with lesions is of great clinical significance but remains challenging due to registration inaccuracy caused by pathologies.

Goal(s): Our goal was to improve the MAS performance of pathological brain images by restoring more accurate normal images form lesion data.

Approach: We integrate a novel subspace-assisted generative model into the MAS framework for estimation of subject-specific posterior normative distribution, which can effectively extract a “hypothetical” normal image from the lesion data, thus enhancing the accuracy of lesion segmentation.

Results: Our method produced significantly improved results in normal recovery and MAS compared to the state-of-the-art methods.

Impact: The proposed method significantly improves the performance of segmentation of MR brain images with lesions, which may provide a useful tool for tissue segmentation in pathological brain images.

Introduction

MULTI-ATLAS segmentation (MAS) is a commonly used method for segmenting normal brain images. It involves transferring label information from multiple brain atlases to a new brain image through image registration and label fusion1-6. However, MAS often falls short when applied to brain images with lesions like strokes and tumors, as these lesions can hinder the accurate registration of normal brain atlases to the target image. Improving the registration of pathological brain images can be approached through methods like masking or inpainting of pathological brain regions. Masking methods, such as Cost Function Masking (CFM)7, identify and exclude the pathological regions during registration, which is sensitive to segmentation inaccuracies. Inpainting methods, like Low-Rank plus Sparse matrix Decomposition (LRSD)8 and spatially constrained low-rank (SCOLOR)9, leverage low-rank and sparse priors from normal images to restore tumor brain images to their normal appearance, improving the registration performance. But due to the curse of dimensionality, these model-based methods could not effectively capture the spatial-intensity normal distributions. In this paper, we propose a novel subspace-assisted generative model designed to capture the subject-specific posterior normative distribution and effectively restore the normal image from lesion data. This model is integrated into a new MAS framework, enabling reliable and accurate segmentation of lesion brain images. We have compared our proposed method with state-of-the-art methods using both synthetic and real MR brain images with stroke lesions, producing very encouraging results.

Methods

The proposed method contains two integral components: 1) subspace-assisted 3D conditional generative model for normal image recovery from the lesion data, and 2) normal recovery-enabled multi-atlas segmentation. An overview of the proposed method is shown in Figure 1.

Subspace-assisted 3D conditional generative model for normal recovery
To estimate the normal image from the lesion data, we proposed to train a conditional generative model to capture the lesion image-specific posterior normative spatial-intensity distribution. The normal image was then estimated based on the conditionally sampled images from the trained generative model in a Maximum-A-Posterior (MAP) sense. In particular, given an image $$$Y=[Y_1,Y_2,\ldots,Y_K]^T$$$ with lesion, we first obtained an initial normal recovered image $$$\hat{X}=[\hat{X}_1,\hat{X}_2,\ldots,\hat{X}_K]^T$$$ leveraging a probabilistic image subspace model:$$\qquad\hat{X}_k=(1-M_k)\odot Y_k+M_k\odot\sum_{r=1}^R\alpha_{kr}\phi_{kr}(x),\qquad 1\leq k\leq K\qquad\quad (1)$$where $$$M_k$$$ is the pre-estimated normal pixels in $$$Y_k$$$ based on the prior intensity distributions of the normal training images $$$\{I_n(x)\}$$$ using Bayesian hypothesis testing, $$$\{\phi_{kr}(x)\}_r$$$ is the set of basis functions obtained by applying principle component analysis to the $$$k$$$th slice of $$$\{I_n(x)\}$$$, $$$\alpha_{kr} \sim p(\{\alpha_{kr}\})$$$ is the coefficients of $$$\phi_{kr}(x)$$$ obtained by MAP estimation:$$\begin{aligned}\{\hat{\alpha}_{kr}\}&=\underset{\{\alpha_{kr}\}}{\operatorname{argmax}}\{\alpha_{kr}\mid Y_k\}\\&=\underset{\{\alpha_{kr}\}}{\operatorname{argmin}}\|(1-M_k)\odot[Y_k-\sum_{r=1}^R \alpha_{kr}\phi_{kr}(x)]\|_2^2+\lambda\log p(\{\alpha_{kr}\})\qquad(2)\end{aligned}$$After $$$\hat{X}$$$ was obtained, we trained a 3D conditional generative adversarial network (GAN)10 to recover the high-order spatial-intensity features for posterior normative distribution learning. More specifically, we learned the mapping from the subspace model-based perturbed images $$$x_p=[x_{p_1},x_{p_2},\ldots,x_{p_K}]^T$$$ to the real normal image $$$I(x)$$$. The perturbed image $$$x_p$$$ was obtained as:$$x_{p_k}=m_k\odot[\sum_{r=1}^R\beta_{k r}\phi_{k r}(x)]+(1-m_k)\odot Y_k,\qquad\beta_{k r}\sim\mathcal{N}(\hat{\alpha}_{kr},\sigma_{kr})\qquad(3)$$where $$$\sigma_{kr}$$$ is the standard deviation of the subspace coefficient $$$\hat{\alpha}_{k r}$$$ of the training dataset, $$$m$$$ is the lesion mask obtained by voxel-wise Bayesian hypothesis testing leveraging the global normative spatial-intensity distribution derived from $$$\hat{X}$$$. The 3D conditional GAN was optimized by:$$\min_G\max_D E_{x\sim p(\{I_n(x)\})}\log D(x)+E_{x_p\sim p(x_p\mid\hat{x},m)}\log(1-D(G(x_p,m)))\qquad\quad(4)$$Thereafter, a set of subject-specific posterior normal images $$$\{\hat{x}_G\}$$$ were obtained using Monte Carlo sampling and the final normal recovered image of $$$Y$$$ was estimated in a MAP sense.

Multi-atlas segmentation
With the normal image recovered from the lesion data, the structural atlas images could be registered to the normal recovered images for improved alignment using SyN11. The estimated deformation fields were then applied to the label images. The final decision was made using majority voting12.

Results

To capture the prior distribution of normal brain tissues, we used 20000 T1w brain images of healthy subjects from Biobank13. We evaluated the performance of normal recovery using simulated data and real public ATLAS14 stroke dataset (N=236). Figures 2 and 3 demonstrated that our proposed method achieved better normal recovery than state-of-the-art methods on both simulation and real data, respectively. Figure 4 shows the improvement in registration performance, especially near the lesion area, using our proposed method. Additionally, we used Mindboggle-10115 dataset (N=101) for the evaluation of MAS, which have 31 manually segmented cortical regions. Figure 5 shows the superior performance of our method in MAS compared to other state-of-the-art methods.

Conclusions

This paper presents a novel method for multi-atlas segmentation of MR brain images with lesions, which utilizes a subspace-assisted 3D generative model to capture the subject-specific posterior normative distribution for normal recovery. The proposed method demonstrated significantly improved performance in registration and segmentation of images with lesions.

Acknowledgements

This work was supported by Shanghai Pilot Program for Basic Research—Shanghai Jiao Tong University (21TQ1400203), the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning, Key Program of Multidisciplinary Cross Research Foundation of Shanghai Jiao Tong University (YG2021ZD28, YG2023ZD22), National Natural Science Foundation of China (62001293).

References

  1. Wang H, Suh J W, Das S R, et al. Multi-atlas segmentation with joint label fusion. IEEE Trans. Pattern Anal. Mach. Intell. 2012; 35(3): 611-623.
  2. Warfield S K, Zou K H, Wells W M. Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans Med Imaging. 2004; 23(7), 903-921.
  3. Langerak T R, van der Heide U A, Kotte A N T J, et al. Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (SIMPLE). IEEE Trans Med Imaging. 2010; 29(12): 2000-2008.
  4. Sabuncu M R, Yeo B T T, Van Leemput K, et al. A generative model for image segmentation based on label fusion. IEEE Trans Med Imaging. 2010; 29(10): 1714-1729.
  5. Awate S P, Whitaker R T. Multiatlas segmentation as nonparametric regression. IEEE Trans Med Imaging. 2014; 33(9): 1803-1817.
  6. Sanroma G, Wu G, Gao Y, et al. Learning to rank atlases for multiple-atlas segmentation. IEEE Trans Med Imaging. 2014; 33(10): 1939-1953.
  7. Brett M, Leff A P, Rorden C, et al. Spatial normalization of brain images with focal lesions using cost function masking. Neuroimage. 2001; 14(2): 486-500.
  8. Liu X, Niethammer M, Kwitt R, et al. Low-rank atlas image analyses in the presence of pathologies. IEEE Trans Med Imaging. 2015; 34(12): 2583-2591.
  9. Tang Z, Ahmad S, Yap P T, et al. Multi-atlas segmentation of MR tumor brain images using low-rank based image recovery. IEEE Trans Med Imaging. 2018; 37(10): 2224-2235.
  10. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. NeurIPS. 2014; 27.
  11. Avants B B, Epstein C L, Grossman M, et al. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal. 2008; 12(1): 26-41.
  12. Aljabar P, Heckemann R, Hammers A, et al. Classifier selection strategies for label fusion using large atlas databases. In Proc Int Conf Med Image Comput Comput-Assist Intervent. 2007; 523-531.
  13. Matthews P M, Sudlow C. The UK biobank. Brain. 2015; 138(12): 3463-3465.
  14. Liew S L, Anglin J M, Banks N W, et al. A large, open source dataset of stroke anatomical brain images and manual lesion segmentations. Sci data. 2018; 5(1): 1-11.
  15. Klein A, Tourville J. 101 labeled brain images and a consistent human cortical labeling protocol. Front Neurosci. 2012; 6: 171.

Figures

Figure 1. Illustration of the overall pipeline of the proposed method.

Figure 2. Normal images recovery from synthetic tumor brain images. The proposed method achieved the minimal residual errors in normal tissue and best recovery of normal details. GT: ground truth.

Figure 3. Normal images recovery from real stroke brain images. The proposed method achieved the best performance in filtering out the lesions.

Figure 4. Registration results of a brain atlas to a real stroke brain image. The registered brain atlas using SyN showed distortions caused by lesion. Conversely, our proposed technique did not exhibit such distortions.

Figure 5. Dice indices of 31 brain regions of segmented synthetic tumor brain images using SyN+MAS, LRSD+MAS, SCOLOR+MAS and Proposed+MAS.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
0876
DOI: https://doi.org/10.58530/2024/0876