4250
Joint PET-MRI Reconstruction with Diffusion Stochastic Differential Model1Inner Mongolia University, Hohhot, China, 2Inner Mongolia Medical University, Hohhot, China, 3Shenzhen Institutes of Advanced Technology, Shenzhen, China, 4Inner Mongolia Medical University Affiliated Hospital, Hohhot, China
Synopsis
Keywords: Image Reconstruction, PET/MR, Artificial Intelligence, Joint reconstruction
Motivation: PET suffers from a low signal-to-noise ratio. Meanwhile, the k-space data acquisition process in MRI is time-consuming by PET-MRI systems.
Goal(s): We aim to accelerate MRI and improve PET image quality.
Approach: This paper proposed a novel joint reconstruction model by diffusion stochastic differential equations based on learning the joint probability distribution of PET and MRI.
Results: Compare the results underscore the qualitative and quantitative improvements our model brings to PET and MRI reconstruction, surpassing the current state-of-the-art methodologies.
Impact: Joint PET-MRI reconstruction is a challenge in the PET-MRI system. This studies focused on the relationship extends beyond edges. In this study, PET is generated from MRI by learning joint probability distribution as the relationship.
Introduction
Recent technological advancements have facilitated the fusion of PET and MRI [1]. PET-MRI allows for the simultaneous acquisition of both functional and anatomical data [2][3][4][5]. The harmonized nature of PET and MRI data makes PET-MRI an invaluable tool in various neurological studies [6]. Structural, contrast, and resolution are disparities in PET and MRI due to their fundamentally different imaging mechanisms. Nevertheless, there are statistical correlations between PET and MRI, particularly when PET employs radiotracers like fluorodeoxyglucose (FDG) [7]. PET often feature lower signal-to-noise ratios and restricted spatial resolution in visual imaging due to the inherently stochastic nature of photon emission processes [8][9][10][11]. In contrast, MRI offers high spatial resolution in anatomical imaging. However, the k-space data acquisition process is time-consuming. The common method reduces data acquisition time for upsampling. Undersampling k-space data leads to aliasing in the reconstructed images [12]. The joint reconstruction process capitalizes on the complementary information of MRI and PET. Meanwhile, the score-based diffusion model can learn the joint probability distribution of multi-modality image [13]. This study aims to develop a joint reconstruction model that effectively harnesses the complementary information residing within both modalities, thereby significantly enhancing the quality of their respective reconstructions.Method
The joint reconstruction can be described as$$\mathrm{Y} = \mathcal{D}\left(\mathrm{X}\right) + \epsilon$$
where different modalities can be written in stacked form
$$\mathrm{Y} = \begin{pmatrix}f\\g\\\end{pmatrix},\mathcal{D} = \begin{pmatrix}\mathcal{A}\\\mathcal{F}\\\end{pmatrix},\mathrm{X} = \begin{pmatrix}u\\v\\\end{pmatrix},\epsilon = \begin{pmatrix}\varsigma\\\xi\\\end{pmatrix}$$
Let $$$u$$$ and $$$v$$$ represent the PET and MRI to be reconstructed, respectively. $$$f$$$ corresponds to the observed PET data, and $$$g$$$ refers to the observed MRI data. $$$\mathcal{A}$$$ signifies the PET forward operator as the discrete Radon transform [14]. $$$\mathcal{F}$$$ stands for the MRI forward operator as the discrete Fourier transform. $$$\epsilon$$$ represents the measured noise. $$$\varsigma$$$ is associated with Gaussian noise, following a normal distribution, i.e., $$$\varsigma \sim \mathcal{N}(0, I)$$$, and $$$\xi$$$ denotes Poisson noise, i.e., $$$\xi \sim P(\lambda)$$$. By reasonable assumptions, the separation of the multi-modality likelihood can be described as
\begin{equation}
p(f,g \mid u,v) = p(f \mid u,v)p(g \mid u,v) = p(f \mid u)p(g \mid v).
\end{equation}
The posterior probability of the PET-MRI image $$$(u,v)$$$ for observed PET-MRI data $$$(f,g)$$$ can then be simplified as
\begin{equation}
p(u,v \mid f,g)\propto p(f \mid u)p(g \mid v)p(u,v) (1)
\end{equation}
We further assume a prior of the form $$$p(\mathrm{u,v)}=\exp(-\alpha \mathcal{R}(\mathrm{u,v})$$$. Maximizing (1) is equivalent to minimizing
\begin{equation}
\begin{aligned}
\mathcal{J}(u,v)&=-\log p(u,v \mid f,g)\\&=- \log p(f \mid u) - \log p(g \mid v) + \alpha \mathcal{R}(u,v)\\ &=\sum \limits_{i}^{M}((\mathcal{A}u)_{i} - \mathrm{f}_{i}\log (\mathcal{A}u)_{i}) + \frac{1}{2\sigma^{2}}\|\mathcal{F}v - g\|_{2}^{2} + \alpha \mathcal{R}(u,v)
\end{aligned}
\end{equation}
$$$(u,v)$$$ can be reconstructed when $$$\log p(u,v)$$$ was obtained by diffusion model. The proposed model as JR-diffusion comprises of the forward diffusion process and the reverse diffusion process. The forward diffusion process, noise is gradually introduced to convert a complex data distribution into a known prior distribution. It is described as
\begin{equation*}
(u_{i+1}, v_{i+1})= (u_{i}, v_{i}) + \sigma_{min} \left( \frac{\sigma_{max}}{\sigma_{min}} \right) ^{i} z, ~ i=1,2,...,N-1,
\end{equation*}
Since $$$\lbrace{\sigma_{i}\rbrace}_{i=1}^{N}$$$ is a geometric sequence, let $$\sigma_{i}=\sigma(\frac{i}{N})=\sigma_{min}\left(\frac{\sigma_{max}}{\sigma_{min}}\right)^{\frac{i-1}{N-1}}, i=1,2,...,N.$$
Conversely, in the reverse diffusion process, noise is removed to transform the prior distribution back into the original data distribution. It can obtain the reconstruction result
\begin{equation*}
(u_{i}, v_{i})= (u_{i+1}, v_{i+1})-s_{\theta}(u_{i+1},v_{i+1},t)-\mathcal{A}^{*}\left[\mathcal{I}-\frac{f_{i+1}}{\mathcal{A}u_{i+1}}\right] -\mathcal{F}^{*}(\mathcal{F}v_{i+1}-g_{i+1})+z_{i}
\end{equation*}
Fig. 1 shows the framework of the JR-diffusion.
Results
Our training dataset sourced from the Alzheimer's Disease Neuroimaging Initiative (ADNI) [15], consisted of 167 individuals and 5010 image pairs. We conducted a comparative analysis encompassing our method, the conventional Linear Parallel Level Sets (LPLS) method [16], and the supervised deep learning technique known as Joint ISTA-Net [9]. Fig. 2 illustrates the reconstruction outcomes with a Cartesian undersampling factor of 3, while Fig. 3 depicts outcomes with a factor of 5. The results emphasize both qualitative and quantitative enhancements achieved through the implementation of JR-diffusion in PET and MRI reconstruction, surpassing contemporary state-of-the-art methodologies. To substantiate the efficacy of this supplementary information, MRI contrast results for individual and joint reconstructions are presented in Fig. 4, and PET contrast results are displayed in Fig. 5. A comparison between single reconstruction MRI and joint reconstruction MRI reveals that the absence of complementary information significantly compromises the quality of the reconstruction.Conclusions
In this paper, we propose JR-diffusion, a novel model for PET and MRI joint reconstruction. It accurately characterizes the joint distribution, serving as a prior for clean image generation from undersampled data. Numerical experiments confirm JR-diffusion's superior performance in joint reconstruction accuracy compared to tradition imaging and supervised deep learning. The accuracy in learning joint probability distributions of PET and MRI primarily drives this improvement.Acknowledgements
Taofeng Xie and Zhuoxu Cui contributed equally to this work. This work was partially supported by the National Natural Science Foundation of China (61871373, 62271474, 81830056, U1805261, 81729003, 81901736, 12026603, and 81971611), the National Key R&D Program of China (2023YFB3811400), the Strategic Priority Research Program of Chinese Academy of Sciences (XDB25000000 and XDC07040000), the High-level Talent Program in Pearl River Talent Plan of Guangdong Province (2019QN01Y986), the Key Laboratory for Magnetic Resonance and Multimodality Imaging of Guangdong Province (2020B1212060051), the Science and Technology Plan Program of Guangzhou (202007030002), the Key Field R&D Program of Guangdong Province (2018B030335001), the Shenzhen Science and Technology Program, Grant Award (JCYJ20210324115810030), and the Shenzhen Science and Technology Program (Grant No. KQTD20180413181834876, and KCXF20211020163408012).References
[1] Catana, Ciprian, Alexander R. Guimaraes, and Bruce R. Rosen. "PET and MR imaging: the odd couple or a match made in heaven?." Journal of Nuclear Medicine 54.5 (2013): 815-824.
[2] Catana, Ciprian, et al. "PET/MRI for neurologic applications." Journal of nuclear medicine 53.12 (2012): 1916-1925.
[3] Cherry, Simon R. "Multimodality in vivo imaging systems: twice the power or double the trouble?." Annu. Rev. Biomed. Eng. 8 (2006): 35-62.
[4] Townsend, D. W. "Multimodality imaging of structure and function." Physics in Medicine & Biology 53.4 (2008): R1.
[5] Ehrhardt, Matthias J., et al. "Joint reconstruction of PET-MRI by parallel level sets." 2014 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC). IEEE, 2014.
[6] Chen, Zhaolin, et al. "From simultaneous to synergistic MR‐PET brain imaging: A review of hybrid MR‐PET imaging methodologies." Human brain mapping 39.12 (2018): 5126-5144.
[7] Torrado-Carvajal A, Catana C. PET/MRI: technical and methodological aspects[M]//Clinical PET/MRI. Academic Press, 2023: 1-33.
[8] Øen, Silje Kjærnes, et al. "Image quality and detectability in Siemens Biograph PET/MRI and PET/CT systems—a phantom study." EJNMMI physics 6 (2019): 1-16.
[9] Yang Y, Wang N, Yang H, et al. Model-driven deep attention network for ultra-fast compressive sensing MRI guided by cross-contrast MR image[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23. Springer International Publishing, 2020: 188-198.
[10] Lindemann, Maike E., et al. "Towards fast whole-body PET/MR: Investigation of PET image quality versus reduced PET acquisition times." PLoS One 13.10 (2018): e0206573.
[11] Kazantsev, Daniil, et al. "A novel technique to incorporate structural prior information into multi-modal tomographic reconstruction." Inverse Problems 30.6 (2014): 065004.
[12] Lustig, Michael, David Donoho, and John M. Pauly. "Sparse MRI: The application of compressed sensing for rapid MR imaging." Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 58.6 (2007): 1182-1195.
[13] Song, Yang, et al. "Score-based generative modeling through stochastic differential equations." arXiv preprint arXiv:2011.13456 (2020).
[14] Burger, Martin, et al. "Total variation regularization in measurement and image space for PET reconstruction." Inverse Problems 30.10 (2014): 105003.
[15] Jack Jr, Clifford R., et al. "The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods." Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine 27.4 (2008): 685-691.
[16] Ehrhardt, Matthias J., et al. "Joint reconstruction of PET-MRI by exploiting structural similarity." Inverse Problems 31.1 (2014): 015001.
Figures
