Introduction

Low-dimensional subspace models have been shown to be effective in improving the speed, resolution, and SNR tradeoffs for MRSI^1-5. Particularly, the SPICE framework exploits the partial separability of high-dimensional MRSI signals and represents each voxel FID as linear combination of a small set of temporal basis functions, dramatically reducing the dimensionality of the imaging problem. While the subspace model has been highly effective, investigations into using effective complementary spatial constraints have been limited¹. Recently, with the advances in deep neural network (NN) based image presentations/generators, deep image prior has been used to improve the spatial coefficient estimation in SPICE⁶. However, it has been shown that the unsupervised deep image prior suffered from noise overfitting, and other GAN-based priors may not have sufficient representation power (inaccurate reconstruction even with full images as the data)^7,8. We propose here a new method to enhance the subspace MRSI reconstruction that uses a subject-specific StyleGAN-based prior as an effective constraint for the unknown spatial coefficients^11,12. We validated the sufficiency of the learned prior in representing images at different contrasts and demonstrated improved subspace reconstruction using the prior. The proposed method may be integrated with more advanced nonlinear spectral prior(s) to create more advanced MRSI reconstruction methods.

Theory and Methods

Subject-adaptive StyleGAN prior
While the potential of deep generative models for image reconstruction has been shown^7-10, there are two major issues for using them for MRSI reconstruction: (1) insufficient image representation accuracy for many generative models (including GAN) and (2) the lack of high-SNR, high-resolution in vivo MRSI training data. We address the first issue by using the powerful multi-scale representation model StyleGAN v2^11-13, and the second issue by a strategy that adapts pretrained StyleGAN to subject-specific image manifold. Intermediate layer optimization (ILO) method¹⁴ (better exploiting the latent-space structures) is also used to address the challenging GAN inversion problem during the adaptation step (further reducing the representation error) and the reconstruction step. Specifically, given a StyleGAN $$$G_{\boldsymbol{\theta_{p}}}$$$ with parameters $$$\boldsymbol{\theta_{p}}$$$ that was pretrained using 100K T1w&T2w images from HCP database¹⁵, we adapt its representation to a subject-specific high-resolution reference image $$$\boldsymbol{x_p}$$$ by solving:$$\widehat{\boldsymbol{w}}, \widehat{\boldsymbol{\theta}}=\arg\min _{\boldsymbol{w}, \boldsymbol{\theta}}\left\|\boldsymbol{\Omega}_{\text {brain }} G_{\boldsymbol{\theta}}(\boldsymbol{w})-\boldsymbol{\Omega}_{\text {brain }} \boldsymbol{x}_{\boldsymbol{p}}\right\|_{2}^{2}+\alpha\left\|\boldsymbol{\theta}-\boldsymbol{\theta}_{\boldsymbol{p}}\right\|_{2}^{2}, (1)$$
where $$$\boldsymbol{w}$$$ denotes the latent variables of GAN, and $$$\boldsymbol{\Omega}_{\text {brain}}$$$ the brain mask (as MRSI only reconstructs the brain region). To solve Eq.(1), the latent $$$\widehat{\boldsymbol{w}}$$$ was determined by ILO with fixed $$$\boldsymbol{\theta}_{\boldsymbol{p}}$$$, and then the $$$\boldsymbol{\theta}$$$ is updated with fixed latent $$$\widehat{\boldsymbol{w}}$$$ . The regularization term constrains that the adapted manifold does not deviate dramatically from the one captured by the pretrained StyleGAN.

Subspace MRSI reconstruction using adapted StyleGAN prior
We propose the following subspace reconstruction formulation using the adapted StyleGAN prior as constraints for the unknown spatial coefficients:
$$\widehat{\boldsymbol{U}}, \widehat{\boldsymbol{w}}_{\text {rec }}=\arg\min _{\boldsymbol{U}, \boldsymbol{w}}\left\|\mathcal{F}_{B}\left(\left(\boldsymbol{\Phi}_{0} \odot \boldsymbol{U}\right) \boldsymbol{V}\right)-\boldsymbol{d}\right\|_{2}^{2}+\frac{\rho}{2}\left\|\boldsymbol{U}-\boldsymbol{\Omega}_{\text {brain }} G_{\widehat{\boldsymbol{\theta}}}(\boldsymbol{w})\right\|_{2}^{2}+\lambda\left\|\boldsymbol{D}_{\boldsymbol{w}} \boldsymbol{U} \boldsymbol{V}\right\|_{2}^{2}, (2)$$
where $$$\mathcal{F}_{B}$$$ represents the Fourier encoding operator with B₀ inhomogeneity effects and $$$\boldsymbol{d}$$$ the noisy (k,t)-space data (assuming nuisance signals removed). The phase $$$\boldsymbol{\Phi}_{0}$$$ was initialized by a SPICE reconstruction with a strong spatial regularization (smooth phase assumption). The first regularization enforces the StyleGAN prior and the second is a mild weighted $$$l_{2}$$$ penalty with an edge-weighted finite-difference operator $$$\boldsymbol{D}_{\boldsymbol{w}}$$$ .

We solved Eq.(2) by first determining $$$\widehat{\boldsymbol{w}}_{\text {rec }}$$$ as:
$$\widehat{\boldsymbol{w}}_{\text {rec }}=\arg\min _{\boldsymbol{w}}\left\|\mathrm{~A}\left(\boldsymbol{\Omega_{\text {brain }}} G_{\widehat{\boldsymbol{\theta}}}(\boldsymbol{w})\right)-\left|\boldsymbol{U^{0}}\right|\right\|_{2}^{2}, (3)$$
and update $$$\boldsymbol{U}$$$ as:
$$\widehat{\boldsymbol{U}}=\arg\min _{\boldsymbol{U}}\left\|\mathcal{F}_{B}\left(\left(\boldsymbol{\Phi}_{\mathbf{0}} \odot \boldsymbol{U}\right) \boldsymbol{V}\right)-\boldsymbol{d}\right\|_{2}^{2}+\frac{\rho}{2}\left\|\boldsymbol{U}-\boldsymbol{\Omega}_{\boldsymbol{b r a i n}} G_{\widehat{\boldsymbol{\theta}}}\left(\widehat{\boldsymbol{w}}_{\boldsymbol{r e c}}\right)\right\|_{2}^{2}+\lambda\left\|\boldsymbol{D}_{\boldsymbol{w}} \boldsymbol{U} \boldsymbol{V}\right\|_{2}^{2}. (4)$$
The GAN inversion subproblem in (3) is solved by ILO with an $$$\mathrm{~A}$$$ accounting for general resolution differences between the generated images and the spatial coefficients. The subproblem in (4) is a GAN prior constrained linear least-squares. The final spatial coefficients were constructed as: $$$\widehat{\boldsymbol{U}}_{\text {rec }}=\boldsymbol{\Phi}_{\mathbf{0}} \odot \widehat{\boldsymbol{U}}$$$.

Results

In vivo data were acquired to evaluate the proposed method (with IRB approval). High-resolution MRSI data were acquired from healthy volunteers using the sequence from Wang et al.¹⁶ with TR/TE=1100/30ms, matrix size=64x64x12, FOV=220x220x64mm³. Interleaved water navigator images (WatNav) were also available from the same data.

We first validated the sufficiency of the adapted StyleGAN. Accurate reconstruction of subject-specific T1w images (from the same session) were produced by the StyleGAN representation (Fig.2, based on both visual inspection and relative $$$l_{2}$$$ errors), which can also be observed for the WatNav images from the MRSI data. Both the original SPICE and the proposed method were used for reconstruction (Fig.3). Less artifacts are observed in the reconstructed metabolite maps from the propose method.

A simulation was performed to further validate that the proposed model can flexibly adapt to contrast changes thus with minimal bias. The ground truth spatiotemporal images were generated from the WatNav images with spatiotemporal interpolation and temporal basis determined by SVD of the ground truth. The SPICE and proposed reconstructions were performed on simulated data with added noise. Fig.4 shows that the proposed method achieved a higher SNR and lower error than the original SPICE method.

Conclusion

We proposed a novel MRSI reconstruction method that combines subspace modeling and a subject-adaptive generative image prior. The adapted GAN prior exhibited sufficient representation capability for the spectroscopic images and achieved improved reconstruction as demonstrated by experimental data.

Acknowledgements

This work was supported in part by NSF-CBET-1944249 and NIH-NIBIB-1R21EB029076A.

References

1. Lam, Fan, et al. "A subspace approach to high‐resolution spectroscopic imaging." Magnetic resonance in medicine 71(2014): 1349-1357.

2. Lam, Fan, et al. "High‐resolution 1H‐MRSI of the brain using SPICE: data acquisition and image reconstruction." Magnetic resonance in medicine 76 (2016): 1059-1070.

3. Ma, Chao, et al. "High‐resolution 1H‐MRSI of the brain using short‐TE SPICE." Magnetic resonance in medicine 77 (2017): 467-479.

4. Guo, Rong, et al. "Simultaneous metabolic and functional imaging of the brain using SPICE." Magnetic resonance in medicine 82 (2019): 1993-2002.

5. Kasten, Jeffrey, et al. "Data-driven MRSI spectral localization via low-rank component analysis." IEEE transactions on medical imaging 32 (2013): 1853-1863.

6. Kuang, Gong, et al. "High Resolution MR Spectroscopic Imaging Using Deep Image Prior Constrained Subspace Modeling." ISMRM 2020.

7. Bora, Ashish, et al. "Compressed sensing using generative models." International Conference on Machine Learning. PMLR, 2017.

8. Ulyanov, Dmitry, et al. "Deep image prior." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

9. Hussein, Shady Abu, et al. "Image-adaptive GAN based reconstruction." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 04. 2020.

10. Narnhofer, Dominik, et al. "Inverse GANs for accelerated MRI reconstruction." Wavelets and Sparsity XVIII. Vol. 11138. International Society for Optics and Photonics, 2019.

11. Karras, Tero, et al. "A style-based generator architecture for generative adversarial networks." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.

12. Karras, Tero, et al. "Analyzing and improving the image quality of stylegan." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.

13. Feng, Ruili, et al. "Understanding Noise Injection in GANs." International Conference on Machine Learning. PMLR, 2021.

14. Daras, Giannis, et al. "Intermediate layer optimization for inverse problems using deep generative models." International Conference on Machine Learning. PMLR, 2021.

15. Van Essen, David C., et al. "The WU-Minn human connectome project: an overview." Neuroimage 80 (2013): 62-79.

16. Wang, Zepeng, et al. “High-Resolution, 3D Multi-TE 1H MRSI Using Fast Spatiospectral Encoding and Subspace Imaging” Magnetic resonance in medicine 00(2021): 1– 16