Introduction

Image priors captured through deep learning have enabled substantial improvements over traditional methods in accelerated MRI reconstruction^1-16. An emerging framework uses task-agnostic priors based on generative models for improved generalization and image quality^17-19, and recent diffusion models in this framework have been particularly promising^20-24. However, conventional diffusion priors define a forward process via Gaussian noise addition, so image formation is expressed as a denoising transformation from random noise onto MRI data²⁰. In contrast, the reconstruction task requires a transformation from undersampled to fully-sampled data¹. Since the learned and required transformations are not aligned, conventional diffusion priors can suffer from limited optimization efficiency, and, as a result, yield suboptimal reconstruction performance²¹.

To address this challenge, here we introduce a novel MRI reconstruction method named Fourier-constrained diffusion Schrodinger bridges (FDB). FDB integrates task-relevant information into its prior by building a novel diffusion bridge that directly transforms between undersampled and fully-sampled data. To do this, FDB’s forward process uses a stochastic degradation operator gradually removing spatial-frequency components, and its reverse process progressively dealiases undersampled data. We demonstrate that FDB yields superior performance against previous traditional, adversarial and diffusion methods.

Methods

FDB: Diffusion bridges are a recent approach in generative modeling that relax the Gaussian assumption in conventional diffusion priors to capture task-relevant transformations for inverse problems²⁵. Recent diffusion bridges in computer vision are either unconstrained²⁵ (limiting learning efficiency) or constrained via deterministic degradation operators^26,27 (limiting generalization capabilities). Instead, FDB uniquely leverages a stochastic operator in Fourier domain to build a novel constrained diffusion bridge that transforms between moderately undersampled and fully-sampled MRI data.

Forward Process: A random frequency-removal operator is cast in FDB via a k-space mask $$$\mathcal{K}_t$$$ at time step $$$t$$$ selecting $$$n$$$ unique frequency components for removal. A peripheral-to-central selection order is enforced according to a point process:
$${r_i}\sim U\left[0,r_{\mathrm{max}}\right];\quad{\phi_i}\sim U\left[0,2\pi\right)\\ \mbox{s.t. }k(r_i,\phi_i)\notin\left\{k:\bigcup\limits_{\tau=1}^{t-1}\mathrm{diag}(\mathcal{K}_{\tau})=1\right\}\mbox{ and }r_i>\bar{r}_t$$
where $$$i\in[1\ n]$$$ is the selected-component index in $$$\mathcal{K}_t$$$, $$$r_i,\phi_i$$$ are polar coordinates of Cartesian k-space points, $$$U$$$ is a uniform distribution, $$$\bar{r}_t$$$ is a radial threshold scheduled as:
$$\bar{r}_t=r_{\mathrm{max}}-\frac{t}{T_f}(r_{\mathrm{max}}-r')$$
$$$r'\approx r_{\mathrm{max}}\sqrt{R'}$$$ to ensure that $$$R'$$$-fold undersampling is achieved at $$$t=T_f$$$.

The degraded images $$$x_t$$$ obtained via frequency-removal and noise-addition operators are given as (Fig.1):
$$x_t={\alpha_t}\Big\{\mathcal{F}^{-1}\big(\prod_{\tau=1}^{t}{\left(\mathrm{I}-\mathcal{K}_{\tau}\right)}\big)\circledast x_{0}\Big\}+{\sigma_t}z,\\ x_t={\alpha_t}(\bar{\kappa}_t\circledast x_{0})+{\sigma_t}z,\\ x_t={\alpha_t}C_tx_{0}+{\sigma_t}z$$
where $$$\mathcal{F}^{-1}$$$ denotes inverse Fourier transform, $$$\alpha_t,\sigma_t$$$ are scale parameters, $$$\circledast$$$ denotes convolution, $$$x_0=S^H\mathcal{F}^{-1}\{X_0\}$$$ denote clean images derived from fully-sampled data ($$$S^H:$$$ Hermitian of coil sensitivities), $$$z$$$ is a standard complex variable. The corruption matrix $$$C_t$$$ is the block Toeplitz form of convolution with $$$\bar{\kappa}_t=\kappa_t\circledast…\circledast\kappa_1$$$, $$$\kappa_t=\mathcal{F}^{-1}(\mathrm{I}-\mathcal{K}_t)$$$.

Reverse Process: Reverse steps for FDB’s novel diffusion process can be parameterized via a neural network $$$G_{\theta}$$$ that estimates a clean image²⁸: $$$\tilde{x}_0=G_{\theta}(x_t,t)$$$. The corresponding training objective is:
$$\min_{\theta}\mathbb{E}_{t,x_0,x_t}[\|(G_{\theta}(x_t,t)-x_0)\|^2]$$
where $$$\mathbb{E}$$$ is expectation over $$$t\in U(0,T_f)$$$.

For reconstruction with a trained FDB prior, we present a sampling algorithm initiated at $$$T_r=\lfloor T_f \frac{R}{R'}\rfloor$$$ with $$$x_{T_r}$$$ taken as the zero-filled reconstruction of undersampled data at $$$R$$$ (Fig.2). Reverse diffusion sampling at $$$t$$$ can be derived as:
$$\dot{x}_{t-1}\,=\,\,x_t+\underbrace{(\alpha_{t-1}-\alpha_t)\mathbb{E}[C_{t}]\tilde{x}_0}_\text{frequency imputation}\\+\underbrace{(\sigma_{t}^2-\sigma_{t-1}^2)\frac{\alpha_t\mathbb{E}[C_{t}]\tilde{x}_0-x_t}{\sigma^2_t}}_\text{denoising}+\sqrt{\sigma_{t}^2-\sigma_{t-1}^2}z$$
Reverse sampling is followed by a data-fidelity projection:
$${x}_{t-1}=\dot{x}_{t-1}+A^H(y-A\dot{x}_{t-1})$$
where $$$A=M\mathcal{F}S$$$ ($$$M$$$: sampling mask).

Analyses: Analyses were conducted on single-coil (T₁,T₂,PD) data in IXI (https://brain-development.org/ixi-dataset/) and multi-coil (T₁,T₂,FLAIR) data in fastMRI³⁰. The (training,validation,test) sets contained (240,60,120) non-overlapping subjects. Each set contained mixed data across contrasts. 2D variable-density undersampling¹ was employed at R=4-8. Training was performed with Adam optimizer, 50 epochs, 0.0002 learning rate.

Results

FDB was comparatively demonstrated against a traditional method (LORAKS³¹), task-specific (rGAN¹⁵) and task-agnostic (GAN_prior¹⁷) adversarial methods, task-specific (CDiffMR²⁷) and task-agnostic (DDPM²⁴) diffusion methods. Performance metrics are listed in Fig.3. On average, FDB achieves improvements of 3.7dB PSNR, 1.2% SSIM at R=4, 1.9dB PSNR, 1.1% SSIM at R=8 over the closest competitor. These improvements are visually apparent in representative reconstructions in Fig.4. While competing methods show residual noise/aliasing or suffer from loss of detailed tissue structures, FDB achieves relatively high spatial acuity and low noise/artifact levels. Generalization performance was examined by considering domain shifts between training and test sets. Performance metrics in Fig.5 indicate that FDB is reliable against domain shifts in sampling density and acceleration rate.

Discussion

Here we introduced FDB, the first diffusion bridge for accelerated MRI reconstruction to our knowledge, that transforms between undersampled and fully-sampled acquisitions. Unlike common diffusion methods, FDB is trained to progressively correct task-relevant degradations due to undersampling, so its reverse-diffusion transformation shows better alignment with the required transformation for MRI reconstruction. Therefore, FDB holds great promise for improving the utility of accelerated MRI reconstruction.

Acknowledgements

This work was supported in part by a TUBITAK 1001 Grant No. 121E488, and by TUBA GEBIP 2015 and BAGEP 2017 fellowships.

References

1. Lustig, M., Donoho, D., Pauly, J.M., Sparse MRI The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, vol. 58, no. 6, pp. 1182–1195 (2007).

2. Haldar, J.P., Hernando, D., Liang, Z.P., Compressed-sensing MRI with random encoding. IEEE transactions on Medical Imaging 30(4), 893–903 (2010).

3. Qin, C., Schlemper, J., Caballero, J., Price, A.N., Hajnal, J.V., Rueckert, D., Convolutional recurrent neural networks for dynamic mr image reconstruction. IEEE transactions on medical imaging 38(1), 280–290 (2018).

4. Wang, S., Su, Z., Ying, L., Peng, X., Zhu, S., Liang, F., Feng, D., Liang, D., Accelerating magnetic resonance imaging via deep learning. In IEEE 13th International Symposium on Biomedical Imaging (ISBI). pp. 514–517 (2016).

5. Hammernik H., Klatzer T., Kobler R., Recht M.P., Sodickson D.K., Pock T., and Knoll F., Learning a variational network for reconstruction of accelerated MRI data. Magnetic Resonance in Medicine, vol. 79, no. 6, pp. 3055–3071 (2018).

6. Mardani, M., Gong, E., Cheng, J.Y., Vasanawala, S., Zaharchuk, G., Xing, L., Pauly, J.M., Deep generative adversarial neural networks for compressive sensing MRI. IEEE Transactions on Medical Imaging 38(1), 167–179 (2019).

7. Zhu, B., Liu, J.Z., Rosen, B.R., Rosen, M.S., Image reconstruction by domain transform manifold learning. Nature 555(7697), 487–492 (2018).

8. Akçakaya, M, Moeller, S, Weingärtner, S, Uğurbil, K., Scan-specific robust artificial-neural-networks for k-space interpolation (RAKI) reconstruction Database-free deep learning for fast imaging. Magnetic Resonance in Medicine 81, 439–453, (2019).

9. Aggarwal, H.K., Mani, M.P., Jacob, M., MoDL Model-Based deep learning architecture for inverse problems. IEEE Transactions on Medical Imaging 38(2), 394–405 (2019).

10. Peng, X., Sutton, B.P., Lam, F., Liang, Z.P., DeepSENSE: Learning coil sensitivity functions for SENSE reconstruction using deep learning. Magnetic Resonance in Medicine, 87(4), 1894–1902 (2020).

11. Küstner, T., Fuin, N., Hammernik, K., Bustin, A., Qi, H., Hajhosseiny, R., Masci, P. G., Neji, R., Rueckert, D., Botnar, R. M., & Prieto, C., CINENet: deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions. Scientific Reports, 10(1) (2020).

12. Polak, D., Cauley, S., Bilgic, B., Gong, E., Bachert, P., Adalsteinsson, E., Setsompop, K., Joint multi-contrast variational network reconstruction (jVN) with application to rapid 2D and 3D imaging. Magnetic Resonance in Medicine, 84(3), 1456–1469 (2020).

13. Kwon, K., Kim, D., Park, H., A parallel MR imaging method using multilayer perceptron. Medical Physics 44(12), 6209–6224 (2017).

14. Eo, T., Jun, Y., Kim, T., Jang, J., Lee, H. J., Hwang, D., KIKI-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magnetic Resonance in Medicine, 80(5), 2188–2201 (2018).

15. Dar, S.U., Yurt, M., Shahdloo, M., Ildız, M.E., Tınaz, B., Cukur, T., Prior-guided image reconstruction for accelerated multi-contrast MRI via generative adversarial networks. IEEE Journal of Selected Topics in Signal Processing 14(6), 1072–1087 (2020).

16. Liu, F., Feng, L., & Kijowski, R., MANTIS: Model-Augmented Neural neTwork with Incoherent k-space Sampling for efficient MR parameter mapping. Magnetic Resonance in Medicine, 82(1), 174–188 (2019).

17. Narnhofer, D., Hammernik, K., Knoll, F., Pock Dominik Narnhofer, T., Pock, T., Inverse GANs for accelerated MRI reconstruction. SPIE Medical Imaging, 11138(9), 381–392 (2019).

18. Liu, Q., Yang, Q., Cheng, H., Wang, S., Zhang, M., Liang, D., Highly undersampled magnetic resonance imaging reconstruction using autoencoding priors. Magnetic Resonance in Medicine, 83(1), 322–336 (2020).

19. Korkmaz, Y., Dar, S.U.H., Yurt, M., Ozbey, M., Cukur, T., Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers. IEEE Transactions on Medical Imaging, 41(7), 1747–1763 (2022).

20. Jalal, A., Arvinte, M., Daras, G., Price, E., Dimakis, A. G., Tamir, J., Robust Compressed Sensing MRI with Deep Generative Priors. Advances in Neural Information Processing Systems, 34, 14938–14954 (2021).

21. Chung, H., Ye, J.C., Score-based diffusion models for accelerated MRI. Medical Image Analysis, 80, 102479 (2022).

22. Xie, Y., Li, Q., Measurement-conditioned denoising diffusion probabilistic model for under-sampled medical image reconstruction. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VI. pp. 655–664.

23. Luo, G., Blumenthal, M., Heide, M., & Uecker, M., Bayesian MRI reconstruction with joint uncertainty estimation using diffusion models. Magnetic Resonance in Medicine, 90(1), 295–311 (2023).

24. Gungor A, Dar, S.U.H., Ozturk, S, Korkmaz, Y., Bedel, H.A., Elmas, G., Ozbey, M., Cukur, T., Adaptive diffusion priors for accelerated MRI reconstruction. Medical Image Analysis 88:102872 (2023).

25. Liu G., Vahdat A., Huang D., Theodorou E.A., Nie W., Anandkumar A., I2SB: Image-to-image Schrödinger bridge. arXiv:2302.05872 (2023).

26. Chung H., Kim J., Ye, J.C., Direct diffusion bridge using data consistency for inverse problems. arXiv:2305.19809 (2023).

27. Huang J., Aviles-Rivero A., Schönlieb C., Yang G., Cdiffmr: Can we replace the gaussian noise with k-space undersampling for fast mri? arXiv:2306.14350,143 (2023).

28. Daras G., Delbracio M., Talebi H., Dimakis A.G., Milanfar P., Soft diffusion: Score matching for general corruptions. arXiv:2209.05442 (2022).

29. Uecker, M., Lai, P., Murphy, M. J., Virtue, P., Elad, M., Pauly, J. M., Vasanawala, S. S., Lustig, M., ESPIRiT--an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magnetic Resonance in Medicine, 71(3), 990–1001 (2014).

30. Knoll, F., Zbontar, J., Sriram, A., Muckley, M.J., Bruno, M., Defazio, A., Parente, M., Geras, K.J., Katsnelson, J., Chandarana, H., Zhang, Z., Drozdzal, M., Romero, A., Rabbat, M., Vincent, P., Pinkerton, J., Wang, D., Yakubova, N., Owens, E., Zitnick, C.L., Recht, M.P., Sodickson, D.K., Lui, Y.W., fastMRI: A publicly available raw k-space and DICOM dataset of knee images for accelerated MR image reconstruction using machine learning. Radiology: Artificial Intelligence 2(1), e190007 (2020).

31. Haldar, J. P., Zhuo, J., P-LORAKS: Low-Rank Modeling of Local k-Space Neighborhoods with Parallel Imaging Data. Magnetic Resonance in Medicine, 75(4), 1499 (2016).