2797
Deep image-pass filter for dynamic MRI reconstruction1Harbin Institute of Technology, Harbin, China, China
Synopsis
Keywords: Machine Learning/Artificial Intelligence, Machine Learning/Artificial Intelligence, deep image prior, unsupervised learning, deep image-pass filter, dynamic MRI
Motivation: For accelerating dynamic MRI, the gradient flow in the unsupervised deep image prior (DIP) methods lacks a direct pathway from the image domain to the network parameters, resulting in suboptimal performance.
Goal(s): To propose a novel approch to address DIP's drawback for robust dynamic MRI reconstruction.
Approach: Deep image-pass filter is proposed, replacing the random noise input of DIP with learnable image and constraint input consistent with output to establish an efficient gradient pathway from image domain.
Results: Experimental results in the reconstruction of both long-axis and short-axis dynamic cardiac cine MRI demonstrate that DIPF outperforms DIP and other state-of-the-art unsupervised methods.
Impact: We have introduced a new formulation for unsupervised MRI reconstruction, which will drive a series of research around this paradigm, including network architecture design, reconstuction models combining DIPF with other data priors, and more.
Introduction
Recently, Ulyanov et al. 1 proposed that the convolutional neural network (CNN) has the ability to retain the intrinsic structure of images in itself. This underlying image-hold feature, which is usually termed Deep image prior (DIP), provides a new perspective for image reconstruction. DIP has been exploited in medical imaging7, also in dynamic MRI reconstruction3.With deep understanding of the DIP, we find that in the case of MRI reconstruction, the training of DIP is guided by undersampled Fourier points, lacking comprehensive representation of individual pixel information. These points only capture aliased image information for each pixel. This discrepancy between domains may considerably curtail the performance potential of DIP.
To address this issue, we propose a novel deep image-pass filter (DIPF) based on DIP for accelerating dynamic MRI. Specifically, our methodology involves configuring the network input as a trainable image initialized by the zero-filled reconstruction. By enforcing consistency between the input and output of the network within the loss function, we establish a pathway for direct gradient flow from the image domain to the network parameters. This design essentially transforms the network into a filter that selectively permits the passage of undistorted images.
Methods
The DIPF model for dynamic MRI reconstruction is formulated as follows,$$\min_{\mathcal{X}, \theta} \left\| \mathsf{A}(\mathcal{X}) - \mathbf{y} \right\|_2^2, \quad \text{s.t.} \quad \mathcal{X} = f_\theta(\mathcal{X}).$$
The main differences between our proposed DIPF and DIP model are twofold. First, the input of the network no longer is random noise or prior information, but the image itself. The image $$$\mathcal{X}$$$ is also learnable instead of fixed as in DIP. Second, the output of the network is subjected to be consistent with the input image. Ideally, when the model optimizes to its best, it yields a clean image X, and the output after the image losslessly passes through the network. Throughout optimization, the network tends to enable X, which resembles the clean image, to pass through without distortion. Significant alterations to artifact-containing images lead to non-compliance with the constraints. Consequently, the network functions as a filter, permitting the passage of clean images while altering those that do not meet the criteria.
The constrained problem can be attacked using a quadratic penalty approach, i.e., by solving
$$\min_{\mathcal{X}, \theta} \left\| \mathsf{A}(f_\theta(\mathcal{X})) - \mathbf{y} \right\|_2^2 + \lambda \left\| \mathcal{X} - f_\theta(\mathcal{X}) \right\|_F^2. $$
The Adam optimizer6 or other gradient-based methods could be used to solve this problem, or in other words, to train the CNN $$$f_\theta$$$.
In DIPF, the gradient for the output of the network regarding the loss function is as follows,
$$\frac{ \partial \mathscr{L} }{ \partial f_\theta(\mathcal{X}) } = \mathsf{A}^H \left( \mathsf{A}(f_\theta(\mathcal{X})) - \mathbf{y} \right) + 2 \lambda \left( f_\theta(\mathcal{X}) - \mathcal{X} \right).$$
The first term provides the gradient flow from the sampled k-space points in the Fourier domain, thereby directing the network to generate the image that is consistent with the sampled data. The second term facilitates the gradient flow from the image domain, which is the key to our method. A direct pathway between image and network parameters is established. Consequently, the image-sensitive CNN operates more effectively and efficiently, adept at learning and preserving the image structure.
Results and discussion
We utilized long- and short-axis cardiac cine MRI data from the public-available dataset, OCMR2, for experiments. We compared our proposed DIPF with DIP1, TDDIP3, NeRP4, and LR5. The DIP method used the same network as our DIPF for fair comparison. TDDIP and NeRP are two state-of-the-art unsupervised methods for accelerating dynamic MRI. LR denotes the low-rank method.The reconstructed PSNRs in multiple sampling cases for the long- and short-axis data were reported in Fig.1. Fig.2 shows the reconstructed images of the long-axis data. The DIPF achieved up to 10 dB improvement over DIP, which illustrated the effectiveness and efficacy of the proposed method. The DIPF provides sharper edges and more details compared with other methods, demonstrating its superiority in robust dynamic MRI reconstruction.
Conclusion
Considering the unique characteristics of k-space acquisition in MRI, we conducted an in-depth exploration into the gradient flow and underlying principles of DIP. A novel unsupervised formulationg is proposed and termed DIPF. The core concept of DIPF involves setting the network's input as a learnable image rather than random noise. By incorporating a consistency constraint between input and output within the loss function, we established a direct gradient flow pathway from the image domain to the network parameters. Essentially, this design transforms the network into a filter, selectively allowing the passage of undistorted images. Experimental results demonstrate that DIPF outperforms DIP and other state-of-the-art unsupervised methods in the reconstruction of both long-axis and short-axis dynamic MRI images.Acknowledgements
This work is supported by Natural Science Foundation of Heilongjiang YQ2021F005 and China NSFC 62371167.References
1. Ulyanov D, Vedaldi A, Lempitsky V. Deep image prior. Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 9446-9454.
2. Chen C, Liu Y, Schniter P, et al. OCMR (v1. 0)--Open-Access Multi-Coil k-Space Dataset for Cardiovascular Magnetic Resonance Imaging. arXiv preprint arXiv:2008.03410, 2020.
3. Yoo J, Jin K H, Gupta H, et al. Time-dependent deep image prior for dynamic MRI. IEEE Transactions on Medical Imaging, 2021, 40(12): 3337-3348.
4. Shen L, Pauly J, Xing L. NeRP: implicit neural representation learning with prior embedding for sparsely sampled image reconstruction. IEEE Transactions on Neural Networks and Learning Systems, 2022.
5. Lingala S G, Hu Y, DiBella E, et al. Accelerated dynamic MRI exploiting sparsity and low-rank structure: kt SLR. IEEE transactions on medical imaging, 2011, 30(5): 1042-1054.
6. Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
7. Gong K, Catana C, Qi J, et al. PET image reconstruction using deep image prior. IEEE transactions on medical imaging, 2018, 38(7): 1655-1665.
Figures
