2800
Robust Magnetic Resonance Reconstruction by Alternating Deep Low-Rank Approach1Department of Electronic Science, Fujian Provincial Key Laboratory of Plasma and Magnetic Resonance, Xiamen University, Xiamen, China, 2College of Physics and Information Engineering, Fuzhou University, Fuzhou, China, 3Department of Applied Marine Physics and Engineering, Xiamen University, Xiamen, China, 4School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, China, 5Institute of Artificial Intelligence, Xiamen University, Xiamen, China
Synopsis
Keywords: Machine Learning/Artificial Intelligence, Image Reconstruction
Motivation: Magnetic resonance reconstruction by deep learning is heavily compromised due to the mismatch between the training and target data, such as the sampling rate of undersampling, the organ and the contrast of imaging.
Goal(s): Reliablely reconstruct magnetic resonance signal in multiple scenes by one trained deep learning model
Approach: Alternating Deep Low-Rank, which combines deep learning solvers and classic low-rank optimization solvers.
Results: Compared with state-of-the-art deep learning methods HDSLR and ODLS, one ADLR trained by coronal PDw knee can provide a lower reconstruction error by about 10% in coronal PDw knees, 15% in sagittal PDw knees, and 30% in axial T2w brains.
Impact: The proposed ADLR can effectively alleviate the drop in reconstruction quality due to the mismatches of attributes between training and target signals of the MR imaging or MR spectroscopy.
Introduction
Undersampling can accelerate the magnetic resonance (MR) signal acquisition but lead to artifacts. Utilizing the low-rank prior originated from exponential model of MR spectroscopy1 and coils correlation of MR imaging2, signals can be usually reconstructed by minimizing the nuclear norm. However, singular value decomposition results in slow computation.Deep learning2-5 is a prominent technology that offers high computation speed. Notably, deep learning is trained on the dataset of MR signals that share some same properties, such as the sampling rate, the organ and contrast of imaging. For one trained deep learning method, mismatched properties between training and target signals usually compromise the reconstruction.
To address this issue, we propose an Alternating Deep Low-Rank (ADLR), which combines deep learning solvers and classic optimization solvers. Experiments demonstrate that ADLR can effectively mitigate the mismatch issue and achieve lower reconstruction errors than state-of-the-art methods.
Method
For a multi-coils 2D MRI k-space signal $$$\mathcal{K} \in \mathbb{C}^{A \times Z \times C}$$$, A, Z and C denote the length of frequency encoding, phase encoding and number of coils, respectively. Inspired by 1D scheme2, the signal is split into A rows of k-space signal $$$\mathbf{K} \in \mathbb{C}^{Z \times C}$$$. The low-rank prior of Hankel form $$$\mathcal{H}\mathbf{K}$$$ of k-space signal K is utilized in the reconstruction model of the undersampled k-space signal Y:$$\underset{\text{K},\mathbf{P},\mathbf{Q}}{\text{min}}\frac{\text{1}}{\text{2}}\text{(}\left\| \text{P} \right\|_{\text{F}}^{\text{2}}\text{+}\left\| \text{Q} \right\|_{\text{F}}^{\text{2}}\text{)+}\frac{\text{λ}}{\text{2}}\left\| {\text{Y}\text{-}\mathcal{U}\text{K}} \right\|_{\text{F}}^{\text{2}}\text{+}{\frac{\text{β}}{\text{2}}\left\| {\mathcal{H}\mathbf{K}\text{-P}\text{Q}^{\text{H}}} \right\|}_{F}^{2}$$
where parameter λ>0 is the data consistency parameter.β>0 is a penalty coefficient and $$$\mathcal{U}$$$ denotes the undersampling operator.
The detailed structure of ADLR for MRI reconstruction (Fig. 1) in the kth block is expressed as follows:
$$\left\{ \begin{matrix} {\text{P}_{DL}^{\text{k}\text{+1}}\text{=}P\left( {\mathcal{H}\mathbf{K}^{\text{k}}\text{Q}^{\text{k}},\text{Q}^{\text{k}},\mathbb{H}_{\mathbf{P}}^{k}} \right)\text{+}\text{P}^{\text{k}}} \\ {\left. {}{}\text{Q}_{DL}^{\text{k}\text{+1}}\text{=}Q\left( \left( \mathcal{H}\mathbf{K}^{\text{k}} \right) \right.^{H}\text{P}_{DL}^{\text{k}\text{+1}}~,\text{P}_{DL}^{\text{k}\text{+1}},\mathbb{H}_{\mathbf{Q}}^{k} \right)\text{+}\text{Q}^{\text{k}}} \\ {\mathbf{K}_{DL}^{\text{k}\text{+}\text{1}}\text{=}S\left( {\mathcal{U}^{\text{*}}\text{Y},\mathcal{H}^{*}\left( \text{P}_{DL}^{\text{k+1}}{\text{(}\text{Q}_{DL}^{\text{k+1}}\text{)}}^{\text{H}} \right),\frac{\text{β}_{DL}^{k}}{\lambda_{DL}^{k}}} \right)} \\ {\text{P}^{\text{k}\text{+1}}{\text{=}\text{β}}^{k}\left( {\mathcal{H}\mathbf{K}_{DL}^{\text{k}\text{+1}}} \right)\text{Q}_{DL}^{\text{k}\text{+1}}\left( {\text{β}{\text{(}\text{Q}_{DL}^{\text{k}\text{+1}}\text{)}}^{\text{H}}\text{Q}_{DL}^{\text{k}\text{+1}}\text{+}\text{I}} \right)^{\text{-1}}} \\ {\text{Q}^{\text{k}\text{+1}}{\text{=}\text{β}}^{k}\left( {\mathcal{H}\mathbf{K}_{DL}^{\text{k}\text{+1}}} \right)^{H}\text{P}^{\text{k}\text{+1}}\left( {\text{β}\left( \text{P}^{\text{k}\text{+1}} \right)^{\text{H}}\text{P}^{\text{k+1}}\text{+}\text{I}} \right)^{\text{-1}}} \\ {\mathbf{K}^{\text{k}\text{+}\text{1}}\text{=}S\left( {\mathcal{U}^{\text{*}}\text{Y},\mathcal{H}^{*}\left( \text{P}^{\text{k}\text{+1}}{\text{(}\text{Q}^{\text{k}\text{+1}}\text{)}}^{\text{H}} \right),\frac{\text{β}^{k}}{\text{λ}^{k}}} \right)} \end{matrix} \right.$$
where the mapping $$$\mathcal{P}$$$ and $$$\mathcal{Q}$$$ are learned by one 2D 6-layer densely connected convolutional network6. $$$\mathbb{H}_{\mathbf{P}}^{k}\text{=}\left\{ \text{P}^{\text{1}}\text{,}\text{P}_{DL}^{\text{i}},\text{P}^{\text{i}} \middle| i\text{=}2,\ldots,k \right\}$$$ and $$$\mathbb{H}_{\mathbf{Q}}^{k}\text{=}\left\{ \text{Q}^{\text{1}}\text{,}\text{Q}_{DL}^{\text{i}},\text{Q}^{\text{i}} \middle| i\text{=}2,\ldots,k \right\}$$$. $$$\mathcal{S}$$$ denotes the data consistency module7.The superscript * and H is the adjoint operator and conjugate transpose, respectively.
Results
To illustrate the performance of ADLR, the MR signal is synthesized by the five-peaks exponential function with Gaussian noise of standard deviation 0.03 (Fig. 2). In the 2nd to 10th blocks , the deep learning solvers reconstruct the signal with a lower RLNE but a higher rank than their input signal. On the contrary, the optimization solvers can effectively reduce the rank of signals, close to the true rank 5 (of noise-free fully sampled signal). As the signal passes by more sequential blocks, the solution with low RLNE is gradually obtained, which reflects the high fidelity of the reconstructed spectra, especially for the peak of low intensity. In the last block, the reconstructed signal achieves a low reconstruction error with low-rank property, almost the same as the noise-free full sampling.
Three multi-coil MRI datasets are used to evaluate the reconstruction robustness of ADLR under different sections and organs, including coronal and sagittal proton density weighted (PDw) knee datasets8 and an axial T2 weighted (T2w) brain dataset9. The total number of slices is 331, 245 and 370, and 80% slices are for training and others for tests. The number of coils in datasets is compressed to 4 coils.
To show the benefit of the optimization solver, ADLR with only deep learning, denoted as ADLR-D, is compared.
Under sampling rate of 25%, when reconstructing the matched coronal knee image and similar sagittal knee image to the training dataset, the proposed ADLR produces the fewest artifacts and lowest RLNE (Fig. 3). When reconstructing the seriously mismatched axial T2w brain images, due to the lack of robustness, HDSLR5, ODLS2 and ADLR-D achieve the high RLNE (>0.12). On the contrary, ADLR achieves the RLNE of about 0.10 (lower than P-LORAKS10) and produces few artifacts, as the result of the introduction of data-independent optimization solvers.
Due to the low-rank prior of MR spectroscopy, the proposed ADLR can be applied in the reconstruction of undersampled MR spectroscopy (Fig. 4). Compared with CS11, LRHMF12, FID-Net4 and DHMF7, ADLR trained by the dataset7 achieves the highest correlation coefficients r2 under the sampling rate 25% and reliable reconstruction (r2>0.99) under the sampling rate 50%.
Conclusion
In this work, we proposed an alternating deep low-rank (ADLR) network, which utilizes the optimization solver and deep learning alternately to solve the low-rank model. Experimental results show that one trained ADLR can faithfully reconstruct MR imaging images of different organs and contrast of imaging, or MR spectroscopy under different sampling rates.Acknowledgements
See more details in the full-length preprint: https://arxiv.org/abs/2211.13479. This work was supported by the National Natural Science Foundation of China (62122064, 61971361, 62331021, 62371410), the Natural Science Foundation of Fujian Province of China (2023J02005 and 2021J011184), the President Fund of Xiamen University (20720220063), and the Nanqiang Outstanding Talents Program of Xiamen University.
The correspondence should be sent to Prof. Xiaobo Qu (Email: quxiaobo@xmu.edu.cn)
References
[1] Qu X, Mayzel M, Cai J-F, et al. Accelerated NMR spectroscopy with low-rank reconstruction. Angew. Chem.-Int. Edit. 2015;54(3):852-854.
[2] Wang Z, Qian C, Guo D, et al. One-dimensional deep low-rank and sparse network for accelerated MRI. IEEE Trans. Med. Imaging 2022;42(1):79-90.
[3] Qu X, Huang Y, Lu H, et al. Accelerated nuclear magnetic resonance spectroscopy with deep learning. Angew. Chem.-Int. Edit. 2020;59(26):10297-10300.
[4] Karunanithy G and Hansen D F. FID-Net: A versatile deep neural network architecture for NMR spectral reconstruction and virtual decoupling. J. Biomol. NMR 2021;75(4):179-191.
[5] Pramanik A, Aggarwal H and Jacob M. Deep generalization of structured low-rank algorithms (Deep-SLR). IEEE Trans. Med. Imaging 2020;39(12):4186-4197.
[6] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017;4700-4708
[7] Huang Y, Zhao J, Wang Z, et al. Exponential signal reconstruction with deep Hankel matrix factorization. IEEE Trans. Neural Netw. Learn. Syst. 2023;34(9):6214 - 6226.
[8] Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 2018;79(6):3055-3071.
[9] Zbontar J, Knoll F, Sriram A, et al. fastMRI: An open dataset and benchmarks for accelerated MRI. 2018;arXiv:1811.08839.
[10] Haldar J P and Zhuo J. P-LORAKS: Low-rank modeling of local k-space neighborhoods with parallel imaging data. Magn. Reson. Med. 2016;75(4):1499-1514.
[11] Kazimierczuk K and Orekhov V Y. Accelerated NMR spectroscopy by using compressed sensing. Angew. Chem.-Int. Edit. 2011;50(24):5556-5559.
[12] Guo D, Lu H and Qu X. A fast low rank Hankel matrix factorization reconstruction method for non-uniformly sampled magnetic resonance spectroscopy. IEEE Access 2017;516033-16039.
Figures
Fig. 1. The overview of ADLR for MRI image reconstruction. (a) The structure of ADLR of K subsequent iteration blocks. The detailed structure in (b) the deep learning solver, and (c) the updating modules P and Q.
Fig. 2. The changing trends of the reconstructed signal over iterations. (a) The RLNE and rank of the reconstructed signal. (b-d) The spectra of the full sampling and the reconstructed signals in deep learning, optimization solver of the 2nd, 5th and 10th blocks.
Fig. 3. Reconstructed images of three MRI datasets. The fully sampled images of (a) coronal PDw knee, (g) sagittal PDw knee, and (m) axial T2w brain and corresponding 1D Cartesian sampling patterns with a sampling rate of 25%. (b-f)(h-l)(n-r) The corresponding reconstructed images and error maps (5
) by P-LORAKS, HDSLR, ODLS, ADLR-D, and ADLR.
Fig. 4. Reconstructions of 2D 1H-15N TROSY spectrum of Ubiquitin under the match and mismatch sampling rates. (a) The fully sampled spectrum. The spectra reconstructed by CS, LRHMF, FID-Net, DHMF and ADLR from (b-f) 25% data and (g-k) 50% data, respectively.