0809

CAMP-Net: The Application of Consistency-Aware Multi-Prior in Deep Learning for Rapid MRI

Liping Zhang¹ and Weitian Chen¹
¹Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China

Synopsis

Keywords: AI/ML Image Reconstruction, Brain

Motivation: Accelerated MRI acquisitions offer reduced imaging scan times but pose challenges in image reconstructions. Tremendous progress has been made to reconstruct accelerated MRI, but it remains challenging to restore high-frequency image details in highly undersampled data.

Goal(s): Our goal is to develop a solution that can restore subtle structures even for highly accelerated MRI.

Approach: We propose CAMP-Net, a consistency-aware multi-prior framework, that leverages scan-specific features with both image and $$$k$$$-space domain knowledge for MRI reconstruction.

Results: Results on a publicly available brain dataset demonstrated that CAMP-Net can achieve high-quality reconstructions with fine brain anatomical structures even at an acceleration factor of 10X.

Impact: The successful restoration of subtle structures for MRI with high acceleration factors can significantly reduce MRI scan time in clinical routines, benefiting patients, increasing the access to MRI, and significantly reducing healthcare cost of MRI.

Introduction

The inherently slow acquisition of MRI can result in motion-related artifacts and patient discomfort. Deep learning-based MRI approaches ^1-8 have demonstrated tremendous progress in reconstruction of accelerated MRI. However, these methods still face limitations in preserving fine image details and textures at high acceleration rates. In this study, we introduce CAMP-Net, a novel unrolling-based Consistency-Aware Multi-Prior Network, for reconstruction of accelerated MRI. Our goal is to restore subtle structural details by jointly learning image, $$$k$$$-space, and calibration priors within each iteration.

Methods

Reconstructing from under-sampled $$$k$$$-space data is an ill-posed inverse problem. Here, we formulate it as a learnable multi-prior optimization problem:
$$\hat{x} = \arg\min_{x} \| \mathcal{A}x - \tilde{y} \| ^2_2 + \sum_{l=1}^{L}\lambda_l \mathcal{P}(\mathcal{D}_{l}x;\theta_l),$$
where $$$\mathcal{P}(\mathcal{D}_{l}x;\theta_l)$$$ is a data-adaptive prior with learnable parameters $$$\theta_l$$$ that incorporate learned knowledge in the specific transform domain $$$\mathcal{D}_{l}$$$, and $$$\lambda_l$$$ is a parameter to balance the influence of the imposed prior and the fidelity of the acquired data.

We propose CAMP-Net to solve the multi-prior optimization problem by leveraging mutually constrained priors from the image, $$$k$$$-space, and calibration data. To capture both intra- and inter-slice correlations, we utilize adjacent slice information, resulting in enhanced priors and improved reconstruction. The architecture of CAMP-Net, depicted in Fig. 1, comprises several modules: Spatial Encoding (SEM), Image Enhancement (IEM), $$$k$$$-space Restoration (KRM), Calibration Consistency (CCM), and Frequency Fusion (FFM). The SEM learns coil sensitivity maps from auto-calibration signal (ACS) data, which are then utilized throughout the iterative learning process. The IEM focuses on extracting image prior knowledge from coil-combined images, specifically targeting artifact removal and low-frequency restoration in the image domain. The KRM module explores correlations within the multi-coil $$$k$$$-space data to restore high-frequency details. It effectively utilizes information from all complex-valued coil images to interpolate missing data from neighboring $$$k$$$-space samples, as shown in Fig. 2(a). To ensure consistent missing data interpolation, the CCM incorporates calibration information from ACS, enabling the learning of self-consistent feature representations that facilitate reliable neighborhood relationships in the KRM module, as shown in Fig. 2(b). At the end of each iteration, the IEM and KRM interact through the FFM, synergistically combining their respective advantages to achieve improved reconstruction.

Data: We demonstrated the proposed CAMP-Net using the multi-coil brain MR raw data from the publicly available Calgary-Campinas dataset ⁹. The data were acquired using T1-weighted 3D gradient-recalled echo acquisitions with 1 mm isotropic resolution and 12-channel coils. The investigated dataset included 47 volumes for training (7332 axial slices), 10 volumes for validation (1560 axial slices), and 10 volumes for testing (1560 axial slices) ⁸. Acceleration factors of 5 and 10 were evaluated using 2D Poisson disk distribution sub-sampling masks, where the center of the k-space was fully sampled within a radius of 16 ¹⁰.

Implementation: The number of iterations (T) was set to 8. The IEM utilized a 4-stage U-Net with initial feature dimensions of 64. The SEM employed the same U-Net as in the works ^5,11. The feature dimensions in the KRM and CCM were matched to the number of coils.

Results and Discussion

CAMP-Net was compared to several state-of-the-art (SOTA) methods, including fastMRI U-Net ¹¹, MultiDomainNet ¹, KIKI-Net ², XPDNet ³, Joint-ICNet ⁴, E2EVarNet ⁵, RIM ⁶, LPDNet ⁷, IterDualNet, ConjGradNet, and RecurrentVarNet ⁸. The evaluation results are presented in Fig. 3. CAMP-Net demonstrated superior performance over others in PSNR and SSIM at both acceleration factors. The violin plots in Fig. 4 consistently showed that CAMP-Net had the highest median, minimal discrepancies, and no identifiable outliers, further confirming its superior performance. Fig. 5 displayed representative reconstructed images and their associated error maps. Note CAMP-Net achieved notable improvements in image reconstruction and restored high-frequency details and textures. These improvements were supported by the reduced error compared to the other methods. In contrast, other methods exhibited limitations in restoring anatomical details. For example, RecurrentVarNet which shows the best performance among the methods being compared, had difficulties to restore the occipital sulcus at a 5X acceleration rate and accurately reconstruct the occipital gyrus at a 10X acceleration, as indicated by the yellow arrows in Fig. 5.

Conclusions

We introduced CAMP-Net, a novel deep learning-based method for accelerated MRI. It incorporates consistency-aware multiple prior information to capture complementary feature representations. This allows for the faithful restoration of subtle anatomical structures in highly accelerated MRI. This method has potential to help routine clinical MRI after further validations.

Acknowledgements

This work was supported by a grant from Innovation and Technology Commission of the Hong Kong SAR (MRP/046/20X); and by a Faculty Innovation Award from the Faculty of Medicine of the Chinese University of Hong Kong.

References

1. M. J. Muckley, B. Riemenschneider, A. Radmanesh, S. Kim, G. Jeong, J. Ko, Y. Jun, H. Shin, D. Hwang, M. Mostapha et al., “Results of the 2020 fastmrichallenge for machine learning mr image reconstruction,” IEEE transactions on medical imaging, vol. 40, no. 9, pp. 2306–2317, 2021.

2. T. Eo, Y. Jun, T. Kim, J. Jang, H.-J. Lee, and D. Hwang, “Kiki-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonanceimages,” Magnetic resonance in medicine, vol. 80, no. 5, pp. 2188–2201, 2018.

3. Z. Ramzi, P. Ciuciu, and J.-L. Starck, “Xpdnet for mri reconstruction: An application to the 2020 fastmri challenge,” arXiv preprint arXiv:2010.07290, 2020.

4. Y. Jun, H. Shin, T. Eo, and D. Hwang, “Joint deep model-based mr image and coil sensitivity reconstruction network (joint-icnet) for fast mri,” in Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 5270–5279.

5. A. Sriram, J. Zbontar, T. Murrell, A. Defazio, C. L. Zitnick, N. Yakubova, F. Knoll, and P. Johnson, “End-to-end variational networks for accelerated mri reconstruc-tion,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings,Part II 23. Springer, 2020, pp. 64–73.

6. K. Lønning, P. Putzky, J.-J. Sonke, L. Reneman, M. W. Caan, and M. Welling, “Recurrent inference machines for reconstructing heterogeneous mri data,” Medicalimage analysis, vol. 53, pp. 64–78, 2019.

7. J. Adler and O. Öktem, “Learned primal-dual reconstruction,” IEEE transactions on medical imaging, vol. 37, no. 6, pp. 1322–1332, 2018.

8. G. Yiasemis, J.-J. Sonke, C. Sánchez, and J. Teuwen, “Recurrent variational network: a deep learning inverse problem solver applied to the task of accelerated mrireconstruction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 732–741.

9. R. Souza, O. Lucena, J. Garrafa, D. Gobbi, M. Saluzzi, S. Appenzeller, L. Rittner, R. Frayne, and R. Lotufo, “An open, multi-vendor, multi-field-strength brainmr dataset and analysis of publicly available skull stripping methods agreement,” NeuroImage, vol. 170, pp. 482–494, 2018.

10. Y. Beauferris, J. Teuwen, D. Karkalousos, N. Moriakov, M. Caan, G. Yiasemis, L. Rodrigues, A. Lopes, H. Pedrini, L. Rittner et al., “Multi-coil mri reconstructionchallenge—assessing brain mri reconstruction models and their generalizability to varying coil configurations,” Frontiers in Neuroscience, vol. 16, p. 919186, 2022.

11. J. Zbontar, F. Knoll, A. Sriram, T. Murrell, Z. Huang, M. J. Muckley, A. Defazio, R. Stern, P. Johnson, M. Bruno et al., “fastmri: An open dataset and benchmarksfor accelerated mri,” arXiv preprint arXiv:1811.08839, 2018

Figures

Fig. 1: The overall architecture of CAMP-Net. It involves taking multiple consecutive slices of zero-filled under-sampled $$$k$$$-space data and their corresponding sampling masks as inputs. The first step is to generate coil sensitivity maps using the SEM from the ACS data. Then, the network utilizes the IEM, KRM, CCM, and FFM to progressively reconstruct multiple slices of images. During the training process, the reconstruction loss is calculated only on the center target slice to provide supervision. (Best viewed in color.)

Fig. 2: The detailed flowchart shows the KRM, CCM, and their interaction. (a) The KRM takes multi-slice multi-coil $$$k$$$-space data as input and utilizes a complex network to refine the $$$k$$$-space data from its neighboring points, along with a surface data layer to remove data imperfections. (b) The CCM takes multi-slice multi-coil ACS data as input and utilizes a complex network and a surface data layer to model the $$$k$$$-space correlations from the neighboring points of the valid ACS region, guiding the KRM to learn consistency-aware correlations. (Best viewed in color.)

Fig. 3: Quantitative evaluation results [mean (standard deviation)] on the Calgary-Campinas dataset at acceleration factors of 5X and 10X. (Best viewed in color.)

Fig. 4: Performance of reconstruction models (colored legend) for the Calgary-Campinas dataset at 5X (left) and 10X (right) acceleration. (Best viewed in color.)

Fig. 5: Brain image reconstruction and error maps for Calgary-Campinas at 5X and 10X accelerations. Each image includes a zoomed-in view with highlighted regions of interest (yellow rectangles) for detailed analysis. The yellow arrow emphasizes restored subtle brain details. (Best viewed in color.)

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)

0809

DOI: https://doi.org/10.58530/2024/0809