4510
Image-feature-understanding data consistency for under-sampled MRI reconstruction1Research and Development Center, Canon Medical Systems (China) Co., Ltd., Beijing, China
Synopsis
Keywords: AI/ML Image Reconstruction, Machine Learning/Artificial Intelligence
Motivation: The optimal selection of data consistency (DC) weight is task-dependent and a challenge in current deep learning unrolled reconstruction network which may result in compromised image quality, and thus deserves further investigations.
Goal(s): To propose a method to obtain adaptive data consistency weight which is superior to existing methods.
Approach: An image-feature-understanding data consistency (IFUDC) modulator is integrated into network to obtain adaptive DC weight based on input images.
Results: Image quality metrics (SSIM, PSNR) of proposed method are higher than those of existing method.
Impact: IFUDC is effective to modulate DC weight adaptively and helps to mitigate the difficulty in optimal DC weight selection.
Introduction
Model driven deep learning reconstruction methods become popular because less training data is required compared with data-driven deep learning methods1. Model driven DL network consists of several cascades, each cascade includes a regularization neural network and a data consistency (DC) module2,3,4. A proper DC weight in DC module controls the tradeoff between the level of noise/artifact reduction from neural network regularization and data fidelity. The DC weight is usually set as a learnable parameter in end-to-end reconstruction training2,3. Nevertheless, the learned DC weight might be suboptimal due to lack of explicit relationship with the neural network regularization result. In this study, we proposed an image-feature-understanding DC (IFUDC) based network to obtain adaptive data consistency weight by extracting the image features of neural network’s output in each cascade.Methods
In DC module, conjugate gradient (CG) optimization method3 is utilized to find target image $$$x$$$ , i.e.,$$x=argmin\lambda\parallel Ax-b \parallel _2^2 + \parallel x-z\parallel^{2}$$where, $$$A$$$ is linear forward operator which multiplies by the sensitivity maps, applies 2D Fourier transform and then under-samples the data, $$$z$$$ is image space regularization network output, $$$b$$$ is acquired k-space, $$$\lambda$$$ is the DC weight which is modulated by IFUDC in our study. The network design is shown in Figure 1, consisting of:
- SME2: Coil sensitivity maps is estimated jointly with the target image $$$x$$$ through a neural network.
- ISR Block: Neural network regularization block in image space using a U-Net as backbone.
- IFUDC modulator: An image-feature-understanding DC modulator extracts image features through a 4-layer CNN, a fully connected layer, and a sigmoid activation function successively to produce DC modulation weight $$$W$$$ . The final DC weight $$$\lambda$$$ is calculated as $$$\lambda=\lambda_0×W$$$ .
- DC Block: Data consistency block, in which CG optimization method is utilized to find target image using DC weight $$$\lambda$$$ .
Network training:
Training and evaluation datasets are collected on Canon scanners (Vantage Titan 3T, Vantage Galan 3T, Vantage Orian, Canon Medical Systems Corporation, Otawara, Japan). Written informed consent were obtained from all subjects. Datasets are listed in Table 1.
The ground truth images are coil combined images reconstructed with fully sampled k-space and coil sensitivity maps created by ESPIRiT algorithm5. Eight cascades are used in our experiments, and network weights are not shared among cascades. Training loss function is mean square error (MSE). All models are trained using the Adam optimizer with a learning rate of 0.0003 for 50 epochs. We observe that training loss converges well without overfitting.
Unrolled reconstruction network without IFUDC is taken as baseline with a learnable DC weight $$$\lambda$$$ . Initial DC weight value $$$\lambda_0$$$ is set as 100 for both baseline and proposed method.
A group of trained-anatomy datasets and a group of untrained-anatomy datasets are tested to evaluate effectiveness of IFUDC-based network.
Results
Quantitative metrics of both trained- and untrained- anatomy datasets are shown in Table 2, including structure similarity (SSIM), peak signal-to-noise ratio (PSNR). IFUDC-based network obtained better image qualities with higher quantitative metrics than those of baseline. The acceleration factor for all anatomies is 4x except knee is 5x.Figure 2 shows typical results of reconstructed images. Images reconstructed by IFUDC-based network have better sharpness (Figure 2, green arrows) and less artifact (Figure 2, red arrows) than the ones reconstructed without IFUDC.
Discussion
To analyze how IFUDC modulates DC weight in cascades, we investigated the relationship between the image quality of DC input which is also neural network regularization output and DC weight estimated by IFUDC modulator.In Figure 3(A), DC weight has reverse evolution trend against DC input image quality. We infer that the IFUDC modulator would adaptively reduce the DC weight when its input image quality is already high, and vice versa.
The effectiveness of IFUDC-based network is also demonstrated in Figure 3(B) across cascades. The image quality of each cascade is improved and progresses more stably compared with baseline.
Conclusion
We introduced an image-feature-understanding DC (IFUDC) based reconstruction network which achieves better image quality, and demonstrated the effectiveness of IFUDC modulator.Acknowledgements
No acknowledgement found.References
1. Liang D, Cheng J, Ke Z, Ying L. Deep MRI Reconstruction: Unrolled Optimization Algorithms Meet Neural Networks. 2019:1-10. arXiv:1907.11711
2. Sriram A, Zbontar J, Murrell T, et al. End-to-end variational networks for accelerated MRI reconstruction. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2020.
3. Aggarwal HK, Mani MP, Jacob M. MoDL: Model-Based Deep Learning Architecture for Inverse Problems. IEEE Trans Med Imaging. 2019;38(2):394-405.
4. Hammernik K, Schlemper J, Qin C, Duan J, Summers RM, Rueckert D. Systematic evaluation of iterative deep neural networks for fast parallel MRI reconstruction with sensitivity-weighted coil combination. Magn Reson Med. 2021;86(4):1859-1872
5. Uecker M, Lai P, Murphy MJ, et al. ESPIRiT - An eigenvalue approach to autocalibrating parallel MRI: Where SENSE meets GRAPPA. Magn Reson Med. 2014;71(3):990-1001. doi:10.1002/mrm.24751
Figures
