INTRODUCTION

Magnetic resonance imaging (MRI), as a non-invasive, radiation-free medical imaging technology, is widely used in clinical disease diagnosis, treatment, and screening. One of the bottlenecks of MRI is the need for long scanning times due to physical constraints. To reduce the acquisition time of MRI data, various undersampling reconstruction methods have gained widespread attention¹. Recently, deep learning has been widely applied in magnetic resonance (MR) image reconstruction, achieving superior performance by learning valuable information from a large amount of data^2-8. Among these methods, some directly learn the mapping relationship from undersampled data to fully sampled data^2-6, while others construct network architectures by unfolding the iterative optimization paradigm derived from the traditional MRI acquisition model^7,8. These methods require the collection of a large amount of data, which can lead to privacy concerns⁹. Federated learning, as a distributed training paradigm, allows collaborative training using data from multiple institutions while protecting patient privacy^10-13. However, the fine structures in the images obtained by existing federated MR image reconstruction methods still need improvement, thus affecting the quality of clinical diagnosis. To enhance the image quality obtained by federated MR image reconstruction methods, we propose a Laplacian attention mechanism to capture fine structure and details for accurate MR image reconstruction from undersampled data.

METHODS

The overall framework of our proposed method is shown in Fig. 1. To capture the fine structure and details of the reconstructed image, we propose a Laplacian attention mechanism to extract multi-scale information and focus on regions of interest. Fig. 1(b) represents the Laplacian attention mechanism, which is a component of the reconstruction model in Fig. 1(c). Specifically, the global descriptors $$$g_{avg}$$$ and $$$g_{max}$$$ are obtained by adaptive average pooling and adaptive maximum pooling. Key features at different scales are then learned by the Laplacian pyramid with dilated convolutions:
$$g_{avg}^{p}(F)=cat\left \{ \sigma (D_{f_{3}}(g_{avg}(F))),\sigma (D_{f_{5}}(g_{avg}(F))),\sigma (D_{f_{7}}(g_{avg}(F))) \right \} \qquad (1)$$
$$g_{max}^{p}(F)=cat\left \{ \sigma (D_{f_{3}}(g_{max}(F))),\sigma (D_{f_{5}}(g_{max}(F))),\sigma (D_{f_{7}}(g_{max}(F))) \right \} \qquad (2)$$
where $$$\sigma$$$ denotes the ReLU function, $$$D_{f_{3}}$$$, $$$D_{f_{5}}$$$, and $$$D_{f_{7}}$$$ denote the dilated convolutions with scale 3, 5, and 7, respectively. To adjust the channel number of the feature map, one convolution is utilized for $$$g_{avg}^{p}$$$ and $$$g_{max}^{p}$$$ respectively, and then the convolved results are summed, and fed into the sigmoid function to obtain the statistical information of different channels.
$$L(F)=\delta (f_{3\times 3}(g_{avg}^{p}(F))+f_{3\times 3}(g_{max}^{p}(F))) \qquad (3)$$
During the federated learning process, each client's model architecture is the same, and model parameters are updated in each communication round. Specifically, in each communication round, the central server sends the aggregated model to each client, and then each client $$$k$$$ trains their local model by minimizing the following loss:
$$\ell ^{k}=\mathbb{E}_{(b^{k},x^{k})\sim D^{k}}\left [ \left \| f_{k}(A^{H}b^{k};\Theta _{C^{k}}) -x^{k}\right \|_{2} \right ] \qquad (4) $$
where $$$\ell^{k}$$$ denotes the training loss of the $$$k^{th}$$$ client and $$$f_{k}(\cdot)$$$ represents the $$$k^{th}$$$ local model parameterized by $$$\Theta _{C^{k}}$$$. Each client is optimized by stochastic gradient descent in local training:
$$\Theta _{C^{k}}^{t,z+1}=\Theta _{C^{k}}^{t,z}-\eta _{k}\bigtriangledown \ell ^{k} \qquad (5) $$
where $$$t$$$ denotes the global epoch, $$$z$$$ denotes the local epoch, and $$$\eta_{k}$$$ represents the learning rate of the $$$k^{th}$$$ client. All clients send the trained weights to the server, and then the global model weight is obtained by aggregating local weights through the federated averaging (FedAvg)¹⁰ method:
$$\Theta _{C}^{t}=\sum_{k=1}^{K}\alpha _{C^{k}}^{t}\Theta _{C^{k}}^{t} \qquad (6) $$
where $$$\Theta _{C}^{t}$$$ denotes the server weights in the $$$t^{th}$$$ global communication round, $$$\alpha _{C^{k}}^{t}$$$ represents the aggregation weight for the $$$k^{th}$$$ client.

RESULTS

An in-house dataset and two public datasets (fastMRI, cc359) are used to evaluate the model's performance. Quantitative evaluation metrics including PSNR and SSIM. Reconstruction results of different methods under a one-dimensional random 4x mask are reported. Fig. 2 represents the qualitative results of different methods on three datasets. Figs. 3 and 4 respectively show the PSNR and SSIM metric values of different methods on the three datasets. From the experimental results, our method achieves the smallest error maps and has the best quantitative metrics on all three datasets, validating the effectiveness of our approach.

CONCLUSION

In this paper, we propose a Laplacian attention mechanism to capture the fine structure and details, effectively enhancing image quality and achieving accurate MR image reconstruction.

Acknowledgements

This research was partly supported by the National Natural Science Foundation of China (62222118, U22A2040), Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application (2022B1212010011), Shenzhen Science and Technology Program (RCYX20210706092104034, JCYJ20220531100213029), and Key Laboratory for Magnetic Resonance and Multimodality Imaging of Guangdong Province (2023B1212060052).

References

[1] Liang D, Cheng J, Ke Z, et al. Deep magnetic resonance image reconstruction: Inverse problems meet neural networks[J]. IEEE Signal Processing Magazine, 2020, 37(1): 141-151.
[2] Zhu B, Liu J Z, Cauley S F, et al. Image reconstruction by domain-transform manifold learning[J]. Nature, 2018, 555(7697): 487-492.
[3] Mardani M, Gong E, Cheng J Y, et al. Deep generative adversarial neural networks for compressive sensing MRI[J]. IEEE transactions on medical imaging, 2018, 38(1): 167-179.
[4] Lee D, Yoo J, Tak S, et al. Deep residual learning for accelerated MRI using magnitude and phase networks[J]. IEEE Transactions on Biomedical Engineering, 2018, 65(9): 1985-1995.
[5] Mardani M, Gong E, Cheng J Y, et al. Deep generative adversarial neural networks for compressive sensing MRI[J]. IEEE transactions on medical imaging, 2018, 38(1): 167-179.
[6] Qin C, Schlemper J, Caballero J, et al. Convolutional recurrent neural networks for dynamic MR image reconstruction[J]. IEEE transactions on medical imaging, 2018, 38(1): 280-290.
[7] Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data[J]. Magnetic resonance in medicine, 2018, 79(6): 3055-3071.
[8] Aggarwal H K, Mani M P, Jacob M. MoDL: Model-based deep learning architecture for inverse problems[J]. IEEE transactions on medical imaging, 2018, 38(2): 394-405.
[9] Kaissis G A, Makowski M R, Rückert D, et al. Secure, privacy-preserving and federated machine learning in medical imaging[J]. Nature Machine Intelligence, 2020, 2(6): 305-311.
[10] McMahan B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Artificial intelligence and statistics. PMLR, 2017: 1273-1282.
[11] Li T, Sahu A K, Zaheer M, et al. Federated optimization in heterogeneous networks[J]. Proceedings of Machine learning and systems, 2020, 2: 429-450.
[12] Liang P P, Liu T, Ziyin L, et al. Think locally, act globally: Federated learning with local and global representations[J]. arXiv preprint arXiv:2001.01523, 2020.
[13] Feng C M, Yan Y, Wang S, et al. Specificity-preserving federated learning for MR image reconstruction[J]. IEEE Transactions on Medical Imaging, 2022.