Introduction

A mainstream framework for reconstruction of accelerated MR acquisitions rests on deep neural network (DNN) architectures^1-9. To recover images given undersampled acquisitions, DNNs typically learn data-driven priors from large training datasets in a supervised fashion^1-8. While DNNs have shown remarkable performance, compilation of large-scale datasets for each anatomy and each protocol is challenging. To mitigate this issue, here we propose a few-shot learning approach for accelerated MRI. The proposed approach consists of a composite deep neural network (COMNET) that fuses subject-driven priors obtained via a physical signal model with data-driven priors obtained from only few training samples.

Methods

The reconstruction problem in COMNET can be formulated as:
$$\widehat{x} = \underset{x}{\arg\min} \quad \lambda \underbrace{||F_ux-y||_2}_\text{Data consistency}+ \underbrace{(G-I)Fx}_\text{Subject-driven Prior}+ \underbrace{|| C(A^*x^u;\theta^*) - A^*x||_2}_\text{Data-driven Prior}$$
where $$$F_u$$$ is the partial Fourier operator defined at the sampled k-space locations, $$$x$$$ is the image to be reconstructed, $$$y$$$ are the acquired k-space data, $$$G$$$ is a linear operator enforcing consistency with a fully-sampled auto-calibration region, $$$C$$$ denotes the purely learning-based model for reconstruction, $$$x^u$$$ is the Fourier reconstruction of undersampled data, and $$$A$$$ and $$$A^*$$$ denote coil sensitivity maps and their conjugate obtained via ERPIRiT¹⁰. COMNET comprises three blocks to enforce data consistency (DC), to enforce subject-driven priors via calibration consistency (CC), and to enforce data-driven priors via network consistency (NC) terms in the objective. A common approach is to connect these blocks in series in an unfolded architecture, and solve the optimization problem by alternating minimization of individual terms⁸. However, this serial structure introduces undesirable dependency among consecutive blocks that can lead to information loss. To address this problem, here we proposed to fuse information from parallel connected NC and CC blocks (Figure 1). An unrolled cascade of subnetworks are then leveraged for image recovery, and the output of the $$$pth$$$ subnetwork receiving input from the previous subnetwork is given by:
$$\begin{align}x_p=f_{DC}(A\gamma_p A^*f_{DC}(f_{NC}(A^*x_{p-1}))+A\eta_p A^*f_{DC}(f_{CC}(x_{p-1})))\end{align}$$
where $$$f_{CC}$$$, $$$f_{NC}$$$, and $$$f_{DC}$$$ denote mappings by CC, NC and DC blocks, $$$x_p$$$ is the output of the $$$pth$$$ sub-network, $$$x_{p-1}$$$ is the output of the $$$(p-1)th$$$ sub-network, and $$$\gamma_p$$$ and $$$\eta_p$$$ are fusion parameters to combine information from the NC and CC blocks. NC block was adopted from² where each network consisted of 1 input layer, 4 convolutional layers each containing 64 channels, and 1 output layer. Real and imaginary parts were recovered using separate network branches. The CC block was implemented via SPIRiT¹¹ where 5 CC projections were performed within each block. The network was trained in an end-to-end manner, where parameters of each sub-network was identical except for the weighing parameters ($$$\gamma$$$ and $$$\eta$$$) that were different for each sub-network. ADAM optimizer¹² was used with a learning rate of $$$10^{-4}$$$, and parameters $$$\beta_1$$$=0.90 and $$$\beta_2$$$=0.99. Network was trained to minimize $$$\ell_1$$$ and $$$\ell_2$$$ norm difference between reconstructed and ground-truth images. Number of epochs was set to 200.
Demonstrations were performed on contrast enhanced T₁-weighted (cT₁), T₂-weighted and FLAIR images from the fastMRI dataset¹³. 30 subjects were reserved for training, 10 for validation and 40 for testing. For a systematic evaluation, cT₁ and FLAIR images were cropped to a final size of 256x320x10 and T₂ images were cropped to 288x384x10 when necessary. Geometric coil compression¹⁴ was utilized to ensure that all MRI data had 5 coils. Acquisitions were retrospectively undersampled at R=4x via random sampling masks generated using normal sampling density. COMNET was compared against a regular DNN consisting of only data-driven priors, and L1-SPIRiT consisting of subject-driven priors coupled with sparsity prior in the Wavelet domain. The number of samples for COMNET and DNN were varied from 2 to 300. All hyperparameters were selected via cross-validation with three-way split of data.

Results

Average PSNR and SSIM values of recovered cT₁- weighted, T₂-weighted and FLAIR images at R=4x are listed in Table. 1. Both DNN and COMNET were trained on 6 cross-sections from a single subject. On average, COMNET achieves 0.48dB higher PSNR and 0.72% higher SSIM compared to the second-best method.
Figure 2 shows PSNR values across recovered cT₁- weighted, T₂-weighted and FLAIR images as a function of number of training samples. DNN, on average, requires around 90 cross-sections from 9 subjects to outperform L1-SPIRiT. COMNET, on the other hand, requires 2,4, and 6 cross-sections from just a single subject to outperform L1-SPIRiT on cT₁- weighted, T₂-weighted and FLAIR images. Importantly, COMNET reduces the number of required samples by at least an order of magnitude compared to DNN.
Figures 3 and 4 show representative T₂-weighted and FLAIR images from L1-SPIRiT, DNN and COMNET. DNN and COMNET were trained on 6 cross-sections from a single subject. COMNET outperforms both L1-SPIRiT and DNN in terms of residual aliasing artifacts.

Discussion

Here, we propose a few-shot learning approach for MR image reconstructions using deep neural networks. The proposed approach synergistically combines subject-driven priors with data-driven priors to address the issue of data scarcity in DNNs for MR image reconstruction.

Conclusion

The proposed approach enables data-efficient training of deep neural networks for MR image reconstruction. Therefore, COMNET holds great promise for improving practical use of deep learning models in accelerated MRI.

Acknowledgements

This work was supported in part by a TUBA GEBIP fellowship, by a TUBITAK 1001 Grant (118E256), and by a BAGEP fellowship awarded to T. Çukur. We also gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research.