INTRODUCTION

Sampling pattern optimization, which attracts a renewed interest due to recent development in the field of deep learning, is an important issue in compressed sensing magnetic resonance imaging (CS-MRI) [1]. Since multi-contrast imaging benefits more the diagnosis of pathological changes, joint optimization of sampling patterns is worth studying to speed up imaging. Nevertheless, most studies of fast multi-contrast MRI focus on joint rapid reconstruction under fixed and independent sampling patterns with an identical sample rate [2]. This sampling scheme does not take the scan time difference of different sequence into consideration. The prior knowledge, that same anatomical information among contrasts could compensate for the quality loss incurred by the unbalanced sample rates among different sampling patterns, would be an important condition to search for the compromise between total scan time and reconstruction quality. The joint optimization of multiple sampling patterns for multi-contrast fast MR imaging is demonstrated to be more advantageous than the use of multiple independent sampling patterns in shortening sampling time and taking advantage of underlying information among contrasts [3,4]. In this study, a neural network which can jointly optimize sampling patterns and simultaneously reconstruct multi-contrast MR images is proposed.

METHODS

A supervised learning framework, that is similar to the Auto-Encoder where the optimized union sampling pattern model and the reconstruction model are respectively encoder and decoder, was proposed. As shown in Figure 1, our method, called JOSPaR (Joint Optimization of Sampling Patterns and Reconstruction) network, is composed of JOSP (Joint Optimization of Sampling Patterns) sub-network $$$g(\cdot)$$$ and Recon (Reconstruction) sub-network $$$f(\cdot)$$$. Given MR fully-sampled image pairs $$$\overrightarrow{x}^{j}$$$ with $$$J$$$ contrasts as the dataset, the JOSP network $$$g(\cdot)$$$ generates $$$J$$$ sampling patterns $$$\overrightarrow{m}^{j}$$$ by stacking $$$J$$$ same blocks which include one Hadamard product layer, two modified sigmoid layers, and one sampling layer. Data consistency (DC) layer, skip connection within the block and dense connection outside the block are used to build Recon network for under-sampled multi-contrast MR reconstruction. The sharable anatomical information of multi-contrast images is learned because all convolution kernels operate in multi-contrast channels or feature maps. Dense connection maintains reconstructed images from previous blocks. Ultimately, the goal of iteratively improving the quality of reconstruction is reached. The loss function is formulated as follows:
$$\mathop{\arg\min}_{\theta,\{\overrightarrow{p}^{j}\}^{J}}\sum_{j=1}^{J}||f(\{F^{H}U(\overrightarrow{m}^{j})F\overrightarrow{x}^{j}\}^{J};\theta)-\overrightarrow{x}^{j}||_{2}^{2}+\lambda||\delta-\frac{1}{I}\sum_{j=1}^{J}\sum_{i=1}^{I}\tau^{j}\overrightarrow{p}^{j}_{i}||_{1}, m^{j}=g(\overrightarrow{1};\overrightarrow{p}^{j})$$
where $$$\delta$$$ and $$$\tau^{j}$$$ ($$$\sum^{J}_{j=1}\tau^{j}=1$$$) are rough target average sample rate and the penalty parameter for the probability density function (PDF) $$$\overrightarrow{p}^{j}$$$ of the $$$j$$$-th contrast. The weight parameter $$$\theta$$$ of Recon network and the PDF of the JOSP network are learnable. $$$I$$$ is the size of an MR image. $$$\lambda$$$ is a balance parameter between reconstructed result and target average sample rate. $$$U(\overrightarrow{m}^{j})$$$ is under-sampled operator based learned pattern $$$\overrightarrow{m}^{j}$$$ of the $$$j$$$-th contrast MR image, $$$F$$$ and $$$F^{H}$$$ are Fourier transform pair. The optimization process of union sampling pattern is guided by the Recon network. After joint training of the two sub-networks, the JOSP network as a sampling generator from the PDF produces one sampling pattern combination $$$\{\overrightarrow{m}^{j}\}^{J}$$$. The learned patterns are then embedded into a corresponding contrast sampling trajectory to complete under-sampled measures what could be recovered using a trained reconstruction network. Experiments were conducted on the MRBrains18 dataset [5]. These multi-contrast images with size 240$$$\times$$$240 of 30 subjects were acquired on a 3T scanner at the UMC Utrecht (the Netherlands), including T₁ weighted imaging with TR = 7.9 ms, TE = 4.5 ms, and T₂ FLAIR imaging with TR = 4416 ms, TE = 15 ms. Usually, the scan time of T₂ weighted imaging is longer than T₁ weighted imaging. We randomly selected 48 pairs and 128 pairs of images from two contrasts as test dataset and train dataset. The multi-contrast MR images reconstructed using fixed and independent variable density patterns with same (balance) or different (unbalance) sample rates between two contrasts were compared with the results of our proposed method. We set $$$\tau^{1}:\tau^{2}=\tau^{\text{T}_{1}\text{-weighted}}:\tau^{\text{T}_{2}\text{-FLAIR}}=1:2$$$ to obtain more sampling points for T₁ weighted imaging and fewer sampling points for T₂ FLAIR imaging to reduce the total scan time.

RESULTS AND DISCUSSION

Figure 2 shows the quantitative comparison of three evaluation metrics (peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and normalized root mean square error (NRMSE)) calculated from the results of three sampling patterns combinations on the MRBrains18 dataset under different total sample rates. The performance of overall sampling patterns optimization is not well at low total sample rate but the improvement brought by learned sample patterns is prominent at high total sample rate. The ratio of the penalty parameter between T₁ weighted and T₂ FLAIR sampling patterns helps to balance the scan time between them. Figure 3 shows the reconstructed results of one example. We can see that the reconstructed images using the learned patterns combination outperforms fixed and independent patterns combination.

CONCLUSION

The proposed JOSPaR network can reconstruct multi-contrast MR images from under-sampled MR images which are acquired using learned sampling patterns combination. It can be applied for increasing reconstruction performance and shorten scan time in a multi-sequences scenario.

Acknowledgements

This work was supported by the National Natural Science Foundation of China under grant numbers 11775184 and 81671674.