2784

Joint Optimization Sampling and Reconstruction of Multi-Contrast MRI under Specific Scan Sequence Combination
Jianing Geng1, Zijian Zhou1, Haikun Qi1, and Peng Hu1
1ShanghaiTech University, Shanghai, China

Synopsis

Keywords: Image Reconstruction, Image Reconstruction, Multi-Contrast, Joint Optimization, Optimized Sampling

Motivation: The current multi-contrast MRI sampling and reconstruction methods cannot efficiently collect complementary information to achieve better reconstruction performance.

Goal(s): A method was designed to generate corresponding sampling masks for each contrast image in a multi-sequence clinical scanning scenario, collect the optimal complementary information for better application in multi-contrast joint reconstruction.

Approach: We jointly optimized the sampling and reconstruction of multi-contrast images, and designed learnable acceleration ratio and decoder feature fusion for images with different contrasts.

Results: The PSNR and SSIM metrics of reconstructed images have significantly improved, and different sampling masks can be generated for different contrasts and sampling order.

Impact: The method of jointly optimizing the sampling and reconstruction of multi-contrast images in a single scan may provide a powerful tool for accelerating and optimizing the MRI scanning process and improving the reconstructed quality of the multi-contrast images.

INTRODUCTION

Multi-contrast magnetic resonance image (MRI) reconstruction has received widespread attention due to its ability to utilize complementary structural information between images of different contrasts to accelerate reconstruction1,2. However, traditional methods only consider multi-contrast feature fusion in reconstruction networks with fixed sampling masks, rarely considering the joint optimization of sampling masks and reconstruction networks, which has been proven to achieve better image reconstruction quality3,4. Meanwhile, most previous methods were aimed at assisting reconstruction using fully sampled images of different contrasts in single sequence scanning scenes. There is no research focused on multi-contrast images both sampling and reconstruction for fixed sequence combinations in a single scan, which may have specific diagnostic purposes.
In this article, we propose a joint method for both optimizing the k-space sampling trajectory and reconstructing the image from under-sampled k-space data. Our method was designed for scanning scenarios with fixed scan combinations. This method achieves its goal by simultaneously optimizing images with different contrasts and jointly optimizing sampling and reconstruction. The proposed method can generate the optimal sampling mask and optimize the sampling of different sequences under fixed sequence combinations to collect more complementary information to assist in reconstruction.

METHOD

Network design: An end-to-end deep learning network is proposed for jointly optimizing sampling masks and reconstructing networks under fixed sampling combinations. The method consists of two parts: sampler and reconstructor. The sampler inherits the design of the LOUPE model and consists of probability layer, rescaling layer, and binarization layer. Specifically, we have designed a learnable acceleration rate that assigns different weights to images with different contrasts. After conducting comparative experiments with different feature fusion strategies, we designed a multi contrast feature fusion approach to understand the code layer, and embedded the timing information of the scanning sequence into the network through a serial structure. The overall architecture is shown in Figure 1.
Dataset and processing: We evaluate the proposed method on MICCAI Brain Tumor Segmentation (BraTS) challenge 2021 dataset, which contains four contrast images: T1w, T2w, T1w contrast enhancement (T1CE) and fluid-attenuated inversion recovery (FLAIR). And different contrast images of the same subject have already been registered. There are a total of 1204 patients in the dataset, and we divided the training set, validation set, and test set in a 7:2:1 ratio. In the data pre-processing stage, we select the 50 central axial slices in the 3D volume, crop every slice into 192x192 and perform the image intensity normalization. To verify the robustness of our method on image pairs with different contrasts, we paired the four contrasts in the dataset and conducted experiments separately.
Experiments: We compared the handcrafted sampling mask Poisson disk and random uniform with UNet based multi-contrast reconstruction. We also compared the multi-contrast version of LOUPE3 method. All the experiments are based on 4-fold total acceleration rate. For metrics, we use peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) to quantitively evaluate the reconstructed images quality.

RESULT

Our method can achieve excellent image reconstruction quality and generate specific and optimal sampling masks and corresponding reconstruction networks based on scanning sequence combinations. The reconstructed images and their metrics are shown in Figure 2 and Table 1, respectively. The proposed method outperforms the baseline multi-contrast sampling and reconstruction method, which shows a statistically significant difference with p-value smaller than 10-100. In addition, as shown in Figure 3, the proposed method can generate specific sampling masks and allocate specific acceleration rate for different contrast sequence combinations.

DISCUSSION

In our method, by implicitly fusing sequence order information in the decoder of the reconstructor, it can generate specific sampling masks for different sequence combinations and images with different contrasts. And the experimental results show that the sampling mask is similar to a Tai Chi shaped ring. The reason may be the conjugate symmetry of simulated k-space data, and the sampling masks tend to sample complementary information with central symmetry.

CONCLUSION

In this article, we propose a method for finding the optimal sampling trajectory and reconstruction network under a fixed contrast scanning combination. It is achieved by simultaneously optimizing images with different contrasts and jointly optimizing sampling and reconstruction. Our method has achieved superior performance and is significantly superior to state-of-the-art multi contrast reconstruction methods in terms of reconstructed image quality.

Acknowledgements

No acknowledgement found.

References

1. Lei Xiang, Yong Chen, Weitang Chang, Yiqiang Zhan, Weili Lin, Qian Wang, and Dinggang Shen, “Deep-learning-based multi-modal fusion for fast mr reconstruction,” IEEE Transactions on Biomedical Engineering, vol. 66, no. 7, pp. 2105–2114, 2018.

2. Salman UH Dar, Mahmut Yurt, Mohammad Shahdloo, Muhammed Emrullah Ildız, Berk Tınaz, and Tolga Cukur, “Prior-guided image reconstruction for accelerated multi-contrast mri via generative adversarial networks,” IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 6, pp. 1072–1087, 2020.

3. Bahadir C D, Wang A Q, Dalca A V, et al. Deep-learning-based optimization of the under-sampling pattern in MRI. IEEE Trans Comput Imaging, 2020;6:1139-1152.

4. Yang J , Li X X , Liu F ,et al.Fast T2w/FLAIR MRI Acquisition by Optimal Sampling of Information Complementary to Pre-acquired T1w MRI.[J]. 2021.DOI:10.48550/arXiv.2111.06400.

5. U.Baid, et al., "The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification", arXiv:2107.02314, 2021

Figures

Figure 1. The overall architecture of the proposed method. The model consists of two parts: sampler and reconstructor. The sampler is used to generate sampling masks for multi-contrast images, consisting of probability layer, rescaled layer, and binarizied layer. The rescaled layer achieves learnable acceleration ratio through learnable parameters and softmax function. The reconstructor is a UNet based denoising network. In each layer of current decoder, the features from former sequence decoder are received and transmitted the current features to the later sequence decoder.

Figure 2. Image results of different sampling and reconstruction method on the BraTS2021 dataset with 4-fold total acceleration rate. In the figure, the ground truth image, reconstructed images and absolute error maps are shown. The bottom of the image shows the sampling mask corresponding to each sequence. The method we propose and the sampling mask generated by LOUPE are both contrast specific, while Poisson and Uniform use the same sampling mask. The upper left corner of the mask shows its sparsity, which is the reciprocal of the acceleration rate.

Figure 3. Comparison of sampling masks based on different scan sequence combinations and contrast. The upper left corner of the mask shows its sparsity, which is the reciprocal of the acceleration rate. The lower right corner shows the sequence order in the simulated scanning combination.

Table 1. Metrics results of different sampling and reconstruction method on the BraTS dataset(T1w,T2w,FLAIRw) with 4-fold total acceleration rate. We calculated the mean value and standard deviation value for PSNR and SSIM metrics of each contrast in the test set.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
2784
DOI: https://doi.org/10.58530/2024/2784