1967
Anatomy-Matching based Multi-contrast MR Super-resolution1Yonsei University, Seoul, Korea, Republic of, 2Dongguk University, Seoul, Korea, Republic of
Synopsis
Keywords: AI/ML Image Reconstruction, Machine Learning/Artificial Intelligence
Motivation: Multi-contrast MRI with a single fully-sampled HR image and multiple under-sampled, LR contrasts can streamline diagnostics while reducing scan times.
Goal(s): To effectively reconstruct under-sampled images by extracting anatomical information from the fully-sampled HR reference.
Approach: Employ contrast/anatomy disentanglement learning to preserve anatomical consistency and restore unique contrast features in the LR images.
Results: Preliminary outcomes indicate superior reconstruction fidelity compared to traditional methods, enhancing diagnostic quality and efficiency.
Impact: By preserving imaging quality while reducing MRI scan times, patient convenience is substantially enhanced, allowing scalability across various contrasts.
Introduction
Multi-contrast MR reconstruction and super-resolution, harnessing the full potential of high-resolution (HR) fully-sampled images for the resolution-enhancement of under-sampled contrast images of low-resolution (LR) can provide substantial utility. Recently, many works, especially using complicated deep-learning techniques, are concentrating on developing a model to blend the information from both HR and LR scans 1,2,3,4,5. This fusion aims to utilize the detailed anatomical data from the reference to inform and enhance the resolution-enhancement process of the low-resolution scans. However, accurate and efficient segregation of the information to be fused is key to maximizing the use of high-resolution reference images while enhancing the reconstruction of low-resolution scans. This strategic separation enables a more effective utilization of the HR anatomical details, ensuring superior restoration of the LR images with greater fidelity to the original diagnostic features.In this work, we propose a novel multi-contrast MR super-resolution framework based on recent Swin-transformer6 based encoder-decoder with the separation of anatomy and contrast with two correlation-based losses. This enables the effective separation of anatomy features from HR scans, which is utilized in the reconstruction of the target low-resolution scans, as shown in Figure.1.
Method
Our frame work consists of two Swin-transformer based encoder-decoder architecture, as in Figure.1. First, the reference HR scans $$$X$$$ are fed into the encoder $$$\textit{E}$$$, with the output of F_X_C and F_X_A which each stand for contrast feature and anatomy feature. Similarly, the target LR images y~ is fed to the weight-shared encoder, with the output of F_Y_C and F_Y_A. Each contrast feature and anatomy feature are fed into decoder, for the reconstruction of HR scans. But, on the premise that both reference HR scans and LR scans have similar characteristics, we add positive correlation loss L_cor on F_X_A and F_Y_A. We chose positive correlation instead of identity loss, because even they share similar characteristics, the LR scans include artifacts and low-resolution features. For the disentanglement of anatomy and contrast, we give negative correlation loss L_Decor on the contrast feature pairs. With the aid of reconstruction losses between original scans and model output, the model successfully learns the anatomy feature of HR scans.For the reconstruction of target HR images, only anatomy features of reference HR scans are fed to the decoder D, with the contrast features from the target LR scans.
Experiment and results
We use FastMRI 7 knee dataset for the evaluation of the proposed method, with methods of 1. Bi-cubic interpolation, 2. U-Net with multi-contrast input. We set x2 resolution enhancement setting and the number of patients for training, validation and test are each 95, 92 and 30 respectively. Considering the clinical situation where acquisition time of proton-density (PD) is shorter than Fat-saturated PD (PD-FS) scans, we chose PD as a HR reference scan and PDFS as a LR target scan. We evaluated the super resolution results with the PSNR, which were 6.042 / 25.627 / 27.180 for each bi-cubic interpolation, multi-contrast input U-Net and ours. The qualitative results are shown in Figure.2, which shows the successful super-resolution results of PD-FS scans.Conclusion and Discussion
Our approach focuses on the precise integration of information from high-resolution reference MR images to inform the reconstruction of low-resolution target scans. By effectively separating the fusion material, we exploit the high-resolution anatomical details to their fullest, significantly enhancing the quality of low-resolution image reconstruction. This method not only ensures the retention of crucial diagnostic features but also opens avenues for the application to multiple contrast images.Acknowledgements
This work was supported in part by Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Science and ICT (2021R1A4A1031437, 2022R1A2C2008983). This work was also supported by Artificial Intelligence GraduateSchool Program at Yonsei University [No. 2020-0-01361], KIST Institutional Program (Project No.2E32271-23-078), and partially supported by theYonsei Signature Research Cluster Program of 2023 (2023-22-0008).References
[1] Li, Guangyuan, Jun Lv, Yapeng Tian, Qi Dou, Chengyan Wang, Chenliang Xu, and Jing Qin. "Transformer-empowered multi-scale contextual matching and aggregation for multi-contrast MRI super-resolution." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20636-20645. 2022.
[2] Feng, Chun-Mei, Huazhu Fu, Shuhao Yuan, and Yong Xu. "Multi-contrast mri super-resolution via a multi-stage integration network." In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VI 24, pp. 140-149. Springer International Publishing, 2021.
[3] Zeng, Kun, Hong Zheng, Congbo Cai, Yu Yang, Kaihua Zhang, and Zhong Chen. "Simultaneous single-and multi-contrast super-resolution for brain MRI images based on a convolutional neural network." Computers in biology and medicine 99 (2018): 133-141.
[4] Lyu, Jun, Bin Sui, Chengyan Wang, Yapeng Tian, Qi Dou, and Jing Qin. "DuDoCAF: Dual-domain cross-attention fusion with recurrent transformer for fast multi-contrast MR imaging." In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 474-484. Cham: Springer Nature Switzerland, 2022.
[5] Li, Guangyuan, Lei Zhao, Jiakai Sun, Zehua Lan, Zhanjie Zhang, Jiafu Chen, Zhijie Lin, Huaizhong Lin, and Wei Xing. "Rethinking Multi-Contrast MRI Super-Resolution: Rectangle-Window Cross-Attention Transformer and Arbitrary-Scale Upsampling." In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 21230-21240. 2023.
[6] Liu, Ze, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. "Swin transformer: Hierarchical vision transformer using shifted windows." In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022. 2021.
[7] Knoll, Florian, Jure Zbontar, Anuroop Sriram, Matthew J. Muckley, Mary Bruno, Aaron Defazio, Marc Parente et al. "fastMRI: A publicly available raw k-space and DICOM dataset of knee images for accelerated MR image reconstruction using machine learning." Radiology: Artificial Intelligence 2, no. 1 (2020): e190007.
Figures
