4517

Generating 3D Volumetric Brain Images via Other Contrast Guidance from 2D Thick Slices
Long Wang1, Lei Xiang1, Ryan Chamberlain1, Xinyu Song2, Xiao-Er Wei2, and Yuehua Li2
1Subtle Medical, Menlo Park, CA, United States, 2Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China

Synopsis

Keywords: AI/ML Image Reconstruction, Multi-Contrast

Motivation: The conventional multiple sclerosis(MS) protocol in the brain normally takes from 30 minutes to several hours. It often involves multiple 3D series for the quantitative metrics and reports.

Goal(s): To reduce the scan time for the brain MS protocols with more than one 3D series

Approach: We proposed a framework that utilized a 3D T1 and 2D FLAIR as inputs to generate a 3D T2 FLAIR.

Results: The method generates comparable image resolution on all orientations as the acquired 3D FLAIR, and outperforms other methods. The lesion segmentation masks show high consistency. Furthermore, the method demonstrates robustness in the cases with motion.

Impact: The method provides the solution of generating a volumetric FLAIR using the same time as a 2D series and achieving comparable resolutions and diagnosis accuracy as the 3D FLAIR.

Introduction

The conventional MS protocol in the brain normally takes from 30 minutes to several hours. This protocol often involves multiple 3D series for the quantitative metrics and reports. It leads to motion artifacts and unpleasant patient experiences. In this work, we proposed a deep learning-based method to scan only one 3D sequence and a set of 2D sequences, and generate the 3D sequence using its 2D sequence and the sharable information from the 3D sequence from other contrasts.

Methods

With IRB approval and patient consent, 50 cases with the 2D FLAIR(initial orientation:axial, thickness:5.0mm, TE:85, TR:7000), 3D T1-weighted (initial orientation:sagittal, thickness:1mm, TE:2.98, TR:2000), and 3D FLAIR(initial orientation:sagittal, thickness:1mm, TE:394, TR:4500) were recruited for the study on a Siemens 3T scanner. The 3D FLAIR is the standard of care (SOC). Among them, 40 cases were used for training and the rest 10 for testing.

The proposed method is illustrated in Fig 1. In the training phase, the SOC and 2D FLAIR were mapped to the 3D T1-weighted using SimpleElastic1. Then a set of 64*64 patches with 3 channels (the adjacent three slides) are extracted from each input. Next, they were concatenated and fed into the network for training. Note that all the training patches are at the sagittal views. The network architecture is a stack of feature components, including a convolution layer as shallow feature extraction and then a set of five residual components with dual aggregation transformer units2. The method was trained with Adam optimization under learning rate as 0.0004 and weight decay as 0.0005. A mixed loss between L1 loss, SSIM loss, and DISTS loss3 was applied. In the test phase, only the input 2D FLAIR was mapped to the 3D T1-weighted images, and the full images were fed into the model weights to generate the 3D FLAIR.

It was observed that for some generated 3D FLAIR cases, the reformatted axial plane has spike artifacts in some areas. It is most likely from the 2.5D network. To remove the artifacts, additional all-plane super-resolution via implicit neural representations4 is applied per subject.

To evaluate the model performance, the inference results were compared with other thru-plane super-resolution methods, such as interpolation by BM4D5,6, and SMORE7. To evaluate the diagnosis accuracy, the MS lesion masks were drawn and compared with the SOC. Furthermore, cases with severe motion artifacts in the input were evaluated as a stress test.

Results

Figure 2 presents the comparison of our results with those obtained from similar techniques, such as BM4D and SMORE, as well as SOC. Figure 3 implies the lesion consistency when MS lesion segmentation is applied on the generated FLAIR and the acquired one. Figure 4 demonstrates how the model performs when the inputs include motion.

Conclusion and Discussion

In this study, we have developed a framework that utilizes shared information across contrasts to generate a 3D FLAIR image from the 3D T1 and a 2D FLAIR series. The generated results show comparable resolution in all orientations. The MS lesion mask on the inference results and the SOC also shows high consistency. Furthermore, the generated FLAIR remains the high resolution even when the inputs are in motion. However, with a larger range of testing, the current framework occasionally yields inference results with minor spiking artifacts at the axial view. These artifacts are effectively mitigated by implicit neural representation methods. Future work will be on expanding the current framework to accommodate more contrast combinations, and the generalization on various scanners and acquisition parameters.

Acknowledgements

No acknowledgement found.

References

1. Marstal, K., Berendsen, F., Staring, M., & Klein, S. (2016). SimpleElastix: A user-friendly, multi-lingual library for medical image registration. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 134-142).

2. Chen, Z., Zhang, Y., Gu, J., Kong, L., Yang, X., & Yu, F. (2023). Dual Aggregation Transformer for Image Super-Resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 12312-12321).

3. Ding, K., Ma, K., Wang, S., & Simoncelli, E. P. (2020). Image quality assessment: Unifying structure and texture similarity. IEEE transactions on pattern analysis and machine intelligence, 44(5), 2567-2581.

4. McGinnis, J., Shit, S., Li, H. B., Sideri-Lampretsa, V., Graf, R., Dannecker, M., ... & Wiestler, B. (2023, October). Single-subject Multi-contrast MRI Super-resolution via Implicit Neural Representations. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 173-183). Cham: Springer Nature Switzerland.

5. Eksioglu, E. M. (2016). Decoupled algorithm for MRI reconstruction using nonlocal block matching model: BM3D-MRI. Journal of Mathematical Imaging and Vision, 56, 430-440.

6. Maggioni, M., Katkovnik, V., Egiazarian, K., & Foi, A. (2012). Nonlocal transform-domain filter for volumetric data denoising and reconstruction. IEEE transactions on image processing, 22(1), 119-133.

7. Zhao, C., Dewey, B. E., Pham, D. L., Calabresi, P. A., Reich, D. S., & Prince, J. L. (2020). SMORE: a self-supervised anti-aliasing and super-resolution algorithm for MRI using deep learning. IEEE transactions on medical imaging, 40(3), 805-817.

Figures

Figure 1. The training framework. Note that LN(*) is the Layer-Norm layer, AC-SA is the adaptive channel self-attention module[2], Conv(*) is the convolution layer, AIM is the adaptive interaction module[2], DW-Conv(*) is the depth-width convolution layer, GELU(*) is the GELU layer.

Figure 2: Comparison of results across BM4D, SMORE, our method, and SOC. Each row from left to right represents BM4D, SMORE, Ours, and SOC respectively. The columns represent the three different views of each case. The views are ordered from top to bottom as sagittal, coronal, and axial.

Figure 3. The lesion masks for the generated 3D FLAIR and the acquired one. The top row represents the generated 3D FLAIR, and the bottom row represents the acquired FLAIR. The pink-colored mask shows the areas of MS lesions.

Figure 4. The case with input motion. From left to right is the input1, input2, the generated FLAIR from the two inputs, and the acquired FLAIR. Unlike other columns with 3D series, the second column is a 2D series and aligned at the nearest acquired position, not the exact position. The inputs and the acquired FLAIR image are with severe motion.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
4517
DOI: https://doi.org/10.58530/2024/4517