5000
ASN: Adaptive Segmentation Network for Visual Pathway Identification in Multi-parametric MR Images1Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China, 2University of Chinese Academy of Science, Beijing, China, 3Zhejiang University of Technology, Hangzhou, China, 4Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application, Guangzhou, China, 5Peng Cheng Laboratory, Shenzhen, China
Synopsis
Keywords: Machine Learning/Artificial Intelligence, Visualization, Adaptive convolution, visual pathway segmentation, deep learning
Motivation: Accurate visual pathway (VP) segmentation is critical for clinical diagnosis and surgical planning. Current deep learning-based methods struggle to capture significant context information, impacting the segmentation precision.
Goal(s): Improve multi-parametric MRI-based VP segmentation by designing an Adaptive Segmentation Network (ASN).
Approach: ASN uses adaptive convolution (AC) to dynamically adjust the kernel based on complementary context, facilitating the integration of contextual information. A spatial attention block selectively extracts relevant regions‘ features in each MRI sequence and fuses them.
Results: ASN's effectiveness is validated by segmenting the VP in MR images from two MRI sequences. It surpasses state-of-the-art techniques in VP segmentation.
Impact: The introduction of ASN, a new multi-parametric MR images segmentation approach, demonstrates superior performance in visual pathway (VP) segmentation in MR images, surpassing existing state-of-the-art techniques. This novel method effectively incorporates context information, leading to improved segmentation performance.
Introduction
Multi-parametric magnetic resonance imaging (MRI) plays a crucial role in the diagnosis and treatment of various diseases affecting the visual system, such as optic neuritis and optic nerve glioma 1, 2. By combining different MRI sequences, such as T1-weighted and fractional anisotropy (FA), a more comprehensive understanding of the visual pathway can be obtained compared to isolated modalities. For instance, T1-weighted imaging is useful in detecting the anatomy situation, while FA can provide information about the integrity of white matter tracts. Accurate segmentation of visual pathway (VP) in multi-parametric MRI is essential for diagnosis and surgical planning, but manual segmentation is time-consuming and prone to errors. Therefore, the development of automatic and reliable algorithms for multi-parametric MRI-based VP segmentation is highly valuable for clinical practice.During the past decade, many multi-parametric MRI segmentation methods have been developed 1-4. These methods can be mainly divided into three categories: early fusion, late fusion, and intermediate layer fusion.
Although these methods have achieved good segmentation results, there still remain some challenges: (i) Current DL-based multi-parametric MRI segmentation methods rely on convolution operations from source images, but they struggle to model long-range context dependencies. However, neuroscience research suggests that the global context capability of neurons is crucial for effectively processing complex perceptual issues 5, 6. (ii) Current DL-based multi-parametric MRI segmentation methods still have limitations in modeling the complex inter-sequence relationships.
To overcome the above challenges, we propose an Adaptive Segmentation Network (ASN), which uses adaptive convolution (AC) to adjust the convolutional kernel based on complementary context. This allows for effective incorporation of global context information and improves the segmentation performance. Additionally, we introduce a spatial attention block to selectively extract features from relevant regions and fuse them. Experimental results on the human connectome project (HCP) dataset validate the superior performance of ASN compared to state-of-the-art techniques.
Methodology
As per neuroscience studies on the importance of global context information in perceptual tasks, we propose a supervised multi-parametric MRI segmentation approach called Adaptive Segmentation Network (ASN) for segmenting multi-parametric MRI images (Figure 1). ASN consists of two key components: AC and spatial attention.Inspired by 7, 8, we use a U-like encoder-decoder structure with standard convolutions being replaced by adaptive convolutions. Specifically, AC modifies the weights of convolutional layers in CNNs based on global context information. By adaptively modulating the convolution kernels, AC enables the extraction of representative local patterns and the composition of discriminative features guided by the global context, leading to improved segmentation performance.
The spatial attention block enables the model to selectively attend to relevant regions or features in each MRI sequence and fuse them. The spatial attention mechanism is implemented by using the adaptive convolution layers in place of standard convolution layers to learn a set of spatial attention maps, which are then multiplied with the feature maps of each imaging sequence to emphasize the most informative regions.
Results and Analysis
In our study, we evaluated the performance of our proposed Adaptive Segmentation Network (ASN) on the HCP dataset (training set: 82, testing set: 10) and compared it with existing state-of-the-art techniques.As shown in Table 1, ASN achieved a Dice similarity coefficient (DSC) of 87.7%, a Hausdorff distance (HD) of 2.13 mm, and an average symmetric surface distance (ASD) of 0.13 mm, surpassing state-of-the-art techniques like FuseNet 2 and TPSN 1. This demonstrates the accurate VP region segmentation capability of ASN. The higher DSC score indicates effective capture of true positives and reduction of false positives. The lower HD and ASD value suggests precise delineation of VP boundaries. Besides, the qualitative results illustrated in Figure 2 confirmed the quantitative results.
The superior performance of ASN can be attributed to its utilization of AC and spatial attention mechanisms. These results underscore the potential of ASN in improving clinical diagnosis and surgical planning. Accurate VP segmentation aids in the identification of diseases affecting the VP, enabling targeted treatment and intervention.
Conclusion
In summary, the proposed Adaptive Segmentation Network (ASN) demonstrated superior performance in segmenting VP in multi-parametric MRI images compared to state-of-the-art techniques. The incorporation of adaptive convolution and spatial attention mechanisms allowed to effectively capture global context information and complex inter-sequence relationships. This improved the segmentation accuracy, as evidenced by high DSC, low HD, and low ASD.ASN has potential applications in clinical diagnosis and surgical planning by aiding in the identification of abnormalities or diseases affecting the visual pathway. Further research can explore the use of ASN in other medical imaging tasks and its generalizability to different datasets.
Acknowledgements
This research was partly supported by the National Natural Science Foundation of China (62222118, U22A2040), Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application (2022B1212010011), Shenzhen Science and Technology Program (RCYX20210706092104034, JCYJ20220531100213029), and Key Laboratory for Magnetic Resonance and Multimodality Imaging of Guangdong Province (2023B1212060052).References
1. S. Li, Z. Chen, W. Guo, Q. Zeng, and Y. Feng, “Two parallel stages deep learning network for anterior visual pathway segmentation,” in Computational Diffusion MRI: International MICCAI Workshop, Lima, Peru, October 2020. Springer, 2021, pp. 279–290.
2. L. Xie, L. Yang, Q. Zeng, J. He, J. Huang, Y. Feng, E. Amelina, and M. Amelin, “Deep Multimodal Fusion Network for the Retinogeniculate Visual Pathway Segmentation,” in The 42nd Chinese Control Conference (CCC 2023), 2023.
3. Cheng Li, Hui Sun, Zaiyi Liu, Meiyun Wang, Hairong Zheng, and Shanshan Wang, “Learning cross-modal deep representations for multi-modal MR image segmentation,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22. 2019, pp. 57–65, Springer.
4. Yang, Q., Guo, X., Chen, Z., Woo, P.Y. and Yuan, Y., 2022. D 2-Net: Dual disentanglement network for brain tumor segmentation with missing modalities. IEEE Transactions on Medical Imaging, 41(10), pp.2953-2964.
5. Li, W., Piëch, V. and Gilbert, C.D., 2004. Perceptual learning and top-down influences in primary visual cortex. Nature neuroscience, 7(6), pp.651-657.
6. Gilbert, C.D. and Li, W., 2013. Top-down influences on visual processing. Nature Reviews Neuroscience, 14(5), pp.350-363.
7. Lin, X., Ma, L., Liu, W. and Chang, S.F., 2020. Context-gated convolution. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16 (pp. 701-718). Springer International Publishing.
8. Tang, W., He, F., Liu, Y. and Duan, Y., 2022. MATR: Multimodal medical image fusion via multiscale adaptive transformer. IEEE Transactions on Image Processing, 31, pp.5134-5149.
Figures
