Introduction

During liver ablation surgery, the probe must avoid vessel to prevent damage to the patient, so we need to locate the hepatic vessel accurately. Vascular structures are sparse, anisotropic, and topologically complex. Simultaneously, MR images have different contrasts, and the intensity distortion caused by changes in the scanning plane and field inhomogeneity leads to blurred vessel edges and varying degrees of noise. Therefore, it’s challenging to perform vessel segmentation on MR images without vessel enhancement. Methods based on CNN/Transformer perform better for tasks with regular and dense targets, but they cannot guarantee vascular connectivity. Compared with regular convolutions, graph structures that allow arbitrary edge connections are more suitable for irregular data structures. Graph neural networks (GNN) allow for irregular receptive fields and can significantly improve vascular connectivity.

Methods

We developed a coarse-to-fine vascular connectivity enhancement network (CF-VCENet) to segment hepatic vessel based on the preoperative MR images of 105 patients with colorectal liver metastases who had previously undergone radical resection of the primary tumor without local recurrence or extrahepatic metastasis. Note that, the MR images were without vessel enhancement. The first stage is the initial segmentation network based on CNN/Transformer, and the second stage is the dual-stream graph attention segmentation refinement network. We selected nine 2D and 3D models with state-of-the-art medical image segmentation methods as the initial segmentation networks and generated superpixels on the segmentation results. We combined the initial network features, vessel probability map, superpixel segmentation results, and ground truth to generate a graph structure and fed it into the second-stage segmentation refinement network. The graph attention network (GAT) only pays attention to the spatial relationship of nodes and ignores the influence between node feature channels. Therefore, we innovatively proposed to combine the GAT and graph channel attention network (GCT) as a dual-stream feature extraction structure, focusing on the spatial attention and channel attention of nodes, respectively. Notably, we aggregated spatial information of a feature map by using both average-pooling and max-pooling operations. The segmentation refinement network is designed as a U-shaped network architecture using a hierarchical feature extraction method, in which modules at each level include GAT, GCT, a self-attention mechanism for balancing cross-view feature contributions, and graph pooling operations. In the encoder layer and the decoder layer of the same layer, we improved the skip connection. We designed a feature fusion module to fuse two features in the encoder spatial and channel coding path, which can provide supplementary information for the decoder. In particular, for spatial features, we developed a combination of 1×3 and 3×1 convolutions to extract the curve structures in vessel from different directions. Of note, our proposed GNN-based dual-stream graph attention vascular connectivity enhancement network is plug-and-play.

Results

Our proposed CF-VCENet outperformed the state-of-the-art vessel segmentation specialized networks and achieved the highest dice coefficient of 0.861±0.022 in the framework with UNETR++¹ as the initial network. Based on nine medical image segmentation networks, the GNN-based dual-stream graph attention vascular connectivity enhancement network we proposed significantly improved the segmentation results. Among them, the framework with 2D U-Net² as the first stage achieved the highest improvement in dice coefficient, from 0.810 to 0.828. The visualization results demonstrated that our method can significantly improve vascular connectivity and solve the false negative problem. We conducted ablation experiments on the modules proposed in the second-stage network, respectively, and the results further illustrated the positive impact of joint feature extraction by GAT and GCT on segmentation results. Specifically, the first proposed GCT considered the influence between graph nodes' channel features and achieved a 0.3% performance improvement with UNETR++ as the initial network.

Discussion

The second-stage segmentation refinement network uses a U-shaped structure to extract node features from low-level to high-level, and GAT and GCT pay attention to both the spatial attention and the channel attention of the graph node features. Therefore, our proposed method can significantly improve vascular connectivity. In addition, after generating superpixels from the initial segmentation results, a subset is selected as potential candidates for vascular connectivity enhancement. We set a low threshold to recruit more candidates so that the superpixels not included in the initial segmentation have the opportunity to be reconsidered as vessel in the second-stage network, which can solve the under-segmentation problem.

Conclusion

Our proposed CF-VCENet can accurately segment the hepatic vessel in MR images without vessel enhancement and ensure promising vascular connectivity. This study assists doctors in avoiding hepatic vessel during preoperative planning and reducing unnecessary damage to the patient's normal tissues.

Acknowledgements

The authors would like to acknowledge the supports of the Ministry of Science and Technology of the People's Republic of China (2023YFC2411400), Shanghai Jiao Tong University Medical Engineering Cross Research Funds (YG2021ZD05), Shanghai Hospital Development Center Foundation (SHDC12021112) and Shanghai Zhangjiang National Independent Innovation Demonstration Zone Special Development Fund Maior Project (GrantZJ2021-ZD-007).