Introduction

Magnetic resonance (MR) imaging has become the preferred imaging technology for liver cancer treatment. The segmentation of intra-hepatic blood vessels is an important step in liver cancer treatment planning and surgery, accurate vessel segmentation can help to identify and avoid critical blood vessels during tumor ablation or resection. Deep learning-based methods have shown great promise in medical segmentation, but their clinical utility is limited by several challenges [1][2][3]. First, many hospitals do not have abundant data for training deep learning models to achieve their advertised performance. Second, the code or pre-trained model for many deep learning methods is not publicly available, making it difficult for hospitals to implement these methods in their own clinical settings.

In this study, we provide a clinically handy foundation model-based hepatic vessel segmentation method that is open source and can achieve a competitive performance with training on only five cases.

Methods

The emergence of foundation models, trained on massive and diverse datasets, has revolutionized intelligent model development. Driven by their remarkable generalization and few-shot learning capabilities, adapting pre-trained large models to diverse downstream tasks has become increasingly attractive, as opposed to the traditional approach of designing and training task-specific models from scratch. The Segment Anything Model (SAM), a recently developed visual foundation model for image segmentation, is pre-trained on a dataset of 11 million natural images. We employ a parameter-efficient few-shot learning strategy [4] to fine-tune SAM, enabling it to achieve competitive performance on the hepatic vessel segmentation task.

Key components in the fine-tuning strategy include: (1) To bridge the gap between 2D natural images and volumetric MR data, we incorporate a set of 3D adapters into each transformer block of the image encoder to extract the third-dimensional information. (2) To enhance the parameter tuning efficiency, we leverage the FacT technique [5] which retains most of the pre-trained SAM weights and only updates lightweight weight increments using as few as five cases. An overview of the fine-tuning framework is illustrated in Fig.1.

We use a proprietary MR dataset to validate the effectiveness of our method. Our dataset contained 60 MR scans with voxel resolution sizes ranging from (0.703, 0.703, 2.0) mm³ to (0.877, 0.877, 2.3) mm³ and a slice number ranging from 64 to 92 in the training data. All 60 scans were acquired from liver cancer patients and manually annotated by physicians. All samples were contrast-enhanced in the venous or delayed phase. In the validation process, we randomly chose a set of five cases for fine-tuning the SAM model, with an additional five cases earmarked for testing. This iterative process was repeated three times. The experiments were conducted using PyTorch 2.0, and all parameter configurations followed the guidelines outlined in the reference [4].

Results

We compare our five-shot fine-tuning method with the highly regarded nnU-Net [6]. The train-test split for nnU-Net was [50,10]. We employ the Dice score, precision, and Recall to assess the segmentation accuracy. For nnU-Net, these metrics yielded scores of 0.845, 0.878, and 0.818. In comparison, our method achieved slightly lower scores, registering values of 0.819, 0.825, and 0.803. Visually, both methods generated results that closely resembled the ground truth. It's noteworthy that, our method demonstrated the remarkable practicality of achieving competitive results with just 5 training cases, in contrast to state-of-the-art networks that necessitate a full ground-up training process. Qualitative visualization of the segmentation mask is presented in Fig.2. We also show the volumetric rendering of segmented hepatic vessels and liver in Fig.3.

Discussion and Conclusion

We have introduced a practical hepatic vessel segmentation method based on a versatile foundational model, designed to be readily applicable to all healthcare institutions. Our open-source approach attains competitive performance levels, even with training data derived from only five cases, potentially exerting a substantial influence on the clinical practices associated with liver surgery.

Acknowledgements

n/a