Introduction

Hepatocellular carcinoma (HCC) is the second leading cause of cancer-specific mortality globally ^1,2. According to multiple guidelines, thermal ablation (TA), surgical resection (SR), and liver transplantation (LT) are recommended as the first-line treatment in patients with HCC in early stage ^3,4. Unfortunately, the high recurrence rate after TA remains a great challenge in clinical practice ⁵. In this study, we proposed a deep learning method to predict the early recurrence (ER) in HCC patients after TA. Two innovations of the proposed method were: 1) integrating interconnected tasks, i.e., tumor segmentation and tumor progression prediction, into a unified model to perform the co-optimization(Tumor Aware); 2) using longitudinal images to take the therapy-induced changes into consideration to utilize the temporal information (Temporal).

Methods

Network Structure
The proposed Tumor Aware Temporal Deep Learning (TAP-DL) framework consisted of two subnetworks: one for tumor segmentation and one for ER prediction. The segmentation subnetwork was a DenseAttention UNet from our previous study ⁶ with multi-channel inputs, which were composed of gadoxetic acid–enhanced MR images with arterial phase (CEMR-A), two portal venous phases (CEMR-V1 and V2) and delayed phase (CEMR-D). ER prediction network was a ConvLSTM ⁷ which can leverage the temporal information in longitudinal images of pre- and post-operative MRI. Inputs at different time points (t=0 and 1) were sequentially processed by the segmentation network with shared parameters. Meanwhile, extracted low- and high-level image features were combined using ConvLSTM to generate the ER prediction result.
To integrate both the tumor segmentation and progression information, TAP-DL used a combined loss function:
$$\mathcal{L}_{total}=\mathcal{L}_{BCE}+{\alpha\mathcal{L}}_{DICE}$$
where $$$\mathcal{L}_{BCE}$$$ denoted the binary cross entropy loss that penalized the ER prediction error and $$$\mathcal{L}_{DICE}$$$ denoted the DICE loss that penalized the tumor segmentation error. $$$\alpha$$$ was empirically set to 0.1.
Compared Method
To explore the true benefit of combing different tasks into a unified network, the ablation studies were performed. First, the DenseAttention UNet was replaced by the ResNet18 ⁸ and features of each layer in ResNet18 were integrated into ConvLSTM to generate the ER prediction. This was a single task learning network for classification only, named ResT-DL; Second, only pre-operative MR images were used as the network input to seek whether the temporal information may provide benefit for ER prediction. This network performed both segmentation and classification, named TA-DL; Third, only single phase of MR image (CEMR-A) was used as the input of TAP-DL, in order to explore the effeteness of utilizing information of multi-phase images. This network was named sTAP-DL.
Experiment Settings
This study retrospectively included 228 HCC patients from two centers with institutional review board approval. Data from center 1 had 135 patients with 3657 multi-phase CEMR images and 70% of patients’ images for training and 30% patients’ images for validation. Data from center 2 had 93 patients with 2225 images for model testing and performance reporting.
CEMR images were acquired using 3D volume interpolated gradient echo sequence at 3.0T scanner1 (Achieva TX, Philips, The Netherlands) or 3.0T scanner2 (Prisma, Siemens, Germany) with gadoxetic acid enhancement. Sequence protocol was: TR=4.0-4.3ms, TE=1.2-2.1ms, FA=10-13^o, FOV=320x320mm², spatial resolution=1.5x1.5 – 2.0x2.0 mm², slice thickness=1-3 mm. Imaging preprocessing include intensity normalization using Z-score method and resolution normalization using nearest interpolation to 1x1x1 mm³.
Evaluation matrixes for ER prediction were area under ROC curve (AUC), accuracy (ACC), sensitivity (SEN), specificity (SPE), positive predict value (PPV) and negative predict value (NPV). Index for tumor segmentation was using DICE coefficient.
Network training lasted for 10 hours of 200 epochs with SGD as the optimizer. The learning rate was initially set to 1e-2 and reduced by half every 30 epochs after the first 75 epochs.

Results

As shown in Figure 2, in testing dataset TAP-DL achieved the highest AUC of 0.801 among all methods for ER prediction. Removing the segmentation subnetwork, the performance of ResP-DL was degraded and obtained AUC of 0.785. Additionally, using only pre-operative information or single phase images can cause accuracy decrease that TA-DL and sTAP-DL had AUC of 0.773 and 0.767.
Results in Table 1 showed the similar trend that TAP-DL can achieve the highest ACC SEN, SPE, PPV and NPV of among all other methods in testing dataset.
Figure 3 illustrates the segmentation results. TAP-DL can obtain highest DICE of 0.735 than TA-DL (DICE=0.706) and sTAP-DL (DICE=0.701), indicating simultaneously learning interconnected tasks may improve performance of both ones.

Conclusion and Discussion

TAP-DL can perform tumor segmentation and tumor progression prediction with higher performance than only doing any single one of them, demonstrating the effective and clinical potential of the proposed method.

Acknowledgements

None