0815

A Prior-Knowledge Embedded Convolutional Neural Network for Extracapsular Extension of the Prostate Cancer at Multi-Parametric MRI
Yihong Zhang1, Ying Hou2, Jie Bao3, Yang Song1, Yu-dong Zhang2, Xu Yan4, and Guang Yang1
1East China Normal University, Shanghai, China, 2the First Affiliated Hospital with Nanjing Medical University, Nanjing, China, 3the First Affiliated Hospital with Soochow University, Soochow, China, 4Siemens Healthcare, Shanghai, China

Synopsis

We proposed an algorithm to incorporate radiologist’s prior-knowledge about location of extension into a CNN model to diagnose the extracapsular extension of the prostate cancer from multiparametric MRI (mpMRI). The model was trained on 596 cases with ensemble learning before validated with an independent validation cohort of 150 cases and an external cohort of 103 cases. Our proposed model achieved an area under receiver operating characteristic curve (AUC) of 0.807/0.728 on the internal/external test cohort, which is better than the traditional model (AUC=0.746/0.723) and the clinical reports by two radiologists (AUC=0.725, 0.632/0.694, 0.712).

Introduction

Multiparametric magnetic resonance imaging (mpMRI) together with the prostate imaging reporting and data system (PI-RADS) are widely used to diagnose the prostate cancer (PCa)1. Accurate preoperative assessment of extracapsular extension (ECE) of the tumor can benefit the patient by impacting the surgical strategy2. However, the diagnosis accuracy depends on the clinical experience and significant variances exist among different readers.

Convolutional neural networks (CNN) has been widely used in computer-aided diagnosis (CAD) and achieved encouraging results3. However, insufficient interpretability of the model remains a major impediment to the clinical applications of CNN-based CAD systems.

In our study, we proposed a prior-attention guided network (PAGNet) which incorporated the radiologist’s prior-knowledge to diagnose ECE. The PAGNet was compared with the ResNeXt4 model, and the clinical reports on both internal and external testing dataset. (Figure 1)

Methods

The data of our retrospective study was from two institutions, and both of them were approved by institutional ethics committee. 746 cases (191 cases with ECE) were collected from Jiangsu Province Hospital (JSPH) on two 3T scanners (Skyra of Siemens Healthcare and MR770 of United Imaging Healthcare). Another 103 (33 positive) cases scanned on Siemens Healthcare 3T Skyra were from the First Affiliated Hospital of Soochow University (SUH). We used the scanning parameters suggested by PI-RADS v21 to get the T2-weighted images (T2WI), diffusion weighted images (DWI) with b-value=1500mm2/s, and the corresponding apparent diffusion coefficient (ADC) map. We randomly split the data from JSPH into 596 (151 positive) training cohort and 150 (40 positive) internal testing cohort. And the data from SUH was used to external testing cohort.

We aligned the DWI and ADC map to the T2WI with Elastix, and resampled all sequence with an intra-slice resolution of 0.5mmx0.5mm. A radiologist with 12-year experiences labeled the region of interest (ROI) of the PCa and the prostate for each case manually. Then an attention map was calculated from these two ROIs to locate the candidate region of the ECE (Figure 2). We extracted the slice with the largest area of the PCa and cropped the resampled images from all sequences to 280x280. All images were normalized by Z-score and augmented by random affine transformation.

PAGNet was designed based on a modified the convolutional block attention module (CBAM)5 with the combination of the spatial attention and the prior attention map. Then CBAM was embedded in ResNeXt to analyze the MR images for the diagnosis of ECE. (Figure 3)

We used a 5-fold cross validation for ensemble learning and the average of predictions of five ensemble models was the final diagnosis. We also used gradient-weighted class activation mapping (Grad-CAM)6 to interpret the model output.

We compared the performance of PAGNet with those of ResNeXt with CBAM but without prior-attention, and the clinical reports by two radiologists based on the 3-score grading system7. Confusion matrix and receiver operating characteristic (ROC) curve were used to evaluate the model. We used Delong test to compare the model performances and set the level of significance to 0.05.

Results

The ROC curve of the models and the diagnosis of the two radiologists were plotted in Figure 4 and the clinical statistics were listed in Table 1. On the internal test cohort, PAGNet achieved higher AUC (0.807) than ResNeXt (0.746) and the clinical reports (0.725, 0.632) (Figure 4 (a), p<0.05). On the external test cohort, PAGNet also achieved performance comparable to those of radiologists (Figure 4 (b)).

We chose two cases randomly and showed their Grad-CAM in Figure 4. It can be seen that PAGNet focused on the candidate region for the positive case and a relative larger region of the boundary of the prostate for the negative case.

Discussion and Conclusion

We proposed a PAGNet with a prior-knowledge to guide the model to learn from the radiologist to notice the region of PCa near the edge of prostate and diagnosis the ECE. The PAGNet achieved a higher AUC on internal test cohort and comparable AUC on the external test cohort than ResNeXt and the clinical reports. We also used Grad-CAM to interpret the output visually, showing the area that model paid special attention to when making diagnosis, which could help radiologist understand the judgment basis of the model, provided reference for them on difficult cases. Though PAGNet was validated on independent validation cohorts from three MRI scanners in two institutions, a more general validation involving more cases from more institutions is still desired. A 3D model built from more data may use more information of the whole gland of the prostate to give more accurate diagnosis. Site-specific transfer learning and conditional generative adversarial network8 may help build a more robust model from limited data.

In conclusion, we proposed PAGNet which embedded prior-knowledge about location of extension to diagnose the ECE from mpMRI. The addition of the prior-attention not only improved the result of the model, but also guided model paid attention to the region where ECE occurred.

Acknowledgements

No acknowledgement found.

References

1. Gupta RT, Mehta KA, Turkbey B, Verma SJJoMRI. PI‐RADS: Past, present, and future. Journal of Magnetic Resonance Imaging. 2020;52(1):33-53.

2. Roethke MC, Lichy MP, Kniess M, et al. Accuracy of preoperative endorectal MRI in predicting extracapsular extension and influence on neurovascular bundle sparing in radical prostatectomy. World journal of urology. 2013;31(5):1111-1116.

3. Shen D, Wu G, Suk H-I. Deep learning in medical image analysis. Annual review of biomedical engineering. 2017;19:221-248.

4. Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated residual transformations for deep neural networks. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017:1492-1500.

5. Woo S, Park J, Lee J-Y, So Kweon I. Cbam: Convolutional block attention module. Proceedings of the European conference on computer vision (ECCV), 2018:3-19.

6. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE international conference on computer vision, 2017:618-626.

7. Mehralivand S, Shih JH, Harmon S, et al. A grading system for the assessment of risk of extraprostatic extension of prostate cancer at multiparametric MRI. Radiology. 2019;290(3):709-719.

8. Mirza M, Osindero S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. 2014.

Figures

Figure 1 The workflow of the research. We trained the PAGNet on training cohort from one-site, and evaluated its performance with cohorts from both institutions. The performance of the proposed model was compared with those of a ResNeXt model and two radiologists.

Figure 2 The boundary of prostate and prostate cancer (PCa) labeled manually (top row) and the attention map (bottom row). Three cases chosen randomly were illustrated: PCa extension (left), gland boundary contacting (middle), and not contacting (right). We calculated the attention map by the rule: as to a voxel, if in the PCa but out of prostate, attention value was 100%; if in PCa and prostate, the attention value was the reciprocal of the distance from voxel to prostate; if out of the PCa, attention value was the reciprocal of the maximum of distance from voxel to prostate and PCa.

Figure 3 The architecture of the PAGNet. Four blocks included 3/4/4/3 bottlenecks with 16/32/64/96 filters. In each bottleneck, the features were convoluted, batch normalized, and activated by ReLU. Then the features were processed by a channel attention(green), and two spatial attention with self-learned (blue) and a prior-attention map, respectively. We concatenated these features and added the initial feature in the bottleneck. At the end of the four bottleneck blocks, average pooling and two fully-connected layers with a dropout (ratio=0.5) were followed to predict the ECE.

Table 1 The statistics of the model performance and the two radiologists on two testing cohorts.

Figure 4 ROC curves (top) and Grad-CAM of PAGNet (bottom). PAGNet performed better than ResNeXt model and clinical reports in the internal test cohort (a), and achieved results comparable to the radiologists on the external cohort (b). The Grad-CAMs of the model on one positive case (c) and one negative case (d) were also shown, in which the red regions indicate where the model paid attention.

Proc. Intl. Soc. Mag. Reson. Med. 29 (2021)
0815