0662

Deep Learning-based Automatic Detection and Segmentation of Brain Metastases Using Multi-Task Learning with 3D Black-Blood and GRE Imaging

Yohan Jun*¹, Yae Won Park*², Yangho Lee¹, Kyunghwa Han², Chansik An³, Seung-Koo Lee², Sung Soo Ahn**², and Dosik Hwang**¹
¹Electrical and Electronic Engineering, Yonsei University, Seoul, Korea, Republic of, ²Department of Radiology and Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, Korea, Republic of, ³Research and Analysis Team, National Health Insurance Service Ilsan Hospital, Goyang, Korea, Republic of

Synopsis

For the detection of brain metastases, either contrast-enhanced 3D gradient echo (GRE) or spin echo (SE) imaging with black-blood (BB) imaging techniques are commonly used. The objective of this study was to evaluate whether a deep learning (DL) model using both 3D BB imaging and 3D GRE imaging may improve the detection and segmentation performance of brain metastases compared to that using only 3D GRE imaging. We demonstrated that the combined 3D BB and 3D GRE DL model can improve the detection and segmentation performance of brain metastases, especially in detecting small metastases.

Introduction

In recent years, several previous studies have proposed various deep learning (DL) methods for the detection and/or segmentation of brain metastases^1-6; however, most of the studies showed substantial numbers of false-positive (FP) findings with generally low segmentation performance (Dice coefficients lower than 0.8). Furthermore, the sensitivity was low in detecting small metastases (smaller than 7 mm). For the detection of brain metastases, either contrast-enhanced 3D gradient echo (GRE) or spin echo (SE) imaging with black-blood (BB) imaging techniques are commonly used^7-8. Compared to 3D GRE images, the 3D SE with BB imaging technique can suppress blood vessel signals, which enables clearer delineation and better detection of small brain metastases⁸. The purpose of this study was to evaluate whether a DL model using both 3D BB imaging and 3D GRE imaging may improve the detection and segmentation performance of brain metastases compared to that using only 3D GRE imaging.

Methods

Patient Population: The institutional review board waived the need for obtaining informed patient consent for this retrospective study. Between January 2018 and December 2019, 188 consecutive patients who followed our brain metastasis protocol including 3D BB imaging and were diagnosed with a newly developed brain metastasis were included in the training set. For the test set, between January 2020 and May 2020, 45 consecutive patients with a newly developed brain metastasis were included. Additionally, 49 patients without a brain metastasis were included after age and sex matching.
MRI Protocol: MRI was performed using various 3.0T MRI scanners (Achieva/Ingenia/Ingenia CX/Ingenia Elition X, Philips Medical Systems; Best, The Netherlands). The parameters for the contrast-enhanced fast SE sequence were as follows: TR/TE, 500/28.9-30 ms; flip angle, 90°, field of view, 20–24 cm; section thickness, 1 mm; matrix, 240×240. The iMSDE pre-pulse included one 90° excitation pulse, two 180° refocusing pulses, and one 90° excitation pulse with motion-sensitized gradients between radiofrequency pulses. The duration between the two 90° pulses was 28.3 ms, and the flow velocity encoding for gradient pulses was 1.3 cm/s. The imaging parameters for contrast-enhanced 3D GRE imaging were as follows: TR/TE, 5.9-8.6/2.8-4.7 ms; flip angle, 8°.
Deep Learning Architecture: To segment the brain metastasis, 3D U-net-based DL models were used. The overall architecture of our DL model is presented in Fig. 1. Three DL models were constructed depending on the input images (combined 3D BB and 3D GRE model, 3D BB model, and 3D GRE model). The model was trained with a 5-fold cross-validation. To regularize the encoder, an additional reconstruction decoder that reconstructed the input images from the extracted features was added at the end of the encoder, inspired by an autoencoder network9. The DL model was implemented using the Python Keras library with a Tensorflow backend. It was trained using the Adam optimizer with β₁=0.9 and β₂=0.999 for 500 epochs with a learning rate of 0.0001.
Statistical Analysis: The performances of the three models were assessed by using lesion-based sensitivity, positive predictive value (PPV), and Dice coefficient. The sensitivities of the combined 3D BB and 3D GRE model and the other two models were compared pairwise using a logistic regression analysis with the generalized estimating equation in a per-segment analysis. We also analysed the interaction between the subgroups with different sizes of metastases and DL models in the logistic regression analysis to assess whether the sensitivities differed based on the different subgroups on the DL models.

Results

A total of 282 patients were included in our study (mean age, 61.7±13.1; 129 females and 153 males). The total number of brain metastases was 1120 (917 and 203 in the training and test sets, respectively). The distributions of the number and sizes of the metastases across patients are shown in Fig. 2. Examples of true-positive inferences in various size that were detected on all three models are shown in Fig. 3. The detection sensitivities and PPVs for the DL models in the patients with brain metastases are summarized in Table 1. The combined 3D BB and 3D GRE model achieved an overall detection sensitivity of 93.1% and PPV of 84.8% for the patients with brain metastases in the test set. Specifically, for metastases <3 mm, ≥3 mm and <10 mm, and ≥10 mm, the sensitivities were 82.4%, 93.2%, and 100%, respectively. The overall sensitivity of the combined 3D GRE and 3D BB model was significantly higher than that of the 3D GRE model (93.1% vs 76.8%, p<0.001) (Fig. 3), and this effect was significantly stronger in subgroups with small metastases (p-interaction<0.001). The overall sensitivity and specificity for the combined 3D BB and 3D GRE model in detecting brain metastases in the patient-by-patient analysis were 100% and 69.4%, respectively. The FP per patient for the combined 3D BB and 3D GRE model, 3D BB model, and 3D GRE models was 0.59, 1.06, and 0.12, respectively. Compared to the combined 3D BB and 3D GRE model with a Dice coefficient of 0.822, the 3D BB and 3D GRE models showed similar or lower Dice coefficients of 0.827 and 0.756, respectively.

Conclusion

The combined 3D BB and 3D GRE DL model may improve the detection and segmentation performance of brain metastases, especially in detecting small metastases.

Acknowledgements

* Yohan Jun and Yae Won Park contributed equally to this work.

** Dosik Hwang and Sung Soo Ahn are co-corresponding authors.

This research received funding from the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, Information and Communication Technologies & Future Planning (2020R1A2C1003886). This research was also supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2020R1I1A1A01071648). This study was financially supported by the Faculty Research Grant of Yonsei University College of Medicine (6-2020-0149). This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (2019R1A2B5B01070488), Brain Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (2018M3C7A1024734), and Y-BASE R&E Institute a Brain Korea 21, Yonsei University.

References

[1] Kamnitsas K, Ledig C, Newcombe VFJ, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal. 2017;36:61–78.

[2] Charron O, Lallement A, Jarnet D, et al. Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network. Comput Biol Med. 2018;95:43–54.

[3] Grøvik E, Yi D, Iv M, et al. Deep learning enables automatic detection and segmentation of brain metastases on multisequence MRI. J Magn Reson Imaging. 2020;51(1):175–182.

[4] Zhang M, Young GS, Chen H, et al. Deep-Learning Detection of Cancer Metastases to the Brain on MRI. J Magn Reson Imaging. 2020;52(4):1227–1236.

[5] Xue J, Wang B, Ming Y, et al. Deep learning-based detection and segmentation-assisted management of brain metastases. Neuro Oncol. 2020;22(4):505–514.

[6] Zhou Z, Sanders JW, Johnson JM, et al. Computer-aided Detection of Brain Metastases in T1-weighted MRI for Stereotactic Radiosurgery Using Deep Learning Single-Shot Detectors. Radiology. 2020;295(2):407–415.

[7] Park J, Kim J, Yoo E, et al. Detection of small metastatic brain tumors: comparison of 3D contrast-enhanced whole-brain black-blood imaging and MP-RAGE imaging. Invest Radiol. 2012;47(2):136–141.

[8] Park J, Kim EY. Contrast‐enhanced, three‐dimensional, whole‐brain, black‐blood imaging: Application to small brain metastases. Magn Reson Med. 2020;63(3):553–561.

[9] Myronenko A. 3D MRI brain tumor segmentation using autoencoder regularization. International MICCAI Brainlesion Workshop. 2018;311–320.

Figures

Fig. 1. Structure of the deep learning (DL) algorithm. The DL model is based on 3D U-net, which consists of an encoder and two decoders: a segmentation decoder that reconstructs the segmentation maps and a reconstruction decoder that regularizes the shared encoder by reconstructing the input images.

Fig. 2. Bar charts showing the distribution of the (a) sizes and (b) number of brain metastases in the training and test sets, respectively.

Fig. 3. Examples of true-positive inferences that were detected on all three models on representative test patients with (a) 63.0-mm (b) 10.0-mm, and (c) 2.8-mm metastases, respectively.

Fig. 4. Examples of true-positive inferences that were detected on the combined three-dimensional (3D) black-blood (BB) and 3D gradient echo (GRE) model on representative test patients. (a) A 4.4-mm metastasis and (b) a 1.7-mm metastasis that were missed on the 3D GRE model but detected on the 3D BB model as well as the combined 3D BB and 3D GRE model.

Table 1. Diagnostic performance of the deep learning (DL) models according to different size of brain metastases in the test set. (Numbers in parentheses are 95% confidence intervals. * p-value refers to the significance among the differences of sensitivities between the combined 3D BB and 3D GRE model and the other model.)

Proc. Intl. Soc. Mag. Reson. Med. 29 (2021)

0662