Introduction

The overarching goal of this work is to develop a computer-aided detection (CAD) approach for finding cancer metastases to the brain as it is well known that secondary cancer is responsible for a large percentage of fatality¹. Accurate detection of multiple metastases is required for designing radiation therapy, which, nowadays, can individually target up to 20 metastases^2,3. The clinical challenge is the small sizes and weak contrast of metastases on brain MRI. Adding to the challenge is that small blood vessels may manifest in similar signal intensity and appearance on a 2D MRI slices and require judicious examination to tell them apart from true brain metastases. We developed a CAD pipeline consisting of a region with convolutional neural network (R-CNN) and an autoencoder for detecting brain metastases.

Methods

We designed a CAD pipeline that consists of a Faster R-CNN and an autoencoder for first sifting through slices of an axial T1-weighted brain MRI for initial detection of brain metastases and then post-process the initial results for optimizing the final performance in terms of detection sensitivity and specificity. The work has been approved by our institutional review board. We retrospectively extracted brain MRIs of 101 patients who had been diagnosed with brain metastases. We retrieved data from 101 metastases patients for a total of 336 scans that had 1535 metastases in them. We retrieved axial 3D SPGR or MPRAGE T1-weighted contrast enhanced MR images sequentially over two years from clinical database. MRI scanners were of both 1.5T and 3.0T field strength from General Electric and Siemens. Imaging parameters were matrix size of 176-512 by 215-512; slice thickness of 0.8862-3.8396 mm; pixel size of 0.4102-1.0547 mm; TR of 6-2400 ms; TE of 1.88-12 ms; FOV of 69-100 mm; and flip angle of 8-150 degrees. The first step, which was Faster R-CNN, consisted it of five convolutional layers, two deconvolutional layers, one region proposal network (RPN) layer, and two fully connected layers⁴. The RPN window slid on the feature map generated by the convolutional layers and deconvolutional layers to obtain the proposals. In this work, we designed the RPN to have nine anchor boxes – for three different aspect ratios at three scales – to scan the feature map. The three aspect ratios were 0.5, 1, and 2. We set the area of the rectangular bounding boxes at the three scales 16², 32², and 64² pixels to capture the various sizes of possible brain metastases. The structure of the Faster R-CNN is shown in Figure 1. The output of the Faster R-CNN were bounding boxes overlaid on the slices of the axial MRIs that were found to have metastases. As inevitably, the outputs of the first step contain true and false detections, we designed a second step, an autoencoder, to reduce the number of false positives while trying to maintain the true positives unchanged. The structure of the autoencoder is shown in Figure 2. The autoencoder took as its input and obtained a new bounding box by finding a set of features of that minimized a cost function . Features can be considered as the most prominent characteristics about the bounding boxes. We then retrained a classifier on to differentiating between true and false positives in to further improve the performance of the whole CAD. In training the second neural network, we took advantage of prior knowledge about the fact that a true brain metastasis typically has a round or blob-like shape while small blood vessels resembles tubular objects across adjacent slices, so we designed the second neural network to take two slices above and two slices below the current slice to form a 2.5D model about the bounding box for classifying between metastases and blood vessels, Figure 3.

Results

For training the Faster R-CNN, we randomly separated the 101 patients into a training group of 73 cases and a testing group of 28 cases, which were the outputs of the Faster R-CNN. The 28 cases processed by the Faster R-CNN were then further separated randomly into two groups for training (20) and testing (8) the autoencoder. The training was implemented by a five-fold cross validation. The eight testing cases consisted of a total of 161 metastases. Our final result showed that the whole pipeline achieved a true positive rate of 0.92 by tumor (13 out of 161 metastases undetected) and a false positive rate of 18 per scan. The receiver’s operating characteristics curve is shown in Figure 4. Examples of final detection is shown in Figure 5.

Discussion

We found that deep learning-based approach can effectively detect brain metastases on MRI. With proper training and post-processing, we achieved a high sensitivity and good specificity. It is expected that, with an expanded training dataset and sophisticated modeling of small blood vessels, deep learning-based CAD can attain higher accuracy.

Conclusion

The relevance of this work to clinical practice is that, as the current clinical guideline requires a human reader to detect and mark as many as 20 brain metastases for radiation therapy planning, the proposed CAD approach can significantly reduce the time and improve the efficiency for diagnosing brain metastases.

Acknowledgements

The work of X.Cao, G.S.Young and X.Xu was supported by NIH award R01LM012434. The work of M.Zhang was supported by NIH award K99LM012874.