2145
Carotid plaque composition segmentation in multi-contrast MRI with U-net1Biomedical Engineering, Yale University, New Haven, CT, United States, 2Biomedical Engineering, Tsinghua University, Beijing, China, 3Bioengineering, University of Washington, Seattle, WA, United States
Synopsis
Carotid plaques may cause strokes, the composition of which is crucial for assessing the risk. While multi-contrast plaque magnetic resonance imaging (MRI) is a powerful technology, it is both tedious and error-prone for a radiologist to review these images, and traditional analytic algorithm relying on manually crafted features perform poorly as well.
We propose a novel approach with deep convolutional neural network (CNN), to be specific, U-net, to segment these plaque tissues. Some modifications on loss functions, convolution patterns and training protocols help our model perform well. On a dataset of 1098 subjects, we show that we achieve significantly better accuracy than previous models.
Background
Carotid atherosclerosisis caused by plaques, which are made up of cholesterol, fatty substances, cellular waste products, calcium and fibrin, clogging the arteries. As plaque builds up, the wall of the blood vessel thickens and narrows the channel within the artery, which reduces blood flow. That, in turn, reduces the amount of oxygen and other nutrients reaching the body. [1] Clinically, plaque compositions are important indicators of their risks. [2] For example, hemorrhagic plaques are vulnerable and highly likely to cause strokes once rupturing. Atherosclerosis plaques can be visualized by high-resolution multi-contrast MRI, and however, the detection and segmentation of plaque composition by radiologists requires intensive training and is also time consuming. In this work, we demonstrate a convolutional neural network (CNN) providing a much more efficient and accurate way for automated segmentation and analysis, compared with traditional Bayesian methods.Methods
The dataset in this study comes from Chinese Atherosclerosis Risk Evaluation study (CARE II) [3], which recruits over 1000 patients from 13 medical centers and hospitals in China, and are collected by Center for Biomedical Imaging Research (CBIR) of Tsinghua University. All MR imaging is performed on 3T MR scanners with 8-channel phase array coil, with the following imaging sequences: TOF, T1W, T2W and MP-RAGE. 16 slice locations per sequence per subject are kept with 4 images representing 4 different contrast weightings. Thus, we have 1098 subjects and 70272 images in total. Each 4 images of the same slice location are co-registered and shared with one pixel-level label. Labeled tissue classes include fibrous tissue, necrotic/lipid core, calcification, hemorrhage and loose matrix. Fig 1 summarizes the preprocessing steps.
U-net is employed as the base model. It is a fully convolutional neural network with a symmetric ‘encoder-decoder’ structure, which can be trained end-to-end from a moderate number of biomedical images. The input of U-net is modified to adapt 4-channel images, and to extract adequate features before merging these 4 imaging sequences. The output of U-net gives a labeled plaque composition map, as is shown in Fig 2.
Random crops and zooms are used for data augmentation. Masks of segmented vessel walls with a pretrained U-net [4] are multiplied to our input images for target localization, since the tissues we are interested in do not show up in lumens or image backgrounds. Focal loss [5] is introduced as the loss function, to tackle with data imbalance between easy and hard tissues. While the loss for well-classified examples are down-weighted, the focal modulating factor is near one and the loss is unaffected when an example is misclassified. Focal loss is then combined with dice loss to optimize apparent image similarity. Some of the convolution units are replaced with deformable ones [6], by adding learnable offsets to regular grid sampling locations to capture internal geometric transformation. A two-stage training scheme, with a lighter-load stage 1 (only images with lesions) and a full-data stage 2 (at a smaller learning rate), can significantly reduce training time and improve segmentation accuracy.
All the training and evaluation are performed with Keras (TensorFlow backend) as the deep learning framework, using an Ubuntu server with one Titan X GPU. We randomly select 20% subjects out of 1098 as the test set, while using the remaining for training.
Results
As can be seen from the evaluation metrics of image segmentation, U-net achieves a significant improvement over MEPPS (Morphology-Enhanced Probabilistic Plaque Segmentation) [7], and is able to complete the segmentation of fibrous tissue, calcification, necrotic/lipid core and hemorrhage. Despite the fact that we can accurately classify the dominating fibrous tissue and thus corresponding pixel accuracies for both methods are high, we also show that our result excluding this class is much higher than that of MEPPS. Table 1 & 2 detail the evaluation results. In some of the test cases, the segmentation of our network even surpasses that of manual labeling, suggesting existence of some compositions that radiologists sometimes fail to recognize, and giving reasonable contours as well (Fig 3 (d)(e), for example).Discussion
In this study, we apply a CNN (U-net) to automatically identify carotid plaque components. U-net outperforms MEPPS in almost all tissue classes and metrics. The improvement mainly comes from the elimination of manually crafted features, as well as the large amount of training data with consistent labels. It has great potential to become a reliable clinic tool in the future.Acknowledgements
No acknowledgement found.References
[1] Atherosclerosis (2017, Apr 30). Retrieved from http://www.heart.org/en/health-topics/cholesterol/about-cholesterol/atherosclerosis
[2] Yuan, C., Mitsumori, L. M., Beach, K. W., & Maravilla, K. R. (2001). Carotid atherosclerotic plaque: noninvasive MR characterization and identification of vulnerable lesions. Radiology, 221(2), 285-299.
[3] Zhao, X., Li, R., Hippe, D. S., Hatsukami, T. S., & Yuan, C. (2017). Chinese Atherosclerosis Risk Evaluation (CARE II) study: a novel cross-sectional, multicentre study of the prevalence of high-risk atherosclerotic carotid plaque in Chinese patients with ischaemic cerebrovascular events—design and rationale. Stroke and vascular neurology, e000053.
[4]Li,J., Chen, S., Zhao, X., Yuan, C., Li, R. (2018). Automatic Segmentation of Carotid Vessel Wall in Multi-Contrast Blackblood Images using Deep Learning. ISMRM 2018.
[5] Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2018). Focal loss for dense object detection. IEEE transactions on pattern analysis and machine intelligence.
[6] Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017). Deformable convolutional networks. CoRR, abs/1703.06211, 1(2), 3.
[7] Liu, F., Xu, D., Ferguson, M. S., Chu, B., Saam, T., Takaya, N., ... & Kerwin, W. S. (2006). Automated in vivo segmentation of carotid plaque MRI with morphology‐enhanced probability maps. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 55(3), 659-668.
Figures
