0024
Deep Learning for Identification of Active Lesions in Multiple Sclerosis Without Administration of Gadolinium Based Contrast Agent1Diagnostic and Interventional Imaging, University of Texas Health Science Center at Houston, Houston, TX, United States, 2Neurology, Icahn School of Medicine at Mount Sinai, New York, NY, United States, 3Neurology, University of Texas Health Science Center at Houston, Houston, TX, United States
Synopsis
Gadolinium based contrast agents (GBCA) are routinely administered for identifying active lesions in multiple sclerosis (MS). Because of the safety concerns with GBCA, alternative methods are highly desirable to identify active lesions without GBCA. We used Deep Learning, specifically multi-layered VGG16 network, to identify active MS lesions on the pre-contrast images. The network was trained using a large number of annotated multi-modal magnetic resonance image volumes (792) acquired as a part of phase 3 clinical. The DL results look quite promising as judged by the accuracy, sensitivity, and specificity of 0.729, 0.861, 0.598, respectively.
Introduction
Not all lesions seen on MRI in multiple sclerosis (MS) patients are active. Gadolinium based contrast agents (GBCAs) are routinely used to identify enhancing lesions that are generally thought to represent active disease. However, recent studies have raised concerns about the safety profile of GBCAs1,2. This is particularly of concern for MS patients who are frequently administered GBCA as a part of clinical management. The administration of GBCA can be substantially reduced or completely eliminated if active lesions can be identified without administering GBCA.
Deep Learning (DL) is a subfield of machine learning has the potential to identify active lesions without GBCA administration. DL training involves calculating a large number of parameters (weights and biases) that requires substantial amounts of annotated data and high performance computational resources. CombiRx is a multicenter, double blinded phase III clinical trial with 1008 enrolled patients3. MRI data on this cohort includes dual fast spin echo (FSE), fluid attenuated inversion recovery (FLAIR), high resolution 3D T1-weighted, and pre- and post-contrast 2D T1-weighted images. We hypothesize that DL can identify active lesions on pre-contrast scans. This hypothesis is tested on the large CombiRx cohort.
Methods
Network Description: The method used in this study is based on transfer learning and involves fine-tuning the VGG16 network, which is a convolutional network (CNN) whose weights were optimized using the ImageNet4. CNNs consist of blocks of convolutional layers followed by max pooling layers and dense layers at the end to label images based on their features. Training procedure consists of two stages. First the VGG16’s dense layers are removed, keeping only the convolutional and max pooling layers. The training set is then iterated through the resulting architecture to generate the bottleneck features (Fig 1). These features are used to train new dense layers for identifying possible active lesions. In this training stage, Adam is used as an optimizer due to its fast convergence and weight-dependent learning rate. Binary-cross entropy is used as the model loss function along with rectified linear unit activation for all but the last layer, which has sigmoid activation. In the second stage, the last convolutional block and the dense layers are trained on the image features. During this training stage all other convolutional blocks in the network are frozen so that the weights of layers in those blocks are not updated. However, in this stage Stochastic Gradient Decent (SGD) optimizer is used, along with a small learning rate and high momentum. This configuration restricts major changes to weights in order to retain previously learned features. Loss and activation functions remain the same as in the first stage.
Input: Input images were obtained from the CombiRx baseline data. Pre- contrast T1-weighted, T2-weighted, and FLAIR images served as the input to the network. The enhancements in the post-contrast T1-weighted images in the CombiRx baseline images served as the ground truth. We used 5-fold cross validation to verify the results.
Computation: We implemented all processing on the Maverick2 cluster at the Texas Advanced Computing Center (TACC) with NVIDIA Tesla V100 graphics processing unit (GPU) cards. Implementation was carried out using the Python Keras library5 and TensorFlow6.
Results
Of the 1008 baseline post-contrast T1-weighted MRI volumes in the CombiRx data, 396 contained enhancing lesions. For equal representation of both classes (with and without enhancements), 396 patients without enhancing lesions were randomly selected from the rest of the image volumes. The training, validation, and test sets included 476 (60%), 158(20%), and 158(20%), respectively. The parameters extracted from the network include accuracy, sensitivity, specificity, predictive positive (PPV) and predictive negative predictive (NPV) values. These are summarized in Table 1.Discussion
An advantage of DL is that it is fully automatic. In this study we investigated the feasibility of applying DL to identify active lesions without administering GBCA. The predicted accuracy of 0.73 with DL is quite promising. As can be seen from the relatively high sensitivity, the network was able to predict cases which showed enhancement with high confidence. The high negative predictive value (NPV) suggests that the network can effectively identify lesions that do not enhance. The accuracy can be further improved by incorporating the T2 lesion information to exploit the fact that enhancements are associated with these lesions. Such studies are in progress.Conclusion
The ability to identify active lesions without GBCA administration using DL can potentially minimize the need for GBCA administration to improve patient safety and reduce costs associated with clinical care.Acknowledgements
This study is funded in part by NINDS/NIH grant #1R56NS105857-01 (PAN) and funds from Endowed Chair in Biomedical Engineering (PAN). We acknowledge the Texas Advanced computing Center, Austin, TX for providing access to the high performance computers.References
1 Fraum TJ, Ludwig DR, Bashir MR, Fowler KJ. Gadolinium-based contrast agents: A comprehensive risk assessment. J Magn Reson Imaging. 2017;46:338-353.
2 Runge VM. Critical Questions Regarding Gadolinium Deposition in the Brain and Body After Injections of the Gadolinium-Based Contrast Agents, Safety, and Clinical Recommendations in Consideration of the EMA's Pharmacovigilance and Risk Assessment Committee Recommendation for Suspension of the Marketing Authorizations for 4 Linear Agents. Invest Radiol. 2017 ;52:317-323.
3 Lublin FD, Cofield SS, Cutter GR, Conwit R, Narayana PA, Nelson F, Salter AR, Gustafson T, Wolinsky JS; CombiRx Investigators. Randomized study combining interferon and glatiramer acetate in multiple sclerosis. Ann Neurol. 2013;73:327-340.
4 Karen Simonyan K, Zisserman A. Presented at Very Deep Convolutional Networks for Large Scale Image Recognition. International. Conference on Learning Representations. arXiv:1409.1556.
5 Chollet F, l Barham P, Chen J et al., “Keras,” 2015. [Online]. Available: https://github.com/%0Afchollet/keras.
6 M. Abadi, Barham P, Chen J et al., “TensorFlow: A System for Large-Scale Machine Learning TensorFlow: A system for large-scale machine learning” in 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16), 2016.
Figures
