Introduction

Tissue segmentation plays a critical role in objective evaluation of the pathophysiological changes in multiple sclerosis (MS) that can help in patient management and evaluating the efficacy of treatment in multi-center clinical trials. Deep learning (DL), a subfield of machine learning, holds a great promise in automatic segmentation of brain MRI. Supervised DL requires large sets of annotated images for training, validation, and testing. CombiRx is a multicenter, double blinded phase III clinical trial with 1008 enrolled patients. MRI data on this cohort includes annotated 2D dual fast spin echo (FSE), fluid attenuated inversion recovery (FLAIR), high resolution 3D T1-weighted, and pre-and post-contrast 2D T1-weighted images. The accuracy of segmentation is thought to improve with multi-channel or multi-contrast input. However, the computational burden dramatically increases with the number of channels. Similarly, the major determinant of segmentation accuracy among the multiple contrasts is not known. In this study, we tried to answer these questions by investigating the segmentation accuracy by varying the number and combinations of contrasts to the input. The CombiRx trial in which multi-modal MRI data were acquired is ideally suited to answer this question.

Methods

U-net, a fully convolutional neural network (FCNN) architecture with proven success in semantic segmentation applications is implemented to investigate dependence of the segmentation accuracy on the number of input channels and channel combinations¹. The network architecture is shown in Fig. 1. The input to the U-net was 1006 (2 image volumes had to be discarded) pre-processed and annotated baseline MRI volumes from the CombiRx MRI database². Image volumes were partitioned into 604 (60%), 202(20%), and 200(20%) volumes for training, testing, and validation respectively. The number of epochs for training and validation was 1000 (1 epoch = 1 loop through the training set). Validation set was used to check the network performance at the end of each epoch, and the testing set was used for a final network assessment on new unseen data by the network. Network training was performed using weighted Dice similarity coefficient as the loss function³. Adam was used as the optimizer due to its adaptive learning rate for updating the network weights. FLAIR,T2-, proton-density-, and pre-contrast T1-weighted images served as the input images for tissue segmentation. The number of multi-contrast images and their various combinations used as inputs to the network are summarized in Table 1. All computations were performed on the Maverick2 cluster at Texas Advanced Computing Center (TACC). Maverick2 contains 4 GTX 1080Ti Graphical Processing Unit (GPU). The software was implemented in Keras using TensorFlow as backend^4,5.

Results

The Dice indices for each of the tissues as a function of the number of channels and their combinations are summarized in Table 1. From this Table, it can be observed that using all the available channels (4 input channels in the CombiRx data) provides the highest Dice scores for each tissue. As the number of input channels decreases, for certain combinations, we still obtain reasonable accuracy in a tissue-dependent manner. Also, the FLAIR sequence appears to contribute most to the segmentation accuracy. For single channel input, FLAIR alone provides acceptable accuracy for all tissues.

Discussion

We used DL to determine the number of channels and the contrast hierarchy on the segmentation accuracy of various brain tissues, including lesions. The segmentation accuracy does depend on the number of channels. However, acceptable accuracy can still be realized with a fewer number of channels. Also, FLAIR seems to be a major contributor to the segmentation accuracy. This is important since in a routine clinical scan the number of image contrasts is generally limited to keep the scan time short.

Conclusion

Acceptable segmentation accuracy can be obtained by using only the FLAIR images as the input to the network. This helps keep the scan time short for automatic tissue segmentation.

Acknowledgements

This work is supported in part by the NIH grant # 1R56NS105857-01 and funds from the Biomedical Engineering Chair. We acknowledge Texas Advanced Computing Center, Austin, TX for providing access to GPU clusters used in this study.