1879
Automatic deep learning segmentation of grey and white matter lesions in 7T MRI data1Division of Radiology, German Cancer Research Center (DKFZ), Heidelberg, Germany, 2Faculty of Medicine, Heidelberg University, Heidelberg, Germany, 3Division of Medical Physics in Radiology, German Cancer Research Center (DKFZ), Heidelberg, Germany, 4Faculty of Physics and Astronomy, Heidelberg University, Heidelberg, Germany
Synopsis
Multiple sclerosis is a chronic inflammatory disease characterized by demyelination. Magnetic resonance imaging (MRI) is an important method for diagnosis and prognosis predictions. The ongoing study presented here shows the use of deep learning algorithms for white and grey matter lesion segmentation in 7T MRI images. Results show high accuracy for patients with high lesion load. Furthermore, it is demonstrated that it is possible to train a neural net to find small cortical lesions, which can be used as a potential biomarker.
INTRODUCTION –
Although manual segmentation of multiple sclerosis (MS) lesions is still the gold standard, it is highly subjective and barely feasible in longitudinal large cohort studies. Therefore, automatic lesion segmentation has been an active research topic for more than 20 years.1 However, research has predominantly been focused on segmenting white matter lesions. Only recently, a number of studies also attempted to segment grey matter lesions, which are often much smaller.2,3 Exploiting the higher resolution and contrast-to-noise ratio achievable using a 7T magnetic resonance imaging (MRI) scanner, small white matter, subcortical white matter, and even grey matter lesions can be detected.4 However, to the best of our knowledge, the use of deep learning architectures like the U-net have not yet been used to develop automatic lesion detection in 7T data. The possibility of more precise segmentation and localization of small cortical lesions can improve prognosis predictions and diagnosis. As manual segmentation is time consuming, automation of this procedure is required, in particular when the number of detectable lesions is large.METHODS –
The study was conducted in accordance with the Declaration of Helsinki. All subjects provided written informed consent and institutional review board approval was received. Twenty-nine patients were scanned on a 7T whole-body scanner (MAGNETOM 7T, Siemens Healthcare, Germany). Some patients were scanned multiple times, and the total number of measurements was 35. The following sequences were used: 3D multi-echo gradient echo (GRE), 3D MP2RAGE, 2D multi-echo turbo spin echo (TSE), and a pre-saturation‐based 2D turbo flash for B1 mapping (Table 1). All sequences were registered using FSL-FLIRT and the Medical Imaging Interaction Toolkit (MITK).5,6 Brain masks were created with the FSL Brain Extraction Tool by using the first echo of GRE.7 This brain mask was used to create brain extracted images with FSLMaths. The B1 maps were used to correct for the field inhomogeneities of the MP2RAGE with github.com/JosePMarques/MP2RAGE-related-scripts.Ground truth lesion segmentations were drawn manually on the MP2RAGE data and on the first echo of the T2-weighted turbo spin echo data in MITK.6 All lesions that included cortical gray matter were classified as cortical lesions (CL).8,9 Both modalities were segmented separately and a merged mask was generated from the T1 and the T2 masks that included a lesion if there was a lesion in either of the two masks.
The data were used for training within the fully-automated nnU-net framework, which is an improved version of the U-net architecture. 10,11 The nnU-net provides a 2D, 3D, and a Cascade U-net model. In this study, only the 3D model was trained on the corrected MP2RAGE data, the T2-weighted data, and on both modalities. The 35 measurements were randomly divided into training and test sets, whereby the latter consisted of seven measurements and all measurements of a single patient were fully contained either in the training or in the test set. For each training set, the nnU-net was trained on a NVIDIA P6000 GPU for about 65 hours. Ground truth lesion masks and automatically generated lesion segmentations were evaluated using the Dice coefficient (DSC), true positive rate (TPR), and volume difference (VD).12
RESULTS –
Figure 1 illustrates the capability of the nnU-net to segment small lesions on representative slices containing grey matter lesions by comparing manual and automatic segmentations. The displayed slices demonstrate CL-I on MP2RAGE, while manual segmentations belong to the merged masks. The nnU-net provides smooth and comprehensible segmentations.Figure 2 shows median DSC for white and grey matter segmentation. Values for median DSC are 0.654 for T1, 0.720 for T2, and 0.706 for T1+T2. Further evaluation metrics are listed in Table 2.
Figure 3 shows a boxplot of DSC and TPR only for the segmentation of cortical lesions in the T1+T2 set. In M5+M6 no CL was found in either. For the remaining data a median DSC of 0.695 and a median TPR of 0.624 were calculated. It is worth mentioning that every CL in the test set was leukocortical, and the volumes of cortical lesions were low (average cortical lesion volume of 0.572ml in the test set).
DISCUSSION AND CONCLUSION –
In comparison to architectures segmenting only white matter lesions of MS patients, to be seen on the leaderboard (https://smart-stats-tools.org/lesion-challenge) of the 2015 longitudinal MS lesion segmentation challenge, the performance in our study was similar. The DSC of the currently best architecture has a median of 0.686. Furthermore, the same correlation between low lesion load and poor DSC was observed. For the segmentation of cortical lesions, our results are comparable to studies segmenting cortical MS lesions.13 However, this has to be evaluated on a larger test set with a higher number of cortical lesions in the future. Automatic machine learning based lesion segmentation combined with the high resolution available when using 7T MRI could therefore help to detect a higher number of small cortical lesions. In future, automatic lesion detection with 7T data could improve disease staging and help explore the impact of grey matter lesions in multiple sclerosis.Acknowledgements
The nnU-net pipeline, publically available at github.com/MIC-DKFZ/nnUNet by Fabian Isensee et al., is kindly acknowledged. The Quadro P6000 used for this research was donated by the NVIDIA Corporation.References
1. D. Garcia-Lorenzo, S. Francis, S. Narayanan, D.L. Arnold, D.L. Collins, Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging, Medical image analysis, 17 (2013) 1-18.
2. M.J. Fartaria, G. Bonnier, A. Roche, T. Kober, R. Meuli, D. Rotzinger, R. Frackowiak, M. Schluep, R. Du Pasquier, J.P. Thiran, G. Krueger, M.B. Cuadra, C. Granziera, Automated Detection of White Matter and Cortical Lesions in Early Stages of Multiple Sclerosis, Journal of Magnetic Resonance Imaging, 43 (2016) 1445-1454.
3. J. Emmerich, S. Flassbeck, S. Schmidt, P. Bachert, M.E. Ladd, S. Straub, Rapid and accurate dictionary-based T2 mapping from multi-echo turbo spin echo data at 7 Tesla, Journal of magnetic resonance imaging : JMRI, (2018).
4. Nielsen AS, Kinkel RP, Tinelli E, Benner T, Cohen-Adad J, Mainero C. Focal cortical lesion detection in multiple sclerosis: 3 tesla DIR versus 7 Tesla FLASH-T2. J Magn Reson Imaging. 2012;35(3):537-542.
5. Jenkinson, M., Bannister, P., Brady, J. M. and Smith, S. M. Improved Optimisation for the Robust and Accurate Linear Registration and Motion Correction of Brain Images. NeuroImage, 17(2), 825-841, 2002.
6. Maleike D, Nolden M, Meinzer HP, Wolf I. Interactive segmentation framework of the Medical Imaging Interaction Toolkit. Comput Meth Prog Bio 2009;96(1):72-83.
7. S.M. Smith. Fast robust automated brain extraction. Human Brain Mapping, 17(3):143-155, November 2002.
8. Calabrese M, Filippi M, Gallo P. Cortical lesions in multiple sclerosis. Nat Rev Neurol 2010;6:438-444
9. Mainero C, Benner T, Radding A, van der Kouwe A, Jensen R, Rosen BR, Klinkel RP. In vivo imaging of cortical pathology in multiple sclerosis using ultra-high field MRI.
10. Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F. Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, Sebastian Wirkert, Klaus H. Maier-Hein. "nnU-Net: Breaking the Spell on Successful Medical Image Segmentation." arXiv preprint arXiv:1904.08128 (2019).
11. O. Ronneberger, P. Fischer, and T. Brox, \U-net: Convolutional networks for biomedical image segmentation,” in MICCAI.Springer, 2015, pp. 234-241
12. Taha, A. A. and A. Hanbury "Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool.”
13. Mario Joao Fartaria, Guillaume Bonnier, Alexis Roche, Tobias Kober, Reto Meuli, David Rotzinger, Richard Frackowiak, Myriam Schluep, Renaud Du Pasquier, Jean-Philippe Thiran, Gunnar Krueger, Meritxell Bach Cuadra, Cristina Granziera. (2016). "Automated detection of white matter and cortical lesions in early stages of multiple sclerosis." J Magn Reson Imaging 43(6): 1445-1454.
Figures
