Background

MS lesions are generally analyzed by semi-automatic or manual methods, often with several raters. Inter-rater variability presents a major limitation in quantitative lesion volume analysis, stemming from inconsistencies in contouring the diffuse edges of hyperintensities. More consistent contouring is a key promise of automatic segmentation, and rapid improvements in deep learning (DL) have shown new promise for fully automatic lesion detection and segmentation. However, convolutional neural networks (CNNs) approaches effectively still determine lesion boundaries by thresholding, which can transfer poorly across different data sets. More fundamentally, diffuse hyperintense image regions by definition lack a discrete boundary. As DL based segmentation is still an emerging method, there are additional notable limitations such as their architectural tradeoffs and dependence on the quantity and accuracy of training data. Therefore, we aimed to develop a new DL framework pairing a CNN and a custom surface feature that could detect hyperintense isocontour in 3 dimensions very sensitively, bounded by the image noise floor. Our goal is to approach the quantification of lesion volume as integrated image hyperintensity, rather than spatial volume inside a boundary, by training a DL algorithm on the combined traditional imaging and specially designed edge feature data.

Methods

Imaging MRI (MPRAGE and FLAIR) were performed on 73 MS patients and 35 healthy controls at 3T. Automatic segmentation of gray matter, white matter and cerebrospinal fluid was performed using SPM8, and manual lesion contouring using Jim6 software (Xinapse Systems). Lesion rating and manual contouring were performed by a neurologist specializing in MS and trained lesion drawers. Training and validation data were split 80%/20% from MRI data sets and 20-30 full-sized image slices were extracted per data set. A custom CNN was implemented, adapted from U-Net1. Additional input channels included image intensity gradient magnitude (edge), position and surface geometry obtained from a custom 3D contouring algorithm. The custom contouring algorithm detects in-plane and out-of-plane isosurface directions, forming a cross-hair shape that is followed a predetermined distance (1~3cm) (Fig. 1.). This provides a set of in-plane (2D) and transverse (3D) isocontour segments that cross at the origin point, without adding the third dimension to the CNN architecture. The isocontours are sub-pixel resolved, reducing their dependence on the image resolution.

Results

A new surface feature was implemented (Fig. 1) to detect diffuse hyperintense isocontours in 3D and convert them into a feature vector for machine learning. A new CNN was implemented based on the U-Net framework. The new CNN architecture uses convolution transpose layers to output smooth, full image-sized lesion prediction maps. Preliminary results show promise for the combined imaging and feature vector based CNN. The surface contour sensitively detects edge features, allowing 3D surface characterization of diffuse lesion hyperintensities, and thus facilitating the extraction of the hyperintense component.

Conclusions

We implemented a novel CNN using a newly developed surface feature for sensitive lesion characterization using DL. We improved the performance of DL-based lesion segmentation by extracting a lesion hyperintensity image component from data in a DL framework. This surface-extraction approach may allow a more consistent quantification of the hyperintense volume, rather than volume solids inside discrete boundaries, which could improve the consistency and robustness of MS lesion volume analysis.

Acknowledgements

Sharon G. Lynch is funded by the National Multiple Sclerosis Society, and she has participated in Multi-center MS trials and grants funded by Biogen, Teva, Novatis, Acorda, Opexa, Roche, Genentech, Genzyme, Sun Pharma, Vaccinex, Actelion, NMSS, and NIH.

References

1. Ronneberger O, Fischer S, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI 2015;234-241.