Introduction

Multiple sclerosis (MS) is an inflammatory disease of the central nervous system, characterized by lesions in the brain and spinal cord [1]. A particular type of MS lesion, called a chronic active lesion, is characterized by an iron-enriched rim of activated macrophages and microglia in histopathology studies [2,3], the presence of which is associated with a more severe disease course [4,5]. Precise identification of these lesions is important for clinical translation, such as using rim+ lesions as an imaging biomarker.
Chronic active lesions (rim+) lesions are typically identified on T2-weighted-Fluid-Attenuated Inversion Recovery (T2-FLAIR) images and then are determined to be rim+ through visual inspection on QSM. However, this process is time-consuming and prone to inter- and intra- rater variability. We propose an automated method, called QSMRim-Net, to identify rim+ lesions using QSM [6] and T2-FLAIR images of the brain (see Fig. 1 for an image example). Our method is a two-branch neural network architecture that fuses QSM and T2-FLAIR imaging features derived from a recently developed residual network [7] with lesion-level radiomic features from the QSM. This is the first method proposed in the literature to identify rim+ lesions using QSM and the first method to fuse convolutional imaging features with radiomic features. We will compare QSMRim-Net with two other recent methods [12,13].

Materials and Methods

Dataset: QSMRim-Net was evaluated on an MS imaging dataset collected at Weill Cornell, consisting of 172 MS patients enrolled in an ongoing prospective database for MS research. Standard whole-brain FLAIR (1 mm isotropic) and QSM () images were acquired on a 3T Magnetom Skyra scanner (Siemens Medical Solutions, Malvern, PA, USA). Rim+ and rim- (MS lesions that are not rim+) lesions were annotated by two human experts, followed the consensus of a third reviewer. After the rim lesion annotation, 177 lesions were identified as rim+ lesions and 3,986 lesions were identified as rim- lesions.
Network Architecture: QSMRim-Net is a two-branch network consisting of three parts: a convolutional network for image feature extraction, a fully connected network for radiomic feature extraction, and a final classifier that outputs the probability that a lesion is rim+ (see Fig. 2). For image feature extraction, we use a deep residual network [7] with 18 layers as our backbone network. We modified the convolutional kernels from 2D to 3D, use two input channels to accommodate the QSM and T2-FLAIR, and use two categories (rim+ and rim-) for the last linear layer. For radiomic feature extraction, radiomic features were calculated on the QSM (described in detail in the section below). The fully connected network for radiomic feature extraction consists of two fully connected layers. The first layer is a linear layer followed by a one-dimensional batch normalization [8], a Swish activation function [9] and a dropout layer. The second layer has the same structure as the first layer, but does not include the dropout layer. To fuse the convolutional and radiomic features, we performed element-wise addition for feature vectors from both the output of the residual network and the fully connected network and processed the new feature vector with another fully connected layer (see classifier in Fig. 2).

Radiomic Feature Analysis: Radiomic features have been shown to be effective in many applications of medical image analysis[ES1] . For QSMRim-Net, radiomic features were calculated over each lesion using the RIA R package [10,11]. Specifically, we calculated five different types of radiomic features: 1) conventional quantitative metrics such as mean intensity and lesion volume, 2) first-order statistics such as harmonic mean and geometric mean, 3) gray-level co-occurrence matrix (GLCM) statistics such as interquartile range and energy sum, 4) gray-level run-length matrix (GLRLM) statistics such as run percentage, and 5) geometric-based parameters such as ratio of lesion surface to volume. In total, 255 radiomic features were calculated over each lesion on the QSM for our model. [ES1]Cite: (Coroller et al., 2016, Liu et al., 2016, Bakas et al., 2017, Sweeney et al., 2021).

Evaluations: Two automated methods, RimNet [12] and APRL [13], have been developed to identify chronic active lesions on T2* phase imaging. Therefore, for use with our data, we adapted these methods to a QSM implementation. More specifically, we implemented three methods for performance comparison: 1) APRL (RF): original APRL method using radiomic features and a random forest; 2) APRL (NN): APRL (RF) method with the random forest replaced by a neural network; 3) RimNet: a multi-model VGG-Net using image features. We used receiver operating characteristic (ROC) curves, precision-recall (PR) curves, F1 score, accuracy, sensitivity, specificity, and PPV as metrics for performance evaluation.

Results

Fig. 4 shows the lesion-wise performance metrics of the proposed QSMRim-Net and the other methods. QSMRim-Net outperformed the competitors in all metrics used for evaluation. With a similar overall accuracy and specificity with other methods, QSMRim-Net resulted in a 10.3% and 9.6% improvement in F1 score, 8.6% and 7.6% improvement in sensitivity and 11.7% and 11.3% improvement in PPV compared to Rim-Net and APRL , respectively. Fig. 3 shows the ROC curves and the PR curves for different methods. The proposed QSMRim-Net has a higher AUC for both the ROC and PR curves than the other methods, indicating improved rim+ lesion identification performance.

Acknowledgements

No acknowledgement found.