Synopsis

Keywords: Machine Learning/Artificial Intelligence, Multiple Sclerosis

The decision process of artificial intelligence is elusive. We proposed a new method that by combining an attention-based convolutional neural network (GAMER-MRI) with the modified Layer-wise Relevance Propagation could reveal relevant regions on quantitative imaging maps in differentiating multiple sclerosis patients with mild-moderate and severe disabilities. The assessment of the relevant regions included the impact of inverting values within the regions and the heatmap on the MNI152 template. Our results show good network performance and identify brain regions relevant to the corticospinal tract. The proposed method might be useful to further explore patterns of brain microstructural alterations associated with disability.

Introduction

Quantitative MRI provides microstructural measures of brain tissue property relating to its main components (e.g. myelin – myelin water fraction (MWF) and quantitative T1 (qT1); axon, qT1 and Neurite density Index (NDI))¹. Those quantitative MR measures (qMRs) are especially suitable to investigate multiple sclerosis (MS) pathology because this latter typically affects the integrity of myelin and axons². Using qMRs, we successfully developed an attention-based convolutional neural network – GAMER-MRI – to obtain attention weights as quantitative proxies for the importance of qMRs in MS applications^3,4. However, this approach suffered a common issue of deep neural networks that the deep layer structure and nonlinear activations hinder the understanding of the decision process5. The Layer-wise Relevance Propagation (LRP)⁵ alleviates this issue by using various rules to redistribute the output based on the contributions of neurons in the network, hereby providing with relevance maps. To find out which brain regions on qMR maps considered by GAMER-MRI determine the classification of MS patients with mild vs severe disability, we proposed a new approach combining GAMER-MRI and modified LRP.

Methods

We enrolled 166 MS patients (100 relapsing-remitting and 66 progressive, 99 females, age range=45.9±14.3). Their disability was measured using the Expanded Disability Status Scale (EDSS, median=2.5). Patients underwent MRI on a 3T whole-body MR system (Siemens MAGNETOM Prisma). Sixty-nine of the patients underwent 2-year follow-up MRI acquisitions. MRI data were reconstructed into three qMRs³: MWF of 0.94x0.94x5 mm³, NDI of 1.8x1.8x1.8 mm³, and qT1 of 1x1x1 mm³. Non-brain tissue was removed by FreeSurfer⁶. qMRs were scaled between 0 and 1 according to the feasible value ranges within brain: MWF ≤30% and qT1 ≤2500 ms. qMRs were co-registered to the NDI reference space using FSL⁷. Forty patients with EDSS ≥5 (severe motor impairment) were assigned to the group with severe disability. The rest of the patients formed the mild-moderate disability group. An independent test dataset (16 in the mild disability group and 5 in the severe group) and a dataset for the stratified three-fold cross-validation were used considering the size of the severe group. GAMER-MRI included feature extraction, gated-attention mechanism, and classification (Fig. 1). The feature extraction was based on DenseNet⁸ with 16 filters at the first 3D convolutional layer, four dense blocks, two dense layers per dense block, and the growth rate equal to four. The hidden feature vectors from the feature extraction had 32 elements. The gated-attention mechanism consisted of the gate and signal branches, where the numbers of neurons in the layers were 16. Patient age was divided by 100 and concatenated to the combined hidden feature vector after the gated-attention mechanism. The classifier was a one-neuron fully connected layer. The binary cross entropy loss was weighted by $$$weight=2-\frac{|EDSS-5|}{5}$$$ so that the heterogeneity within groups could be better reflected during training. The evaluation metric was the area under the receiver operating characteristic curve (AUC). To alleviate overfitting, data augmentation included random flipping, random 90-degree rotation, random Gaussian noise of zero mean and standard deviation equal to 0.1, and random affine transformation with maximum rotation ±30 degrees and ±20% scaling. In addition, AdamW9 (a regularized optimizer) with a learning rate equal to 0.00005 was used. The weighted sampler was implemented to tackle the group imbalance. The modified LRP included redistribution based on attention weights, absolute contributions of the signal and gate branches (Fig. 2), and the combination of the relevance maps and the qMR maps. The relevant regions at different thresholds for the relevance values were obtained. To assess if relevant regions were relevant, the qMR values within the regions were inverted ($$$qMR=1-qMR$$$) and the subsequent reduction of the AUC was the measure of the importance of the regions. Regions obtained by the original LRP and other explainability methods (Saliency¹⁰ and Integrated Gradients¹¹) underwent the same procedure for comparison. To explore the potential group effect, the relevant regions of patients were transformed to the MNI152 template¹²

Results and Discussion

In Table 1, we report the average performance on the cross-validation folds and the test dataset. The performance indicated GAMER-MRI learned useful representation for further interpretation. According to the obtained attention weights, qT1 was the measure that best discriminated clinical severity in our cohort of MS patients, followed by NDI. This might be due to qT1 providing a comprehensive representation of the damaged microenvironment or more details revealed by the higher resolution and white-grey matter contrast. The reduction of the AUC in Fig. 3 shows that the proposed method revealed the most relevant regions. This resonates with our previous findings that the attention weights are importance proxies of qMRs. Identified regions on the MNI152 template in Fig. 4 include the left caudate, the left thalamus, the putamens, and the internal capsules. Those are structures that are either part of the corticospinal tract or regions involved in motor circuits, which are strongly related to EDSS and disability¹³.

Conclusions

In summary, our work showed that the proposed combined approach could classify patients having a severe motor impairment and identify relevant regions on qMR maps. Future work will aim at (i) investigating the pathological meaning of the relevant regions; (ii) integrating other qMRs such as quantitative susceptibility mapping and magnetization transfer saturation.

Acknowledgements

This project is supported by Swiss National Funds PZ00P3_154508, PZ00P3_131914 and PP00P3_176984 and we thank all the patients for their participation.