Mohammed A. Al-masni1, Woo-Ram Kim2, Eung Yeop Kim3, Young Noh4, and Dong-Hyun Kim1
1Department of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seoul, Korea, Republic of, 2Neuroscience Research Institute, Gachon University, Incheon, Korea, Republic of, 3Department of Radiology, Gachon University College of Medicine, Gachon University, Incheon, Korea, Republic of, 4Department of Neurology, Gachon University College of Medicine, Gachon University, Incheon, Korea, Republic of
Synopsis
Lacunes are small cerebrospinal fluid-filled
lesions that are generated by the occlusion of penetrating deep branches of cerebral
arteries. Early detection of lacunes could decrease the possible clinical
implications such as dementia, gait impairment, and lacunar stroke. In this
study, we propose a deep learning 3D multi-scale residual network for lacunes identification
using FLAIR and T1-MPRAGE MR images. We redesign the proposed network via
applying multiple parallel paths using different input scales. This enables to extract
more robust contextual global features and hence achieve better detection
performance. The proposed work exhibits its ability to distinguish true lacunes
from non-lacunes.
Introduction
Lacunes or Lacunar infarcts are small
lesions that result due to the blockage of penetrating branches of the main
cerebral arteries. Lacunes have round cavity shapes of less than 15 mm in
diameter and located in the basal ganglia or non-cortical white matter 1. They have been considered
as diagnostic biomarkers of various cerebrovascular diseases including
dementia, gait impairment, and lacunar stroke. Magnetic resonance imaging (MRI)
is the preferred scanning tool for lacune screening. More specifically, clinicians
usually examine the presence of lacunes using fluid-attenuated inversion
recovery (FLAIR) and T1-weighted magnetization prepared rapid gradient echo
(T1-MPRAGE). Although these image modalities assist neuroradiologists to
improve the identification of lacunes, manual screening via naked eyes is still
labor-intensive, subjective, and time-consuming. Thus, the development of a computer-aided
detection system for lacune is extremely important through which a second
opinion can be provided to neuroradiologists. Methods
The goal of the proposed work is to
discriminate between suspicious lacune candidates that have been priorly
assigned, leading to provide confidence on the lacune samples by
differentiating between them and lacune mimics.
[Dataset]
A total of 288 subjects were acquired
using two MRI protocols (i.e., FLAIR and T1-MPRAGE), including 696 lacunes. The
ground-truth labeling of lacunar infarcts was performed by expert
neuroradiologists. In the case of lacune patches generation, we captured 3D
multi-scale patches surrounding each labeled lacune at three different scales
of 32×32×5, 48×48×5, and 64×64×5 voxels from both FLAIR
and T1-MPRAGE images. To increase the number of positive samples (i.e., lacune
patches), we augmented every generated 3D patch six times by applying four rotations
and two different shifting operations, providing a total of 4,176 samples. For
the non-lacune samples, we randomly extracted an equal amount of 4,176 negative
samples in order to train the network using a balanced dataset and avoid any
bias towards the majority class.
[Proposed
3D ResNet]
In this work, we implemented a 3D version of the ResNet to discriminate between
the suspicious lacune and non-lacune candidates. The detailed structure of the
proposed 3D ResNet is illustrated in Figure 1. The network consists of two
prior convolutional layers followed by three residual blocks, including two
different bypass connections. Each residual block starts by adding residual
features generated by an additional convolutional layer, while the second
connection is performed by attaching the identity mapping of the preceding
layer to be added to the main path. All the utilized convolutional layers are
with filter sizes of 3×3×3 and feature maps of k1=12, k2=24,
k3=36, and k4=48. The proposed network learns global
contextual representations from six parallel paths (i.e., three different
scales for each imaging modality). Then, we concatenate all the generated
features from these six paths and pass them into two fully connected neural
network (FCNN) layers with 100 and two neurons, respectively.Results
The overall evaluation during the
training process is demonstrated in Figure 2. This figure shows how the
performance accuracy was improved and the loss function was declined throughout
the training epochs for both training and validation sets. The results show the
capability of the proposed network, trained using three various scales of two
different image modalities (i.e., FLAIR and T1-MPRAGE), on the differentiation
between lacune and non-lacune candidates. As an overall evaluation through all five-fold
tests, the 3D ResNet correctly identifies 4,026/4,176 lacunes and 3,797/4,176
non-lacunes, obtaining sensitivity and specificity of 96.41% and 90.92%,
respectively. We also achieve an overall precision of 91.40%, F1-score of 93.84%,
and accuracy of 93.67%. Further, we illustrate the ROC curves and their AUCs of
the proposed work over the five-fold cross-validation in Figure 3. The proposed
work achieves an overall AUC of 93.67% for all fold tests. In addition, Figure
4 presents some lacune and non-lacune exemplars that were truly identified and
detected through the proposed 3D multi-scale residual networks. The drawn
circles on the brain MR FLAIR and T1-MPRAGE images reflect the locations of
suspicious candidates that were tested using the proposed network.Discussion
In this study, the proposed network
has the ability to learn multi-scale global features, which is designed as six
parallel paths (i.e., three different scales for each imaging modality). Then,
the extracted features from all paths are concatenated and passed into the FCNN
classifier. The proposed network can promote the neuroradiologists’ decision
with a second opinion by providing a supportive interpretation of each potential
candidate.
The main limitation of this study is
that the proposed work is a semi-automated technique, which requires minimal
user intervention. A mechanism should exist which first selects the potential
suspicious candidates. After that, the network directly learns multi-scale
features from these candidates in a supervised manner. However, providing an
efficient and fully automated single network to tackle the detection of lacunes
can be a complicated and challenging task. Due to this, some researchers
attempted to solve this issue by providing two-cascaded approaches 2-5, where the first stage was
responsible to screen and detect potential candidates, while the second stage
intended to discriminate the remaining candidates.Conclusion
The proposed work could be feasible
for clinical usage by providing a supportive decision towards lacunar infarcts
detection.Acknowledgements
This research was supported by the
Brain Research Program through the National Research Foundation of Korea (NRF)
funded by the Ministry of Science, ICT & Future Planning
(2018M3C7A1056884). This work was also supported by a grant of the Korea
Healthcare Technology R&D Project through the Korea Health Industry
Development Institute (KHIDI), funded by the Ministry of Health & Welfare,
Republic of Korea (grant No: HI14C1135).References
- Wardlaw,
J. M. et al. Neuroimaging standards
for research into small vessel disease and its contribution to ageing and
neurodegeneration. Lancet Neurol 12, 822-838, doi:Doi
10.1016/S1474-4422(13)70124-8 (2013).
- Ghafoorian, M. et al. Deep multi-scale location-aware 3D convolutional neural
networks for automated detection of lacunes of presumed vascular origin. Neuroimage-Clin 14, 391-399, doi:10.1016/j.nicl.2017.01.033 (2017).
- Dou, Q.
et al. Automatic Detection of Cerebral Microbleeds From MR Images via 3D
Convolutional Neural Networks. IEEE
Transactions on Medical Imaging 35,
1182-1195, doi:10.1109/Tmi.2016.2528129 (2016).
- Al-masni, M. A., Kim, W.-R., Kim, E. Y.,
YoungNoh & Dong-HyunKim. Automated detection of cerebral microbleeds in MR
images: A two-stage deep learning approach. NeuroImage:
Clinical 28, 102464 (2020).
- Liu, S. F.
et al. Cerebral microbleed detection using Susceptibility Weighted Imaging
and deep learning. Neuroimage 198, 271-282,
doi:10.1016/j.neuroimage.2019.05.046 (2019).