4799

Hypothalamus semi-automatic segmentation from MR images using Convolutional Neural Networks

Lívia Rodrigues¹, Thiago Rezende², Ariane Zanesco², Ana Luiza Hernandez², Marcondes França², and Letícia Rittner¹

¹Medical Image Computing Lab, School of Electrical and Computer Engineering (FEEC), University of Campinas, Campinas, Brazil, ²Department of Neurology, School of Medical Sciences, University of Campinas, Campinas, Brazil

Synopsis

Hypothalamus is a small structure of the brain with important role in sleep, body temperature regulation and emotion. Some diseases as schizophrenia can be attributed to volumetric change on hypothalamus, usually measured through Magnetic Resonance Imaging (MRI). However, hypothalamic morphological landmarks are not always clear and manual segmentation can become variable, leading to inconsistent data on literature. On this project, hypothalamus was automatically segmented using convolutional neural networks (CNNs) . Three independent CNNs were trained, one for each view of volumetric MRI, obtaining final dice of 0.787 for axial view, 0.781 for sagittal and 0.747 for coronal view.

INTRODUCTION

Hypothalamus is a gray matter structure located bellow the thalamus and is a part of the limbic system. It presents an important role in sleep, body temperature regulation, appetite and emotion¹. There are many studies in the literature linking altered hypothalamus volume to some diseases, such as schizophrenia ^1,2, Behavioral-Variant Frontotemporal Dementia ³, mood disorders ¹ and so on. Despite many studies in the literature use hypothalamus segmentation, it is still done manually and susceptible to human mistakes and different morphology approaches. For being a small region and hard to be clearly visualized in MRI images (Fig 1), it is difficult to determine its morphological landmarks ⁴. Manual segmentation procedures vary from one author to another, making it harder to compare volumetric results. For instance, Goldstein et al.² reported increased volumetric founds in schizophrenic patients, while Komp et al. ⁵ reported preserved volumes. Due high variability and inconsistency on hypothalamic manual segmentation, it is desirable to have an automatic or semiautomatic approach, with low human interaction. In this work we trained convolutional neural networks (CNNs), more specifically, the U-net architecture(*6) to segment the human hypothalamus in T1-weighted MRI images.

MATERIALS AND METHODS:

For this project, we used 177 T1-weighted MR images of the brain, 240x240x180 pixels, acquired on a 3T Phillips Achieva scanner. Patches of 60x60 were extracted around the hypothalamus, using an user provided seed. Patches which had less than 35 pixels belonging to the hypothalamus were then discarded, in order to reduce the imbalance of classes (pixels from hypothalamus versus pixels from background). The extracted patches were normalized by the maximum gray level value of the MR image and standardized as zero mean and unit variance. In order to increase the variability of the model, data augmentation was performed through rotations of 30 degrees and translations of 10 pixels, creating four times the amount of data we had at the beginning. Finally, we used a tensorflow-based implementation of a U-net ⁷ and binarized the output of the network, a grayscale image with intensity varying from 0 to 1, representing the probability of each pixel to be or not a part of hypothalamus. The final threshold used for binarization is better explained on Tab.1.

RESULTS AND DISCUSSION:

We divided the initial MRI dataset into training set (80%) and test set (20%), before patch extraction. We compared the performance of our method applied independently to each one of the three views: axial, sagittal and coronal (Fig.2 and Tab.2). As ground truth, we used manual segmentation of hypothalamus performed by specialists. We could not find any automatic or semi automatic method for hypothalamus segmentation on the literature, reason why we can not compare our method directly with any results. A possible comparison is with some works that can be found on small structures of the brain. For instance, Aljabar et al. ⁸ segmented the accumbens (also a small structure) via atlas, and found a dice coefficient of 0.751. Analyzing all three different CNNs trained, we notice that axial view images were easier to segment (dice coefficient = 0.787). Axial view had the worst precision result, showing that it probably marked more background pixels as hypothalamus than the other two CNNs. On the other hand, it returned the highest recall, meaning that this CNN was more assertive on finding pixels belonging hypothalamus. This result was expected due the symmetry presented on hypothalamus when viewed through this position. Despite coronal view also presents symmetry, its morphological landmarks are even harder to distinguish. A possible reason why this network got the worst recall and dice and best precision is the fact that this CNN predicted more positive pixels (ie, pixels that originally belong to hypothalamus) as background than the other two CNNs. Threshold set up is also a critical part of the project. Even if our CNN returns a good prediction, a bad tuning of this parameter yields to a non-satisfactory binary mask.

CONCLUSION:

This work presented a semi-automatic method for hypothalamus segmentation using CNNs. We can not perform a direct comparison with other hypothalamus segmentations, since to the best of our knowledge, there are only manual methods described on the literature. In future works, results can be improved by creating a consensus voting between all three networks and a small network to find the best threshold for the binarizarion step. Also, the method can be made fully automatic by finding the patches without human interaction

Acknowledgements

(FAPESP - process CEPID 2013/07559-3) and the Brazilian National Council for Scientific and Technological Development (CNPq – process 308311/2016-7)

References

1. Gabery, S., Georgiou-Karistianis, N., Lundh, S.H. et al. Volumetric analysis of the hypothalamus in huntington disease using 3t mri: The image-hd study. PLoS ONE (2015)

2. Goldstein, J.M., Seidman, L.J., Makris, N., et al. Hypothalamic abnormalities inschizophrenia: sex effects and genetic vulnerability. Biological psychiatry. 2007; 61:935-945

3. Piguet, O, Peterse, A, Yin Ka Lam, B, et al. Eating and hypothalamus changes in behavioral-variant frontotemporal dementia. ANN NEUROL. 2011; 69: 312-319

4. Tognin, S., Rambaldelli, G., Perlini, C., et al. Enlarged hypothalamic volumes in schizophrenia. Neuroimaging. 2012; 204(2-3):75-81

5. Klomp, A., Koolschijn, P.C., Pol, H., et al. Hypothalamus and pituitary volume in schizophrenia: a structural MRI study. The international journal of neuropsychopharmacology. 2017; 15(2):281-288

6. Ronneberger, O., Fischer, P., Brox, T. U-net: Convolutional networks for biomedical image segmentation. MICCAI. 2015; 9351: 234–241

7. Akeret, J., Chang, C., Lucchi, A., et al. Radio frequency interference mitigation using deep convolutional neural networks. Astronomy and Computing. 2017;18: 35-39

8. Aljabar, P., Heckemann, R.A., Hammers, A., et al. Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy. Neuroimaging. 2009; 46(3):726-738

Figures

Figure 1: From left to right: Axial, Sagittal and Coronal view of a brain T1-weighted MR Image. Highlighted in red, we can see the small region that represents the hypothalamus. We can see that it is a symmetric structure when analyzed from axial or coronal view, despite its morphological landmarks are more clear on axial and sagittal view for manual segmentation.

Figure 2: Examples of qualitative results for final segmentation. The first row correspond to axial results, the second to sagittal and the last row to coronal results. First column contains the test image (patch), second column contains the manual segmentation, third column shows the network output (grayscale). Fourth column shows the final binary mask, after applying a threshold and last column shows the difference between predicted and manual mask. We can see that the output of U-Net is a grayscale image and the threshold applied on it will have a great influence on the final binary result.

Table 1: Here we can see the influence of threshold parameter on the final segmentation results. The dice coefficients reported here are for validation set on axial view. As we can see, the threshold setup is a crucial step of the project and can directly influence the final results. p represents the gray level intensity of the U-net output, on other words, the probability of each pixel be or not part of hypothalamus. The midrange of p can be defined as (max(p) +min(p))/2

Table 2: Final dice, precision and recall for each CNN trained for axial, sagittal and coronal views. Despite all three presented similar results, axial view returned a slightly better dice coefficient and recall, meaning that this view is easily trained.

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)

4799