4520
Asymmetry-Aware Pretraining: A Path to Universal Brain MRI Analysis Model1School of Computer Science, University of Sydney, Sydney, Australia, 2Brain and Mind Centre, University of Sydney, Sydney, Australia, 3Sydney Neuroimaging Analysis Centre, Sydney, Australia, 4School of Mathematics and Statistics, University of Sydney, Sydney, Australia, 5Department of Neurology, Royal Prince Alfred Hospital, Sydney, Australia
Synopsis
Keywords: Diagnosis/Prediction, Brain
Motivation: Many brain disorders present with asymmetrical features, which can be utilized to advance the classification of disease through deep learning.
Goal(s): We aim to investigate how asymmetry awareness can improve model performance to identify brain disorders, subsequently facilitating better early diagnosis and intervention.
Approach: Our approach involved initial pretraining with a focus on asymmetry awareness using conditional contrastive learning. Subsequently, finetuning was performed for various downstream tasks with limited training data.
Results: Our approach is proven superior in identifying brain disorders compared with baseline methods that either directly trained for specific diseases or with pretrained model without considering symmetrical features of input images.
Impact: Our research pioneers the integration of asymmetry awareness in pretraining models for the detection of brain disorders. The superior performance fully illustrated the feasibility and potential of leveraging the asymmetry nature of brain disease for broad deep learning tasks.
Introduction
Brain disorders often manifest as changes in the brain's symmetry, with both structural and functional alterations observed across various neurological and psychiatric conditions1. Some conditions, such as brain tumors and Alzheimer's disease (AD)2,3, exhibit more pronounced asymmetrical effects as shown in Figure 1 (b, c), with distinct implications for diagnosis and treatment4. This kind of asymmetrical features could be used to enhance the performance of deep learning models to identify brain images with diseases.Lately, SwinUNETR5 and SwinMM6 have substantially advanced the field and demonstrated remarkable performance improvement in learning-based medical image analysis after being pretrained on a substantial volume of medical images. Our research aims to emphasize the critical role of asymmetry awareness for symmetrical anatomical structures, particularly for tasks related to brain disease recognition. To achieve this, we constructed a dataset of brain MRI images that covers both healthy (assumed symmetrical) brains and unhealthy (assumed asymmetrical) brains, and employed pretraining techniques to enhance the model's performance universally, illustrated the impact of asymmetry awareness in advancing the accuracy and efficiency of identifying brain disorders.
Methods
We curated a pretraining dataset comprising 3,509 T1-weighted images of healthy subjects from various publicly available datasets and 2,917 T1-weighted images of patients sourced from the BraTS7 and ADNI8 datasets. Details of the dataset are summarized in Figure 2. As shown in Figure 3, all the data underwent a series of preprocessing steps, followed by resizing to a uniform dimension of [178, 208, 178]. We employed a 4-layer 3D Swin Transformer as the encoder network, utilizing a patch size of 4 for partitioning and embedding the entire brain image as input.In the pretraining process, each input image was augmented with random intensity shift, duplicated and flipped along the mid-sagittal plane. The modified pair of images were then processed through the encoder network to generate vectorized representations. We utilized a conditional contrastive loss function to incorporate symmetry information, which encourages the pairs of healthy images to have closer features and enlarges feature dissimilarity for the pairs created by images with disorders. The process was carried out with a learning rate of 6e-6 for 1,000,000 steps.
Following pretraining, the encoder network is utilized for downstream disease identification tasks, including Focal Epilepsy9, ADHD10, and Schizophrenia11. For each disease, the pretrained encoder network was attached with a randomly initialized MLP layer as the classification head, and the preprocessed images were used to train a separate classification model using a cross-entropy loss. We employed 5-fold cross-validation for all three diseases. Finetuning was conducted with a learning rate of 1e-5 for 10 epochs. For evaluation we compared models trained with or without asymmetry-aware pretraining, distinguishing "Direct Train" from the rest of our approaches. Second, we investigated the impact of incorporating asymmetry awareness, contrasting our proposed method with the "Baseline" model. Finally, we assessed the effectiveness of our asymmetry-aware pretraining, examining the performance of "Zero-Shot" learning without finetuning.
Results
As shown in figure 4, our model, initialized with asymmetry-aware pretrained weights, consistently outperformed counterparts on all three downstream tasks, which showed the importance of considering asymmetry in the pretraining phase. We also evaluated the effectiveness of asymmetry awareness without finetuning. Notably, our pretrained model demonstrated superior zero-shot performance in certain metrics in the Focal Epilepsy and Schizophrenia identification tasks, surpassing "Direct Train" models. This suggests that our pretrained model encapsulates valuable information that generalizes to new tasks, even when no task-specific data is available, which demonstrates the robustness and potential of our approach for improving brain MRI analysis tasks with limited training data.Discussion
Our study highlights the potential of asymmetry-aware pretrained models for symmetrical anatomy analysis such as human brain. We observe significant performance gains in zero-shot settings for Focal Epilepsy and Schizophrenia, both characterized by prominent brain asymmetry. In contrast, ADHD, with subtler brain asymmetry12, shows marginal improvements. The superior performance observed in all the downstream tasks when finetuning with limited data, supports the value of pretraining techniques in harnessing rich asymmetry-related features from a large dataset. The ability to outperform models directly trained on specific tasks demonstrated the importance of universality and transferability in our model's representations.Conclusion
Our research introduced an emerging approach to medical image analysis for symmetrical anatomy using deep learning. The asymmetry-aware pretrained model exhibits a significant advantage in various brain disease analysis and illustrated the potential of pretraining methods in enhancing model universality, particularly benefiting tasks with limited data such as medical imaging.Acknowledgements
The authors would like to express their gratitude to the BCR scholarship for their financial support, which enabled this research and its contributions to medical imaging and healthcare.References
[1] David E Ross, John D Seabaugh, Jan M Seabaugh, Claudia Alvarez, Laura Peyton Ellis, Christopher Powell, Christopher Reese, Leah Cooper, Katherine Shepherd, and for the Alzheimer’s Disease Neuroimaging Initiative. Journey to the other side of the brain: asymmetry in patients with chronic mild or moderate traumatic brain injury. Concussion, 8(1):CNC101, 2022.
[2] Cheng Yang, Suyu Zhong, Xiaolong Zhou, Long Wei, Lijia Wang, and Shengdong Nie. The abnormality of topological asymmetry between hemispheric brain white matter networks in alzheimer’s disease and mild cognitive impairment. Frontiers in aging neuroscience, 9:261, 2017.
[3] Christian Wachinger, David H Salat, Michael Weiner, Martin Reuter, and Alzheimer’s Disease Neuroimaging Initiative. Whole-brain analysis reveals increased neuroanatomical asymmetries in dementia for hippocampus and amygdala. Brain, 139(12):3253–3266, 2016.
[4] EDWARD L MERRIN. Schizophrenia and brain asymmetry: An evaluation of evidence for dominant lobe dysfunction. The Journal of Nervous and Mental Disease, 169(7):405–416, 1981.
[5] Yucheng Tang, Dong Yang, Wenqi Li, Holger R Roth, Bennett Landman, Daguang Xu, Vishwesh Nath, and Ali Hatamizadeh. Self-supervised pre-training of swin transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20730–20740, 2022.
[6] Yiqing Wang, Zihan Li, Jieru Mei, Zihao Wei, Li Liu, Chen Wang, Shengtian Sang, Alan L Yuille, Cihang Xie, and Yuyin Zhou. Swinmm: masked multi-view with swin transformers for 3d medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 486–496. Springer, 2023.
[7] Ujjwal Baid, Satyam Ghodasara, Suyash Mohan, Michel Bilello, Evan Calabrese, Errol Colak, Keyvan Farahani, Jayashree Kalpathy-Cramer, Felipe C Kitamura, Sarthak Pati, et al. The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314, 2021.
[8] Clifford R Jack Jr, Matt A Bernstein, Nick C Fox, Paul Thompson, Gene Alexander, Danielle Harvey, Bret Borowski, Paula J Britson, Jennifer L. Whitwell, Chadwick Ward, et al. The alzheimer’s disease neuroimaging initiative (adni): Mri methods. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine, 27(4):685–691, 2008.
[9] Fabiane Schuch, Lennart Walger, Matthias Schmitz, Bastian David, Tobias Bauer, Antonia Harms, Laura Fischbach, Freya Schulte, Martin Schidlowski, Johannes Reiter, Felix Bitzer, Randi von Wrede, Attila Rácz, Tobias Baumgartner, Valeri Borger, Matthias Schneider, Achim Flender, Albert Becker, Hartmut Vatter, Bernd Weber, Louisa Specht-Riemenschneider, Alexander Radbruch, Rainer Surges, and Theodor Rüber. "An open presurgery mri dataset of people with epilepsy and focal cortical dysplasia type ii", Scientific Data, 10:475, 2023.
[10] James R. Booth, GE Cooke, Jessica Gayda, Rubi Hammer, Marisa N. Lytle, MA Stein, and Michael Tennekoon. "working memory and reward in children with and without attention deficit hyperactivity disorder (adhd)", 2021.
[11] Joan Soler-Vidal, Paola Fuentes-Claramonte, Pilar Salgado-Pineda, Nuria Ramiro, María Ángeles García-León, María Llanos Torres, Antonio Arévalo, Amalia Guerrero-Pedraza, Josep Munuera, Salvador Sarró, Raymond Salvador, Wolfram Hinzen, Peter McKenna, and Edith Pomarol-Clotet. "brain correlates of speech perception in schizophrenia patients with and without auditory hallucinations", 2022.
[12] Ningning He, Lena Palaniyappan, Zeqiang Linli, and Shuixia Guo. Abnormal hemispheric asymmetry of both brain function and structure in attention deficit/hyperactivity disorder: a meta-analysis of individual participant data. Brain Imaging and Behavior, 16(1):54–68, 2022.
[13] Bharat B Biswal, Maarten Mennes, Xi-Nian Zuo, Suril Gohel, Clare Kelly, Steve M Smith, Christian F Beckmann, Jonathan S Adelstein, Randy L Buckner, Stan Colcombe, et al. Toward discovery science of human brain function. Proceedings of the national academy of sciences, 107(10):4734–4739, 2010.
[14] Xi-Nian Zuo, Jeffrey S Anderson, Pierre Bellec, Rasmus M Birn, Bharat B Biswal, Janusch Blautzik, John Breitner, Randy L Buckner, Vince D Calhoun, F Xavier Castellanos, et al. An open science resource for establishing reliability and reproducibility in functional connectomics. Scientific data, 1(1):1–13, 2014.
[15] A Rowland, M Burns, T Hartkens, J Hajnal, D Rueckert, and Derek LG Hill. Information extraction from images (ixi): Image processing workflows using a grid enabled image database. Proceedings of DiDaMIC, 4:55–64, 2004.
[16] Kate Brody Nooner, Stanley J Colcombe, Russell H Tobe, Maarten Mennes, Melissa M Benedict, Alexis L Moreno, Laura J Panek, Shaquanna Brown, Stephen T Zavitz, Qingyang Li, et al. The nki-rockland sample: a model for accelerating the pace of discovery science in psychiatry. Frontiers in neuroscience, 6:152, 2012.
[17] Pamela J LaMontagne, Tammie LS Benzinger, John C Morris, Sarah Keefe, Russ Hornbeck, Chengjie Xiong, Elizabeth Grant, Jason Hassenstab, Krista Moulder, Andrei G Vlassenko, et al. Oasis-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheimer disease. MedRxiv, pages 2019–12, 2019.
Figures
