0868
Deep Learning Based Multi-Scale Approach for Precision Medicine and Quantitative Imaging in Glioblastoma1Radiology, Harvard Medical School, Boston Children's Hospital, Boston, MA, United States, 2James Cook University, James Cook University, Townsville, Australia, 3Department of Data Science, Sejong University, Seoul, Korea, Republic of, 4Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, Hong Kong, 5School of Physics, Mathematics and Computing, The University of Western Australia, Perth, Australia
Synopsis
Keywords: Diagnosis/Prediction, PET/MR, Glioblastoma, WSI
Motivation: Glioblastoma (GBM) is a fast-growing invasive brain tumor that presents unique treatment challenges. Early diagnosis requires manual segmentation using MRI and histopathological image analysis.
Goal(s): Our proposed model can facilitate medical personnel in an efficient and accurate diagnosis of glioblastoma.
Approach: We present a multiscale multilevel approach based on deep learning for precision medicine and quantitative imaging in GBM capturing image feature and providing wide-ranging contextual information.
Results: Our method predicted the overall survival of GMB patients with an average accuracy of 88.63% and 91.7% DSC (Unet: 84% DSC; Swin Transformer: 87% DSC) on BraTS 2020.
Impact: Our model surpasses state-of-the-art methods in Glioblastoma (GBM) segmentation and predicts patient survival with 88.63% accuracy. This research work assists in precise and efficient diagnoses of GBM, potentially contributing to early disease detection and treatment strategies.
Introduction
Glioblastoma (GBM) is one of the common malignant brain tumours with a poor prognosis. According to the World Health Organisation (WHO) grading system, tumours of grade III-IV are considered malignant and difficult to treat. Therefore, GBMs are called IV tumours and are highly invasive and through the brain parenchyma along the blood vessels and white matter, gradually progressing to the meninges. Early detection of malignant tumours is important for effective treatment and a better prognosis. In this research work, we propose a model that captures both local and global features in input images(PET/MRI images and histopathology images). We designed an automated method to provide automatic segmentation, as well as classification in Astrocytoma (A), Oligodendroglioma (O), and Glioblastoma Multiforme(G) based on the features from the different modules and predict the overall survival (OS).Proposed Method
Our proposed model has four modules, namely: i) the WSI module, and ii) the MRI/PET module. For the WSI-based approach, 300 patches were sampled from each case of WSI and were fed into the proposed pipeline for patch-level classification in our experiments. Subsequently, the glioma subtype for each case can be determined by choosing the subtype with the highest votes, and the vote distributions are normalized to the probabilities as confidence scores for each glioma subtype. Our WSI Module framework is based on a deep learning model. We combined CNN and superpixel [1] into our feature Extraction Branch for acquiring local-level features from input histopathology images. This branch follows the Prototype Selection architecture for the reclassification. Each prototype is classified into either of the four classes; A, G, O, and I. The feature extraction and prototype selection combined with the Majority Voting Patch Level (MVPL) method [2] captures all dependencies on patch-level in image feature learning and provides wide-ranging contextual information. On the other hand, for the 18F-FET PET/MRI module, we trained PET and ground truth using an encoder (VGG16)-Decoder model for MRI, a U-Net-based deep 3D multi-level CNN is trained on the dataset from the challenge of BraTS2020 and the CPM-RadPath 2020 [3]. We used the in-house dataset to pre-train our model. The fine-tuning process requires splitting fine-tuning datasets into training sets, validation sets, and test sets. We did an 80% and 20% train/test split and applied 13-fold cross-validation. For some datasets such as BraTS 2020, CPM-RadPath, and EGD were already distributed into training, validation, and testing data sets, no further changes were made to these sets.Results & Discussion
Our proposed model achieved an average DSC of 0.891 with 0.894 precision. For each benchmark dataset, we assessed the performance of our proposed model in terms of DSC, IoU, F1-score, Precision, and Recall in Table 2. Ourproposed model acquired 96.3% average DSC for BraTS2020, 92.34% for TCGA-GBM, Glioblastoma Multiforme (TCGA, PanCancer Atlas), Personalised OncoGenomics(POG) study, and 96.72% for UK Biobank. Furthermore, our model outperformed Unet and SwinTransformer on our in-house dataset by achieving 89% DSC, while Unet achieved 76% DSC and Swin Transformer achieved 81% DSC. Our model evaluation results outperform Unet and the Swin Transformer, revealing the potential of our model for medical imaging.Conclusion
In this paper, we present a deep learning-based Multi-Scale Multi-Level approach for precision medicine and quantitative imaging in Glioblastoma (GBM). Our proposed model captures image feature learning and provides wide-ranging contextual information using multiple modules including an MRI module, PET module, and WSI module for segmentation and overall survival prediction. Our model results have outperformed the state-of-the-artmethods – Unet and Swin Transformer, in GBM segmentation on BraTS 2020 and our in-house dataset. With appropriate pre-training, our proposed model can be optimized for any type of medical image segmentation. This model is designed to facilitate medical personnel in diagnosing glioblastoma; therefore, we constantly received feedback from the Department of Neurology at Royal Prince Alfred Hospital. This model is to be used as a“second reader” and therefore requires a physician for final approval. Precision treatment of GBM can slow tumor growth and help improve patient prognosis. Based on TCIA databases combined with a radiomics approach and PET/MRI-based segmentation, this study confirms that the proposed GBM multiscale approach exhibited higher performance in GBM patient data compared to using either module individually. Based on the above-optimized model, a personalized treatment recommendation system for GBM can be developed to accurately predict patient prognosis.Acknowledgements
We would like to acknowledge the publicly available dataset providers for BraTS 2020, CPM-RadPath, UCSF PDGM, EGD, and IvyGAP. We are also grateful to the clinicians at the Department of Neurology at Royal Prince Alfred Hospital.References
[1] Ziheng Wang, Xiongkuo Min, Fangyu Shi, RuinianJin, Saida S Nawrin, Ichen Yu, and Ryoichi Nagatomi.Smeswin unet: Merging cnn and transformer for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 517–526. Springer, 2022.
[2] Olivier Petit, Nicolas Thome, Clement Rambour, LoicThemyr, Toby Collins, and Luc Soler. U-net transformer:Self and cross attention for medical image segmentation.In International Workshop on Machine Learning in Medical Imaging, pages 267–276. Springer, 2021.
[3] Tahsin Kurc, Spyridon Bakas, Xuhua Ren, Aditya Bagari, Alexandre Momeni, Yue Huang, Lichi Zhang,Ashish Kumar, Marc Thibault, Qi Qi, et al. Segmentation and classification in digital pathology for glioma research:challenges and deep learning approaches. Frontiersin neuroscience, 14:27, 2020.
Figures
