1949
Categorizing liver stiffness in children and adults through deep learning and multiparametric MRI with segmented liver and spleen data1Department of Radiology, Cincinnati children's hospital medical center, Cincinnati, OH, United States, 2Department of Radiology, Cincinnati children's hospital medical center, 45429, OH, United States, 3Department of Radiology, University of Wisconsin-Madison, Madison, WI, United States, 4University of Wisconsin-Madison, Madison, WI, United States, 5Michigan Medicine, University of Michigan, Ann Arbor, MI, United States, 6New York University Langone Health, New York, NY, United States, 7Cincinnati children's hospital medical center, Cincinnati, OH, United States
Synopsis
Keywords: Diagnosis/Prediction, Machine Learning/Artificial Intelligence, Liver, Elastography
Motivation: To address the limited accessibility of Magnetic Resonance Elastography (MRE) for liver stiffness assessment.
Goal(s): To develop AI-based pipeline for categorizing subjects into no/mild (<3 kPa) and moderate/severe (≥3 kPa) liver stiffening using multiparametric MRI images.
Approach: Our model contains two main components: segmentation and classification. We employed a Swin-UNETR model to segment liver and spleen tissues from multiparametric MRI images. Then, we developed a Swin Transformer-based model for liver stiffness stratification. We used multi-site ten-fold cross-validation to evaluate our models’ performance.
Results: Our best model achieved an Area Under the Receiver Operating Characteristic (AUROC) curve of 0.84 for liver stiffness categorization.
Impact: Offering an accessible and accurate method for liver stiffness categorization, our research may enhance patient care, reduce healthcare costs, and expand the availability of this vital diagnostic tool, benefiting clinicians, researchers, and, ultimately, patients with liver disease, worldwide.
Chronic liver disease (CLD) remains a leading cause of morbidity and mortality in both children and adults in both United States and worldwide despite advances in management and treatment [1, 2]. CLD is commonly progressive with resultant liver fibrosis [3]. Early diagnosis and treatment may arrest or even reverse liver injury and fibrosis, thereby slowing the progression of CLD. Magnetic resonance elastography (MRE) is being increasingly used to establish the presence and severity of liver fibrosis. MRE assesses tissue stiffness by utilizing a vibrating passive driver placed on the right upper quadrant of the abdomen to generate shear waves in the liver [1, 4]. Despite the potential benefits of MRE in reducing the need for liver biopsy in some patients, it has associated drawbacks, including increased scan time, patient discomfort, and additional healthcare costs related to necessary hardware/software, image post-processing, and potentially image acquisition. In this current study, we aimed to develop and validate DL models for stratifying liver stiffness based on routinely acquired clinical noncontrast T1w and T2w MR images from a large multi-site, multi-vendor study sample of children and adults with known or suspected chronic liver disease.
Methods
Study cohort
In this HIPAA-compliant, IRB-approved, multi-site retrospective study, patients with known or suspected CLD who underwent clinical abdominal MRI examinations with MRE assessment of liver stiffness between 2011 and 2022 were identified from four institutions, including Cincinnati Children's Hospital Medical Center (CCHMC), New York University (NYU), University of Michigan (UM), and the University of Wisconsin (UW). For each institution, we extracted axial T1w and axial T2w MR images. For each patient, we obtained liver stiffness measurements using MRE data from electronic health records, serving as the reference standard for categorizing liver stiffness. We categorized patients into two groups: those with no/mild liver stiffening [<3 kPa] and those with moderate/severe liver stiffening [≥3 kPa]. Details regarding the demographics and liver stiffness characteristics of our patient population are presented in Table 1, categorized by site.
Deep learning model
Our deep learning model comprises an input imaging layer and two models, encompassing automated segmentation and classification. Firstly, the input imaging layer has two distinct input channels for T1w and T2w images, as illustrated in Figure 1. This input consists of multiple 2D images comprising axial T1w and T2w MR images. Secondly, we developed a Swin-UNETR [5], which involves utilizing a Swin Transformer as the encoder within the U-shaped network (U-Net) structure. We employ the Swin-UNETR model to segment the liver and spleen from T1w and T2w images. Later, we selected the 11 middle slices from each image and resized them to dimensions of 224x224.
Thirdly, we designed a transfer learning block utilizing the pre-trained Swin Transformer [6], originally trained on approximately 14.2 million natural images. Additionally, we integrated an adaptive learning block featuring trainable layers for capturing the individual latent features from the 11 segmented liver and spleen images of each subject. The adaptive learning block includes a fusion layer, four dense layers, and a two-way SoftMax classifier for categorizing the severity of liver stiffening.
We evaluated model performance using accuracy, sensitivity, specificity, and AUROC in multi-site 10-fold cross-validation (CV) experiments.
Results
In a multi-site ten-fold cross-validation, our proposed deep learning model demonstrated the ability to categorically classify the severity of liver stiffening using combined segmented liver and spleen from T1w and T2w MR images, achieving a mean accuracy of 76.8% (95% CI: 75.6, 78.0%), specificity of 79.2% (95% CI: 77.7, 80.7%), sensitivity of 73.3% (95% CI: 71.4, 75.4%), and an AUROC of 0.84 (95% CI: 0.83, 0.85). However, the model's performance slightly decreased when we employed unsegmented T1w and T2w images, resulting in a mean accuracy of 75.2% (95% CI: 74.0, 76.5%), specificity of 76.8% (95% CI: 75.2, 78.3%), sensitivity of 72.9% (95% CI: 70.9, 74.9%), and an AUROC of 0.83 (95% CI: 0.82, 0.84). Additional results using individual pulse sequences, including segmented and unsegmented T1w or T2w images alone, are detailed in Table 2.
Conclusions
Our deep learning models’ performance demonstrate a notable advantage when utilizing segmented images compared to unsegmented images. The incorporation of segmented liver and spleen data enhances models’ ability to accurately classify liver stiffness. Further improvements in performance likely can be achieved by incorporating additional pulse sequences as well as clinical data, and ultimately someday may lead to fewer MRE imaging and/or percutaneous liver biopsy procedures.
Acknowledgements
This work was supported in part by NIH, United States (R01-EB030582, R01-EB029944) and Academic and Research Committee Awards of Cincinnati Children’s Hospital Medical Center.References
[1] Li, H., He, L., Dudley, J. A., Maloney, T. C., Somasundaram, E., Brady, S. L., ... & Dillman, J. R. (2021). DeepLiverNet: a deep transfer learning model for classifying liver stiffness using clinical and T2-weighted magnetic resonance imaging data in children and young adults. Pediatric radiology, 51(3), 392-402.
[2] Lee, Jeong Hyun, et al. "Deep learning with ultrasonography: automated classification of liver fibrosis using a deep convolutional neural network." European radiology 30.2 (2020): 1264-1273.
[3] Trout, A. T., Anupindi, S. A., Gee, M. S., Khanna, G., Xanthakos, S. A., Serai, S. D., & Dillman, J. R. (2020). Normal Liver Stiffness Measured with MR Elastography in Children. Radiology, 297(3), 663-669.
[4] Pollack, B. L., Batmanghelich, K., Cai, S. S., Gordon, E., Wallace, S., Catania, R., & Borhani, A. A. (2021). Deep learning prediction of voxel-level liver stiffness in patients with nonalcoholic fatty liver disease. Radiology: Artificial Intelligence, 3(6).
[5] Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H. R., & Xu, D. (2021, September). Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. In International MICCAI Brainlesion Workshop (pp. 272-284). Cham: Springer International Publishing.
[6] Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., & Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 10012-10022).
Figures
