3235
Association of MR image quality measures with diagnostic accuracy and inter-reader agreement of PI-RADS for detection of prostate cancer1Case Western Reserve University, Cleveland, OH, United States, 2Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States, 3University Hospitals Cleveland Medical Center, Cleveland, OH, United States, 4Cleveland Clinic Foundation, Cleveland, OH, United States, 5Emory University, Atlanta, GA, United States
Synopsis
Keywords: Data Processing, Data Processing, Image processing
Radiologists’ ability to detect clinically-significant prostate cancer on MRI is affected by image quality. While subjective guidelines are provided by the prostate imaging reporting and data system (PI-RADS) and other recent methods, there is a need to develop objective and quantitative metrics of MRI quality. In this multi-reader study, we derive MRI quality metrics using an image processing-based open-source software tool and evaluate their association with radiologist-assigned PI-RADS scores to detect clinically-significant prostate cancer (csPCa). We observe that higher quality MRI is associated with improved detection of csPCa on MRI.
Introduction
Prostate Imaging Reporting and Data System (PI-RADS) guidelines have been established to standardize radiologists' interpretation of prostate cancer (PCa) on multi-parametric MRI (mpMRI)1. PI-RADS scores (range 1-5) reflect the probability of PCa lesions being clinically significant (csPCa) defined as International Society of Urologic Pathology Grade Group (IGG) > 1 on biopsy1,2. PI-RADS is influenced by several factors including variable image quality (Figure 1) limiting its usage for confirmatory non-invasive PCa diagnosis3,4,5. More recently, biparametric MRI (bpMRI) including T2-Weighted (T2W) and Apparent Diffusion Coefficient (ADC) maps derived from Diffusion Weighted MRI (DWI) are being widely explored for PCa diagnosis8. Recent studies have outlined the need to provide radiologists with high quality MRI for accurate, non-invasive diagnosis of PCa and consequently identified the need to standardize assessment of prostate MRI quality6. While PI-RADSv2 introduced minimum technical standards to reduce image quality variance, they were found to be inadequate3. Recently, PI-QUAL guidelines have been proposed to assess diagnostic quality of prostate MRI. However, they are subjective and can still be influenced by reader interpretation7. Quantitative metrics derived from computational image processing can provide an objective means of assessing MRI quality. In this study, we aim to use the open-source tool MRQy to derive quantitative image quality metrics of prostate bpMRI and evaluate their association with (a) accuracy of PI-RADS in detecting csPCa and (b) inter-reader agreement between PI-RADS scores, on a dataset of n=99 patient studies evaluated by n=4 experienced radiologists.Methods
Multi-parametric MRI scans of N=99 patients from the ProstateX Challenge9 dataset were used in this study. IGG estimated from targeted biopsies were used to determine csPCa (IGG > 1). Four experienced radiologists (R1-R4) assigned PI-RADS v2.1 scores to each lesion using mpMRI. They were provided with coordinate locations of positive biopsy core but were blinded to biopsy grade. Imaging characteristics of the bpMRI protocol are provided in Table 1. MRQy is an open-source, quantitative and unsupervised tool that estimates relative differences in MRI quality within a given cohort in terms of 15 computed measurements10. We derived image quality measurements of the T2W and ADC sequences using MRQy and rank-ordered them by their coefficients of variation (Table 2). Metrics that show significant variations across dataset capture inherent variations. The top two metrics with the largest variation were SNR1 and SNR2. SNR1 is a signal-to-noise ratio metric defined as the ratio of foreground standard deviation to that of the background. SNR2 is another signal-to-noise ratio defined as the ratio of mean intensity of the foreground to the standard deviation of the background. Higher SNR measurements represent better image quality. We computed median SNR1 and SNR2 values over the entire dataset individually for T2W and ADC and used them as thresholds to partition the dataset into four groups (GLL, GLH, GHL, GHH), subscripts representing low and high values for T2W and ADC, respectively. Accuracy of PI-RADS within each of the four groups and for each reader (R1-R4) were computed using the area under the receiver operating characteristics curve (AUC). Fleiss’ Kappa (κ) was used to determine inter-reader agreement of PI-RADS within each group.Results
Median thresholds of SNR1 and SNR2 quality metrics for T2W were 10.6, 9.1 and 35.7, while those for ADC were 12.1, 16.9 and 50.0, respectively. Mean AUC of PI-RADS assigned by four radiologists within GLL, GLH, GHL, and GHH groups based on SNR1 were 0.54 ± 0.08, 0.58 ± 0.17, 0.57 ± 0.09, and 0.63 ± 0.11. For SNR2, they were 0.57 ± 0.08, 0.64 ± 0.11, 0.49 ± 0.13, and 0.66 ± 0.14, respectively (Tables 3A). The observed AUC were statistically significant (p<0.05). Fleiss’ Kappa (κ) of PI-RADS scores between the readers for GLL, GLH, GHL, and GHH groups based on SNR1 were 0.31, 0.21, 0.20, and 0.15. For SNR2, they were 0.32, 0.24, 0.20, and 0.12, respectively (Table 3B). We did not observe statistically significant differences in κ between the groups.Discussion
Our study presents the initial result of evaluating the relationship between quantitative image quality measures from prostate MRI and radiologists’ ability to identify clinically significant prostate cancer (csPCa). We observed that radiologists’ detection of csPCa in terms of PI-RADS scores was significantly improved in the group of patients with higher quality metrics for bpMRI. We also observed that higher quality ADC maps were associated with better PI-RADS accuracy compared to T2W images (Figure 2). However, inter-reader agreement of PI-RADS scores within the patient groups clustered by quality were inconclusive and remain to be explored in larger studies and in future work. Our results demonstrate that image quality can impact performance of radiologists in identifying csPCa on MRI.Conclusions
Quantitative metrics of image quality can be used to distinguish between low- and high-quality prostate MRI. Accuracy of radiological assessment of clinically significant prostate cancer in terms of PI-RADS is significantly higher in patients with higher quality MRI, resulting in improved non-invasive prostate cancer diagnosis. Future work involves development of machine learning frameworks for automated estimation of objective and diagnostic quality of prostate MRI.Acknowledgements
Research reported in this publication was supported by the National Cancer Institute under award numbers R01CA249992-01A1, R01CA202752-01A1, R01CA208236-01A1, R01CA216579-01A1, R01CA220581-01A1, R01CA257612-01A1, 1U01CA239055-01, 1U01CA248226-01, 1U54CA254566-01, National Heart, Lung and Blood Institute 1R01HL15127701A1, R01HL15807101A1, National Institute of Biomedical Imaging and Bioengineering 1R43EB028736-01, National Center for Research Resources under award number 1 C06 RR12463-01, VA Merit Review Award IBX004121A from the United States Department of Veterans Affairs Biomedical Laboratory Research and Development Service the Office of the Assistant Secretary of Defense for Health Affairs, through the Breast Cancer Research Program (W81XWH-19-1-0668), the Prostate Cancer Research Program (W81XWH-15-1-0558, W81XWH-20-1-0851, W81XWH-18-1-0524), the Lung Cancer Research Program (W81XWH-18-1-0440, W81XWH-20-1-0595), the Peer Reviewed Cancer Research Program (W81XWH-18-1-0404, W81XWH-21-1-0345, W81XWH-21-1-0160), the Kidney Precision Medicine Project (KPMP) Glue Grant and sponsored research agreements from Bristol Myers-Squibb, Boehringer-Ingelheim, Eli-Lilly and AstraZeneca. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the U.S. Department of Veterans Affairs, the Department of Defense, or the United States Government.References
[1] Barentsz JO, et al. Synopsis of the PI-RADS v2 Guidelines for Multiparametric Prostate Magnetic Resonance Imaging and Recommendations for Use. Eur Urol. 2016 Jan;69(1):41-9. PMID: 26361169; PMCID: PMC6364687.
[2] Seo JW, et al. PI-RADS Version 2: Detection of Clinically Significant Cancer in Patients With Biopsy Gleason Score 6 Prostate Cancer. AJR Am J Roentgenol. 2017 Jul;209(1):W1-W9. PMID: 28418690.
[3] Sackett J, et al. Quality of Prostate MRI: Is the PI-RADS Standard Sufficient? Acad Radiol. 2021 Feb;28(2):199-207. PMID: 32143993; PMCID: PMC8459209.
[4] Karanasios E, et al. Prostate MRI quality: clinical impact of the PI-QUAL score in prostate cancer diagnostic work-up. Br J Radiol. 2022 May 1;95(1133):20211372. Feb 18. PMID: 35179971.
[5] Greer MD, et al. Interreader Variability of Prostate Imaging Reporting and Data System Version 2 in Detecting and Assessing Prostate Cancer Lesions at Prostate MRI. AJR Am J Roentgenol. 2019 Mar 27:1-8. PMID: 30917023; PMCID: PMC8268760.
[6] Maarten de Rooij, Bas Israël, Tristan Barrett, Francesco Giganti, Anwar R. Padhani, Valeria Panebianco, Jonathan Richenberg, Georg Salomon, Ivo G. Schoots, Geert Villeirs, Jochen Walz, Jelle O. Barentsz, Focus on the Quality of Prostate Multiparametric Magnetic Resonance Imaging: Synopsis of the ESUR/ESUI Recommendations on Quality Assessment and Interpretation of Images and Radiologists’ Training, European Urology, Volume 78, Issue 4, 2020, Pages 483-485, ISSN 0302-2838.
[7] Giganti F, Allen C, Emberton M, Moore CM, Kasivisvanathan V; PRECISION study group. Prostate Imaging Quality (PI-QUAL): A New Quality Control Scoring System for Multiparametric Magnetic Resonance Imaging of the Prostate from the PRECISION trial. Eur Urol Oncol. 2020 Oct;3(5):615-619. doi: 10.1016/j.euo.2020.06.007. Epub 2020 Jul 6. PMID: 32646850.
[8] Thestrup KC, Logager V, Baslev I, Møller JM, Hansen RH, Thomsen HS. Biparametric versus multiparametric MRI in the diagnosis of prostate cancer. Acta Radiol Open. 2016 Aug 17;5(8):2058460116663046. doi: 10.1177/2058460116663046. Erratum in: Acta Radiol Open. 2017 Mar 14;6(3):2058460117699757. PMID: 27583170; PMCID: PMC4990814.
[9] Armato SG 3rd, et al. PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images. J Med Imaging (Bellingham). 2018 Oct;5(4):044501. PMID: 30840739; PMCID: PMC6228312.
[10] Sadri A, et al. MRQy: An Open-Source Tool for Quality Control of MR Imaging Data. Med Phys [Internet]. 2020 Dec; 47(12):6029-6038.
Figures
Figure 1. Bi-parametric MRI sequences of differing quality with corresponding PI-RADS v2.1 scores assigned by readers R1-R4
Table 1. Imaging parameters of prostate MRI scans from ProstateX Challenge Dataset9.
Table 2. Coefficients of variation (CV) of quality metrics derived from MRQy software of both ADC and T2W sequences.
Table 3A. Accuracy of PI-RADS scores assigned by four readers in classifying PCa by quality metric-based groups for both ADC and T2W sequences. Table 3B. Inter-reader agreement of PI-RADS scores in classifying PCa by quality metric-based groups for both ADC and T2W sequences.
Figure 2. Confusion matrix representations of diagnostic accuracy of PI-RADS v2.1 in terms of mean AUC and inter-reader agreement in terms of mean Fleiss kappa (κ) among 4 experienced radiologists