The short-course is broken down into three sections:
First Hour: Foundations of Imaging Studies
Second Hour: Statistical Methods in Imaging Studies
Third Hour: Advanced Methods
Preparing to meet with a biostatistician means knowing the right questions to ask about study design and analysis. These meetings should be collaborative in nature, sharing information about the physics and medicine with the biostatistician while the biostatistician works to understand the study objectives and considers biases that could be detrimental to answering the objectives. Iteration of the study objectives and design indicates good collaboration. In six mini presentations, Drs. Alonzo and Obuchowski will present key concepts in study design and analysis for common imaging studies and their methods, including quantitative imaging biomarkers and ROC analysis. This foundation will help you to collaborate effectively with biostatisticians.
Part 1: This is the first lecture on the foundations of imaging studies, with a focus on study design. We will represent the design of imaging studies as four building blocks. We will discuss each block, as well as common problems and possible solutions. Two classic diagnostic accuracy studies, representing prospective and retrospective designs, will be compared. Commonly used strategies, such as pairing, randomization, enrichment, and interim analyses, will be discussed.
Building blocks of imaging studies
o Study Objective
§ Active statement about specific steps to answer research question.
§ What does a good objective look like?
· Should include relevant population(s), endpoint
· Example
o Imaging Test under investigation
o Reference Standard
§ Source of information, completely different from test under evaluation, which tells the true condition status of subject
§ Common examples
§ What if there is no reference standard?
· Correlation
· Agreement
· Interchangeability
o Endpoints
§ Sensitive, reliably measured, clinically relevant
§ CAD Example
· Prospective vs. Retrospective studies
o Study flow o Examples
· Strategies to improve study efficiency
o Paired design § Better than randomization
o Blinded design
o Augmented and enriched samples
o Cross-over vs. sequential designs
o Randomization
§ Simple
§ Block
§ Stratified
o Interim analyses
§ Multiplicity problem
§ Stopping rules
Part 2: This is the first lecture on statistical methods used in imaging studies, with a focus on methods for quantitative imaging biomarkers (QIBs). We will begin with definitions and examples of QIBs used in medicine. We will then discuss measures of the technical performance of QIBs. Unlike diagnostic and screening tests where the focus is on diagnostic accuracy, for QIBs we focus on bias, precision, and the property of linearity. These measures of performance will be used to find a threshold for determining when a true change has occurred in a subject and for quantifying the true value or true change.
· Quantitative Imaging Biomarkers
o Definition
o Examples
· Technical Performance
o Motivation for studying technical performance
o Bias
§ Systematic deviation from ground truth
§ Fixed and proportional bias
§ Linearity
o Precision
§ Closeness of agreement among replicates
§ Repeatability vs. Reproducibility
§ Test-retest studies
§ Within-subject standard deviation (wSD)
§ Within-subject coefficient of variation (wCV)
§ Precision profile
o Aggregate vs. Disaggregate; Scaled vs. Unscaled metrics
§ ICC § Coverage probability curve
· Applications
o Confidence in true biomarker value
o Threshold for distinguishing true change from measurement noise
§ Repeatability coefficient
This is the first lecture on advanced statistical methods for imaging studies, with a focus on multi-reader multi-case (MRMC) Receiver Operating Characteristic studies. We will discuss the role of multiple-reader studies, the assessment of inter-reader variability, and the design and analysis of ROC studies with multiple readers. Studies where readers are tasked with finding lesions on an image, as in screening studies, will be discussed, along with the ROC methods to handle multiple possible true and false positives. Basic concepts in MRMC sample size planning with multiple readers will be addressed.
· Goals of multi-reader imaging studies
o Assess inter-reader variability
§ Kappa
o Estimate performance of a population of readers
o Compare two modalities
· Multi-reader ROC study designs
o Correlated data
§ Paired reader, paired subject (traditional)
§ Paired reader, unpaired subject
§ Unpaired reader, paired subject
§ Hybrid design
o Sample size considerations
· Multi-reader ROC analysis
o Reader variation in latent thresholds
o MRMC software
o Clustered data
§ Location bias
Obuchowski NA, Gazelle GS. Handbook for Clinical Trials of Imaging and Image-Guided Interventions. 2016 Wiley and Sons, Inc., New York
Zhou XH, Obuchowski NA, McClish DL. Statistical Methods in Diagnostic Medicine. 2nd edition 2011. Wiley and Sons, Inc., New York
McGowan LD, Bullen JA, Obuchowski NA. Location bias in ROC studies. Statistics in Biopharmaceutical Research 2016; 8: 258-267
Sullivan DC, Obuchowski NA, Kessler LG, et al. Metrology Standards for Quantitative Imaging Biomarkers. Radiology 2015; 277: 813-825.