0583
Improving Reproducibility in Quantitative 4D Flow MRI Using AI-Driven Fully-Automated Processing and Analysis1Northwestern University, Chicago, IL, United States, 2Ann & Robert H. Lurie Children’s Hospital, Chicago, IL, United States
Synopsis
Keywords: Quantitative Imaging, Quantitative Imaging, 4D flow MRI, hemodynamics
Motivation: Reproducibility is fundamentally an issue for quantitative MRI, and any human intervention required for processing can be a significant source of variability.
Goal(s): This study aims to improve reproducibility in quantitative 4D flow MRI by removing all human input from processing, using AI-driven tools.
Approach: Hemodynamic parameters quantified by a fully automated neural-network-based processing tool for 4D flow MRI were compared to quantifications performed by two sets of human observers.
Results: Moderate but appreciable limits of agreement were observed between quantifications performed by different human observers. Quantified values from fully-automated processing were comparable to those from humans, but all inter-observer variability was eliminated.
Impact: This study offers a stable baseline for improving measurement reliability in quantitative 4D flow MRI by removing all manual human inputs required.
Introduction
Quantification of cardiovascular hemodynamics, through parameters such as maximal aortic velocities or regional kinetic energy, can offer powerful clinical insights, for example in the context of managing aortic valve disease or aortic dissection. Such parameters and others can be derived directly from time-resolved 3D phase-contrast (4D flow) MRI data. However, a longstanding challenge for both quantitative MRI in general and 4D flow MRI in particular is establishing reproducibility of derived values. While several software tools for processing 4D flow MRI in a standardized manner have become mainstream in recent years, significant variability in quantifications made with these tools has been observed1. One source of variability for these quantifications is the need for some manual human intervention to process the MRI data, such as segmentation of vessels or selecting correction settings. Recent work to fully automate all steps of 4D flow MRI processing using AI techniques has provided a framework for removing all manual input from the quantification of hemodynamic parameters2,3. Here, we demonstrate the degree of variability that could be removed from quantification by using this approach for fully-automated processing, thus improving the potential reproducibility of quantitative 4D flow MRI.Methods
For evaluation of parameter stability through automated or manual processing, a collection of datasets were processed both completely automatically, using a previously-developed AI-based tool3 (fig.1), and completely manually. Manual processing was performed twice per dataset by different observers ("manual1", "manual2"; divided among six total). Automated processing ("auto-AI") was also performed twice to demonstrate determinism of the quantifications.Data were acquired by 4D flow with prospective-/retrospective-gating at 1.5T/3T, resolution 1.5-2.5×1.5-2.5×1.5-4.5mm3/30-41ms, R=2-8 mixed GRAPPA/GRAPPA+CS, VENC 80-300cm/s, sagittal-oblique with thoracic aorta coverage. In total, 246 datasets were acquired from a mix of healthy controls (n=101), and adult (n=127) and pediatric (n=18) BAV patients, at multiple separate MR sites at two institutions.
The tool for fully-automated handling of 4D flow MRI data automatically identified processing/analysis needed and used modular routines for standard pre-processing corrections (noise-masking, un-aliasing, background phase-removal), aortic vessel segmentation (ascending [AAo], arch, descending [DAo]), and quantitative parameter calculation. Pre-processing corrections and segmentations used individually-trained neural networks (U-Net with DenseNet blocks)4-6. The tool then used the corrected velocity data and vessel segmentations with algorithmic calculation of regional volume-averaged maximal velocity (Vmax), kinetic energy (KE), energy loss (EL) and stasis (fig.2)7. The same algorithmic operations were performed to derive values from the manually processed datasets (fig.2). Bland-Altman analysis calculating mean bias and limits-of-agreement was used to compare manual1 vs. manual2, auto-AI vs. manual1, auto-AI vs. manual2, and auto-AI vs. auto-AI (re-run).
Results
For quantifications of Vmax, KE, EL, and stasis, there was moderate or relatively close agreement between human observers (fig.3). Agreement for KE was the most discrepant. The limits of agreement for Vmax, KE, EL, and stasis were ±0.13m/s, ±0.24uJ, ±0.02uJ, and ±7.45%-points respectively in human vs. human quantification, and bias was low in all parameters. In comparison of auto-AI vs. human quantification of the same parameters, the limits of agreement were comparable to those of human vs. human (fig.3). For auto-AI vs. manual1 or manual2, they were ±0.09m/s or ±0.13m/s, ±0.20uJ or ±0.22uJ, ±0.01uJ or ±0.02uJ, and ±3.76%-points or ±7.84%-points respectively, also with low bias for all parameters. Lastly, in comparison of auto-AI vs. auto-AI, absolutely no differences in quantifications were observed between the first and second run.Discussion
The quantification stability from manual1 to manual2 for the hemodynamic parameters evaluated here was acceptable, and it would not be expected to introduce significant confounds in a large cohort (>100 subjects) study. However, in smaller cohorts, it could pose a challenge for observing subtler hemodynamic differences, with the degree of inter-observer variability potentially overwhelming the effect size. In contrast, the automated AI-driven had no 'inter-observer' discrepancy. While such a comparison of the method against itself is somewhat tautological, it illustrates how the approach completely eliminates inter-observer variability by removing all human input. No ground truth reference values were available to assess the accuracy of any observer, but the AI-based quantifications did not show significant bias when compared to either set of human observations. This suggests that the automated quantifications should not be in any lower regime of accuracy than the current standard with manual quantification steps.Conclusion
Complete automation of processing for quantitative MRI is an important factor for improving reproducibility. While establishing a high degree of reproducibility overall is a longstanding challenge for MRI, with system imperfections and site-specific factors potentially contributing to variability of quantifications, removing manual inputs establishes a stable baseline for improving measurement reliability.Acknowledgements
No acknowledgement found.References
1. Oechtering TH, Nowak A, Sieren MM, Stroth AM, Kirschke N, Wegner F, et al. Repeatability and reproducibility of various 4D Flow MRI postprocessing software programs in a multi-software and multi-vendor cross-over comparison study. J Cardiovasc Magn Reson. 2023 Mar 28;25(1):22.
2. Scott MB, Berhane H, Baraboo J, Rigsby CK, Robinson JD, McCarthy PM, et al. Fully automated aortic 4D flow MRI large-cohort analysis using deep learning. Proceedings of the ISMRM. 2021;451.
3. Johnson EMI, Berhane H, Weiss EK, Sodhi A, Jarvis K, Scott M, et al. End-to-end Automation of Quantitative Processing for 4D Flow MRI in the Aorta: Demonstration and Evaluation in 271 Subjects. Proceedings of the ISMRM. 2023;933.
4. Berhane H, Scott M, Elbaz M, Jarvis K, McCarthy P, Carr J, et al. Fully automated 3D aortic segmentation of 4D flow MRI for hemodynamic analysis using deep learning. Magn Reson Med. 2020 Oct;84(4):2204–18.
5. Berhane H, Scott M, Baraboo J, Rigsby C, Robinson J, Allen B, et al. Deep Learning based Automatic Multi-Regional Segmentation of the Aorta form 4D Flow MRI. Proceedings of the ISMRM. 2021;2647.6. Berhane H, Scott MB, Barker AJ, McCarthy P, Avery R, Allen B, et al. Deep learning–based velocity antialiasing of 4D‐flow MRI. Magnetic Resonance in Med. 2022 Jul;88(1):449–63.
7. Jarvis K, Pruijssen JT, Son AY, Allen BD, Soulat G, Vali A, et al. Parametric Hemodynamic 4D Flow MRI Maps for the Characterization of Chronic Thoracic Descending Aortic Dissection. Magnetic Resonance Imaging. 2020 May;51(5):1357–68.
Figures
