0805
Rapid Motion Correction with Deep Learning for First-Pass Cardiac Perfusion MRI1Department of Radiology, Northwestern University Feinberg School of Medicine, Chicago, IL, United States, 2Department of Biomedical Engineering, Northwestern University, Evanston, IL, United States, 3Department of Radiology and Imaging Sciences, National Institutes of Health, Bethesda, MD, United States, 4Department of Electrical Engineering, Northwestern University, Evanston, IL, United States, 5Department of Computer Science, Northwestern University, Evanston, IL, United States, 6Division of Cardiology, Internal Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, United States
Synopsis
Motion correction (MoCo) is an important pre-processing step for pixel-by-pixel myocardial blood flow (MBF) quantification from cardiac perfusion MRI. It may also improve throughput of visual evaluation of perfusion images. One commonly used method for MoCo is optical flow (OF), which requires a moderate level of computational demand. In this study, we sought to perform rapid MoCo of respiratory motion on cardiac perfusion images using deep learning (DL). Our results show that the proposed DL MoCo performs 418-times faster than the reference OF approach without loss in accuracy.
Introduction
Although a qualitative evaluation of cardiac perfusion MRI is the clinical norm, myocardial blood flow (MBF) quantification from perfusion images offers numerous advantages, including improving diagnostic accuracy1, particularly in multi-vessel obstructive disease and microvascular dysfunction2, delivering prognostic value3, 4 , and improving the precision for longitudinal studies. One key pre-processing step in pixel-wise MBF mapping is motion correction (MoCo). In addition, MoCo may increase the throughput of visual evaluation of cardiac perfusion images by making it easier on eye tracking. While conventional optical flow (OF) based MoCo has been shown effective5, the lengthy processing time may hinder its inline implementation. In this study, we sought to develop a neural network to learn OF-based MoCo procedure and test whether it is capable of rapidly correcting respiratory motion in perfusion image series without loss in accuracy compared with the traditional OF-based method.Methods
Data preparation: We retrospectively identified 100 patients who underwent clinical stress perfusion protocol with free breathing (mean age = 59 ± 14years; 45 females; 6 short-axis slices [3 stress + 3 rest] with 64 frames) to train (80 Patients) and test (20 patients) our network. Previously described OF-based MoCo method5 was used as the reference. We visually examined the MoCo results by OF and excluded those with considerable residual respiratory motion, in order to avoid learning the wrong model.Network training and testing: There were two main components of our network: a 3D U-Net and a spatial transformer network (STN)6 (Fig. 1). The uncorrected and fixed (typically late contrast enhancement frames) images were concatenated in the first step. The resulting concatenated images were used as input to the 3D U-Net to predict the deformation field. The STN network then deformed the uncorrected images using the deformation field. Normalized cross correlation (NCC) between the network output images and fixed images was used as the objective function. In addition, mean squared error (MSE) function between the network output images and reference images was added to maintain the signal intensity fidelity across time. We fine-tuned the weights of the NCC and MSE loss terms to achieve a good balance between motion correction and signal intensity fidelity. For training, we used 288 2D+time images derived from 80 randomly selected patients (64 frames per slice; training time = 9 hours 34 min). For testing, we used 87 2D+time images derived from 20 remaining patients.
Image analysis: Structural similarity index (SSIM)7 and mutual information (MuInfo)8 were calculated on sequential image pairs (e.g. frame1-frame2, frame2-frame3…) of the perfusion images to evaluate frame-to-frame structural alignment. Signal variation (SV) was calculated as the standard deviation of the second derivatives of signal-time curves for each pixel within the myocardium manually segmented on a single frame at peak wall enhancement, in order to infer alignment accuracy.
Results
Results: Figure 2 shows signal intensity-time profiles and curves from one representative patient. Compared with uncorrected images, the signal intensity-time profiles and signal intensity-time curves were smoother for both MoCo results. As summarized in table 1, the mean processing time per slice on average was 418 times shorter for our DL MoCo (0.26 ± 0.002 s) than the reference OF-based MoCo (108.55 ± 18.75 s) (p<0.05). The SSIM, MuInfo, and SV were worse for uncorrected (SSIM: 0.83 ± 0.05; MuInfo: 1.76 ± 0.60; SV: 183.07 ± 31.97); the corresponding metrics improved with OF-based MoCo (SSIM: 0.97 ± 0.01; MuInfo: 2.47 ± 0.78; SV: 87.78 ± 14.77) and further improved with DL MoCo (SSIM: 0.98 ± 0.01; MuInfo: 3.41 ± 0.26; SV: 52.12 ± 10.69).Conclusion
This study demonstrates that our DL network performs MoCo 418-times faster than the reference OF-based method without loss in accuracy, thereby enable inline implementation. The residual motion was significantly less for DL MoCo than OF-based MoCo results (p<0.05 for all SSIM, MuInfo and SV). This was made possible by fine-tuning the weights of NCC and MSE loss terms, where our intention was to further minimize residual motion compared with OF-based MoCo. Future studies include implementation of an-end-to-end network that incorporates image reconstruction, motion correction, myocardial segmentation, and MBF quantification for clinical translation of quantitative cardiac perfusion MRI.Acknowledgements
This work is supported by National Institutes of Health (R01HL116895, R01HL138578, R21EB024315, R21AG055954, R01HL151079, R21EB030806) and American Heart Association (19IPLOI34760317).References
1. Mordini FE, Haddad T, Hsu LY, Kellman P, Lowrey TB, Aletras AH, Bandettini WP and Arai AE. Diagnostic accuracy of stress perfusion CMR in comparison with quantitative coronary angiography: fully quantitative, semiquantitative, and qualitative assessment. JACC Cardiovasc Imaging. 2014;7:14-22.
2. Patel AR, Antkowiak PF, Nandalur KR, West AM, Salerno M, Arora V, Christopher J, Epstein FH and Kramer CM. Assessment of advanced coronary artery disease: advantages of quantitative cardiac magnetic resonance perfusion analysis. J Am Coll Cardiol. 2010;56:561-9.
3. Brown LAE, Onciul SC, Broadbent DA, Johnson K, Fent GJ, Foley JRJ, Garg P, Chew PG, Knott K, Dall'Armellina E, Swoboda PP, Xue H, Greenwood JP, Moon JC, Kellman P and Plein S. Fully automated, inline quantification of myocardial blood flow with cardiovascular magnetic resonance: repeatability of measurements in healthy subjects. J Cardiovasc Magn Reson. 2018;20:48.
4. Knott KD, Seraphim A, Augusto JB, Xue H, Chacko L, Aung N, Petersen SE, Cooper JA, Manisty C, Bhuva AN, Kotecha T, Bourantas CV, Davies RH, Brown LAE, Plein S, Fontana M, Kellman P and Moon JC. The Prognostic Significance of Quantitative Myocardial Perfusion: An Artificial Intelligence-Based Approach Using Perfusion Mapping. Circulation. 2020;141:1282-1291.
5. Benovoy M, Jacobs M, Cheriet F, Dahdah N, Arai AE and Hsu LY. Robust universal nonrigid motion correction framework for first-pass cardiac MR perfusion imaging. J Magn Reson Imaging. 2017;46:1060-1072.
6. Max Jaderberg KS, Andrew Zisserman, Koray Kavukcuoglu. Spatial Transformer Networks. 2016.
7. Wang Z, Bovik AC, Sheikh HR and Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004;13:600-12.
8. Maes F, Collignon A, Vandermeulen D, Marchal G and Suetens P. Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging. 1997;16:187-98.
Figures
