We propose a deep learning approach for reconstructing undersampled k-space data corrupted by motion. Our algorithm achieves high-quality reconstructions by employing a novel neural network architecture that captures the correlation structure jointly present in the frequency and image spaces. This method provides higher quality reconstructions than techniques employing solely frequency space or solely image space operations. We further characterize the motion severities for which the proposed method is successful. This analysis represents the first step toward fast image reconstruction in the presence of substantial motion, such as in pediatric or fetal imaging.
1. Andre, J. B. et al. Toward Quantifying the Prevalence, Severity, and Cost Associated With Patient Motion During Clinical MR Examinations. J. Am. Coll. Radiol. 12, 689–695 (2015).
2. Aggarwal, H. K., Mani, M. P. & Jacob, M. MoDL: Model-Based Deep Learning Architecture for Inverse Problems. IEEE Trans. Med. Imaging 38, 394–405 (2019).
3. Akçakaya, M., Moeller, S., Weingärtner, S. & Uğurbil, K. Scan-specific robust artificial-neural-networks for k-space interpolation (RAKI) reconstruction: Database-free deep learning for fast imaging. Magnetic Resonance in Medicine vol. 81 439–453 (2019).
4. Cheng, J. Y., Mardani, M., Alley, M. T., Pauly, J. M. & Vasanawala, S. S. DeepSPIRiT: generalized parallel imaging using deep convolutional neural networks. In Annual Meeting of the International Society of Magnetic Resonance in Medicine (2018).
5. Hammernik, K. et al. Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 79, 3055–3071 (2018).
6. Han, Y., Sunwoo, L. & Ye, J. C. k -Space Deep Learning for Accelerated MRI. IEEE Trans. Med. Imaging 39, 377–386 (2020).
7. Hyun, C. M., Kim, H. P., Lee, S. M., Lee, S. & Seo, J. K. Deep learning for undersampled MRI reconstruction. Phys. Med. Biol. 63, 135007 (2018).
8. Lee, D., Yoo, J. & Ye, J. C. Deep residual learning for compressed sensing MRI. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) 15–18 (2017).
9. Putzky, P. & Welling, M. Invert to Learn to Invert. in Advances in Neural Information Processing Systems (eds. Wallach, H. et al.) vol. 32 446–456 (Curran Associates, Inc., 2019).
10. Quan, T. M., Nguyen-Duc, T. & Jeong, W.-K. Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss. IEEE Trans. Med. Imaging 37, 1488–1497 (2018).
11. Schlemper, J., Caballero, J., Hajnal, J. V., Price, A. N. & Rueckert, D. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans. Med. Imaging 37, 491–503 (2018).
12. Yang, Y., Sun, J., Li, H. & Xu, Z. Deep ADMM-Net for Compressive Sensing MRI. in Advances in Neural Information Processing Systems (eds. Lee, D., Sugiyama, M., Luxburg, U., Guyon, I. & Garnett, R.) vol. 29 10–18 (Curran Associates, Inc., 2016).
13. Yang, G. et al. DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction. IEEE Trans. Med. Imaging 37, 1310–1321 (2018).
14. Duffy, B. A. et al. Retrospective correction of motion artifact affected structural MRI images using deep learning of simulated motion. (2018).
15. Haskell, M. W. et al. Network Accelerated Motion Estimation and Reduction (NAMER): Convolutional neural network guided retrospective motion correction using a separable motion model. Magn. Reson. Med. 82, 1452–1461 (2019).
16. Johnson, P. & Drangova, M. Conditional generative adversarial network for three-dimensional rigid-body motion correction in MRI. in (2019).
17. Küstner, T. et al. Retrospective correction of motion-affected MR images using deep learning frameworks. Magn. Reson. Med. 82, 1527–1540 (2019).
18. Oksuz, I. et al. Detection and Correction of Cardiac MRI Motion Artefacts During Reconstruction from k-space. in Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 695–703 (Springer International Publishing, 2019).
19. Usman, M., Latif, S., Asim, M., Lee, B.-D. & Qadir, J. Retrospective Motion Correction in Multishot MRI using Generative Adversarial Network. Sci. Rep. 10, 4786 (2020).
20. Singh, N. M., Iglesias, J. E., Adalsteinsson, E., Dalca, A. V. & Golland, P. Joint Frequency- and Image-Space Learning for Fourier Imaging. arXiv [cs.CV] (2020).
21. Mueller, S. G. et al. Ways toward an early diagnosis in Alzheimer’s disease: The Alzheimer’s Disease Neuroimaging Initiative (ADNI). Alzheimer’s & Dementia vol. 1 55–66 (2005).
Figure 1. Maps of correlation coefficients between a single pixel (circled) and all other pixels in image (left) and frequency (right) space representations of the brain MRI dataset (click to view an animation of correlation patterns at several pixels). Both maps show strong local correlations useful for inferring missing or corrupted data. Frequency space correlations also display conjugate symmetry characteristic of Fourier transforms of real images.
Figure 4. Top: Example reconstructions with motion induced at 3% of scanning lines. The Interleaved and Alternating architectures more accurately eliminate the 'shadow' of the moved brain and the induced ringing and blurring compared to the single-space networks.
Bottom: Comparison of all four network architectures on test samples with motion at varying fractions of lines. For nearly every example, the joint architectures (Interleaved and Alternating) outperform the single-space baselines.
Figure 5. Top: Example Interleaved network reconstructions with motion at varying fractions of lines (columns). Reconstruction quality degrades as motion is present in more k-space lines.
Bottom: Reconstruction error as a function of motion parameters, with 95% confidence intervals. The largest motion parameters within a slice are reported as fractions of their maximum possible value (e.g., image width is the maximum possible horizontal translation). Reconstruction quality is more directly affected by the fraction of lines with motion than by translation or rotation size.