Applications of Machine Learning: Data Acquisition & Image Reconstruction
Shanshan Wang1, Xin Liu1, and Hairong Zheng1
1Paul C. Lauterbur Research Center for Biomedical Imaging, SIAT, Chinese Academy of Sciences, Shenzhen, P.R.China, shenzhen, China

Synopsis

Machine learning, especially deep learning, has shown great potential in accelerating MR imaging lately. To accelerate MR imaging with deep learning, the sampling trajectories can be Cartesian or Non-Cartesian subsampling patterns. While the reconstruction methods can be roughly categorized into end-to-end data-driven learning reconstruction methods and model based unrolled iterative learning reconstruction methods. This educational lecture will briefly go through these methods and provide a starting point for researchers interested in this field.

Introduction

Machine learning has been very popular. Even in 1994 there were works using neural network for NMR spectral estimation [1]. Lately, deep learning, as an important branch of machine learning, has attracted unprecedented public attention, showing great potential for diverse medical imaging tasks. Specifically, from 2016, deep learning has been incorporated into the fast MR imaging task, which draws valuable prior knowledge from big datasets to facilitate accurate MR image reconstruction from limited measurements [2,3]. In the meantime and thereafter, there are different works proposed [4-22].

Methods

Let x be the image to be recovered and y be the corresponding k-space measurements, its typical reconstruction model can be described as follows
$\arg\min_{x}||y-Ex||_2^2+R(x)$
where E is the acquisition encoding matrix, which can consist of the sampling trajectory, Fourier encoding matrix and multi-coil sensitivities for parallel imaging, and R(x) is the prior knowledge regularization that assists solving the inverse reconstruction problem. To accelerate MR imaging with deep learning, the sampling trajectories can be Cartesian or Non-Cartesian subsampling patterns. The former one includes 1D uniform, 1D random sampling and 2D random sampling patterns, etc. While for the latter one, there are radial, spiral sampling schemes and so on so forth. Cartesian sampling is widely used in conventional MRI and also can be easily incorporated into deep learning based MR imaging techniques. Non-Cartesian sampling schemes (e.g. radial or spiral sampling) offer advantages over Cartesian schemes but its reconstruction normally needs extra regridding layers. For different body parts scanning, it is normally a balance of flexibility and efficiency of k-space sampling, motion insensitivity, and the ability to generate images with high spatio-temporal resolution from limited data.
For deep learning MR reconstruction methods, based on the framework and its inputs and outputs, reconstruction methods can be roughly categorized into two types, end-to-end data-driven learning reconstruction methods and model based unrolled CSMRI learning reconstruction. The first category learns a mapping between data pairs, the input of which can be undersampled k-space data or aliased image while the output is fully-sampled k-space or artifact-free groundtruth images. For the latter one, these methods unrolled the traditional iterative reconstruction or CS-MRI methods for solving the above formulation. Both two category methods have their pros and cons. While data-driven one can assist more accurate MR reconstruction if the testing data follow the similar distribution as the training data and the dataset is big, the model driven method has stronger generalization capability and better theoretical explanations.

Summary

Deep learning methods have shown big potential for the next generation of fast MR image technique. While most of the methods show encouraging performances and point out interesting directions, there are also some misconceptions happening from time to time and ignorance of the data bias and domain shift issues, which care must be given to, so as to fight for a brighter future. Opportunity always lies in Challenges. More robust methods with strong theoretical explanations could be available in the near future if we continue to devote efforts in this direction.

Acknowledgements

Supported by the National Natural Science Foundation of China (61871371, 81830056), Key-Area Research and Development Program of GuangDong Province (2018B010109009), Science and Technology Planning Project of Guangdong Province (2017B020227012), the Basic Research Program of Shenzhen (JCYJ20180507182400762), Youth Innovation Promotion Association Program of Chinese Academy of Sciences (2019351).

References

  1. S.Venkataraman, Z-P Liang, R.L. Magin, A neural network approach to NMR spectral estimation, ISMRM 1994.
  2. Wang S, Su Z, Ying L, Peng X, Zhu S, Liang F, Feng D, Liang D. Accelerating magnetic resonance imaging via deep learning. IEEE 13th International Symposium on Biomedical Imaging (ISBI); 2016. p 514-517.
  3. Yang Y, Sun J, Li H, Xu Z. Deep ADMM-Net for compressive sensing MRI. Advances in neural information processing systems; 2016. p 10-18.
  4. Kinam Kwon Dongchan Kim HyunWook Park, A parallel MR imaging method using multilayer perceptron, Medical physics, 2017
  5. Hammernik K, Klatzer T, Kobler E, Recht MP, Sodickson DK, Pock T, Knoll F. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 2018;79(6):3055-3071.
  6. Lee D, Yoo J, Tak S, Ye JC. Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks. IEEE Trans Biomed Eng 2018;65(9):1985-1995.
  7. Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature 2018;555(7697):487-492.
  8. Mardani M, Gong E, Cheng JY, Vasanawala SS, Zaharchuk G, Xing L, Pauly JM. Deep Generative Adversarial Neural Networks for Compressive Sensing MRI. IEEE Trans Med Imaging 2019;38(1):167-179.
  9. Han Y, Sunwoo L, Ye JC. k-Space Deep Learning for Accelerated MRI. IEEE Trans Med Imaging 2019.
  10. Aggarwal HK, Mani MP, Jacob M. MoDL: Model-Based Deep Learning Architecture for Inverse Problems. IEEE Trans Med Imaging 2019;38(2):394-405.
  11. Zhang J, Ghanem B. ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018. p 1828-1837.
  12. Schlemper J, Caballero J, Hajnal JV, Price AN, Rueckert D. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans Med Imaging 2018;37(2):491-503.
  13. Qin C, Schlemper J, Caballero J, Price AN, Hajnal JV, Rueckert D. Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans Med Imaging 2019;38(1):280-290.
  14. Akcakaya M, Moeller S, Weingartner S, Ugurbil K. Scan-specific robust artificial-neural-networks for k-space interpolation (RAKI) reconstruction: Database-free deep learning for fast imaging. Magn Reson Med 2019;81(1):439-453.
  15. Mardani M, Sun Q, Donoho D, Papyan V, Monajemi H, Vasanawala S, Pauly J. Neural proximal gradient descent for compressive imaging. Advances in Neural Information Processing Systems; 2018. p 9573-9583.
  16. J. A. Fessler, "Optimization Methods for Magnetic Resonance Image Reconstruction: Key Models and Optimization Algorithms," in IEEE Signal Processing Magazine, vol. 37, no. 1, pp. 33-40, Jan. 2020. 16)
  17. F Liu, A Samsonov, L Chen, R Kijowski, L Feng, SANTIS: Sampling‐augmented neural network with incoherent structure for MR image reconstruction, Magnetic resonance in medicine, 2019
  18. Eo T, Jun Y, Kim T, Jang J, Lee HJ, Hwang D. KIKI-net: Cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn. Reson. Med. [Internet] 2018. doi: 10.1002/mrm.27201
  19. Yang G, Yu S, Dong H, et al. DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction. IEEE Trans. Med. Imaging [Internet] 2018;37:1310–1321. doi: 10.1109/TMI.2017.2785879. 19)
  20. TH Kim, JP Haldar, Learning-based computational MRI reconstruction without big data: from linear interpolation and structured low-rank matrices to recurrent neural networks, - Wavelets and Sparsity XVIII, 2019 20)
  21. Shanshan Wang, Huitao Cheng, Leslie Ying, Taohui Xiao, Ziwen Ke, Hairong Zheng and Dong Liang, DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution, Magnetic resonance imaging, 2020, DOI: 10.1016/j.mri.2020.02.002 21)
  22. Shanshan Wang, Ziwen Ke, Huitao Cheng, Sen Jia, Leslie Ying, Hairong Zheng, Dong Liang. DIMENSION: Dynamic MR Imaging with Both K-space and Spatial Prior Knowledge Obtained via Multi-Supervised Network Training, NMR in Biomedicine: 2019 , DOI:10.1002/nbm.4131
Proc. Intl. Soc. Mag. Reson. Med. 28 (2020)