Machine learning methods have found wide use in MRI reconstruction, with a recent focus on artificial neural networks, in particular convolutional neural networks. In this talk, we will overview both model-based and data-driven machine learning approaches for reconstruction. We will also consider practical aspects of implementing deep artificial neural networks for MRI reconstruction.
Model-based ML techniques have been used in MRI reconstruction, primarily for image regularization. Earlier works have utilized a dictionary model to represent “blocks” of image data in a sparse manner (1,2). These methods were used in conjunction with a compressed sensing framework (3,4) to improve upon reconstruction using sparsity in pre-defined transform domains. Such dictionaries may be learned for a given dataset or from a training database, ultimately generating an adaptive linear transformation of the data, while the overall reconstruction process is non-linear. Similar ideas were explored in other works (5,6).
The utility of non-linear processing was also explored from other perspectives. Specifically, the non-linear GRAPPA approach (7) utilized the so-called kernel methods (8) to improve k-space interpolation used in GRAPPA (9). The main idea of kernel methods is to map the data to a higher-dimensional feature space, using pre-defined non-linear transforms. In this feature space, linear estimation is performed, which corresponds to a non-linear operation in the underlying data space. This method was shown to reduce parallel imaging artifacts in certain applications. However, the choice of the non-linear transforms, i.e. kernels, is heuristic, thus not leading to a fully data-driven approach.
Another line of work utilizes non-linear manifold learning for dynamic MRI reconstruction (10-12). These methods model the imaging data as a low-dimensional non-linear “surface,” which is characterized by a few underlying unknown parameters. The manifold structure can be learned in multiple ways, including Laplacian eigenmaps (13) that learn local geometry, as well as kernel methods (8) that allow regularization in higher-dimensional feature spaces. These two approaches were used for dynamic MRI reconstruction in (11) and (12), respectively. These manifold learning methods also utilize pre-determined non-linearities, either to determine local distances among points on the manifold or as mappings to the feature space.
Recent efforts have focused on more data-driven approaches that aim to learn non-linear relationships from the MRI data. Most of these methods utilize artificial neural networks, along with large training databases of images/measurements to perform the learning. A common line of approach learns to represent the main features of the underlying images using convolutional neural networks (CNNs) (14-18). In effect, these methods extend on the earlier dictionary models, by incorporating more complicated data-driven non-linear models. Deep learning of the CNNs is performed using imaging databases, and various ideas including unfolding of reconstruction iterations (14) and generative adversarial networks (17) have also been explored.
Deep learning approaches have also been used to infer other parts of the MRI acquisition system. In (19), the process of generating images from raw data was studied for various MRI acquisitions. A neural network with fully-connected layers, trained on a large database, was utilized to provide a uniform reconstruction framework for various acquisition schemes. In (20), CNNs were used to learn the redundancies among the channels of the receiver coil-array from scan-specific ACS data, without the need for training databases. Thus, this work extends on the linear convolutions (9) and kernel-based approaches (7) used previously for parallel imaging.
Implementation of MRI reconstruction techniques based on neural network approaches are subject to certain other practical considerations. As we study these different approaches, we will also look at different loss functions, optimization approaches for minimizing such losses, backpropagation, and other implementation points.
1. Doneva M, Bornert P, Eggers H, Stehning C, Senegas J, Mertins A. Compressed sensing reconstruction for magnetic resonance parameter mapping. Magn Reson Med 2010;64(4):1114-1120. 2. Ravishankar S, Bresler Y. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans Med Imaging 2011;30(5):1028-1041.
3. Lustig M, Donoho DL, Pauly JM. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn Reson Med 2007;58(6):1182-1195.
4. Block KT, Uecker M, Frahm J. Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint. Magn Reson Med 2007;57(6):1086-1098.
5. Akcakaya M, Basha TA, Goddu B, Goepfert LA, Kissinger KV, Tarokh V, Manning WJ, Nezafat R. Low-dimensional-structure self-learning and thresholding: Regularization beyond compressed sensing for MRI Reconstruction. Magn Reson Med 2011;66(3):756-767.
6. Huang Y, Paisley J, Lin Q, Ding X, Fu X, Zhang XP. Bayesian nonparametric dictionary learning for compressed sensing MRI. IEEE Trans Image Process 2014;23(12):5007-5019.
7. Chang Y, Liang D, Ying L. Nonlinear GRAPPA: a kernel approach to parallel MRI reconstruction. Magn Reson Med 2012;68(3):730-740.
8. Schölkopf B, Smola AJ. Learning with kernels : support vector machines, regularization, optimization, and beyond. Cambridge, Mass.: MIT Press; 2002.
9. Griswold MA, Jakob PM, Heidemann RM, Nittka M, Jellus V, Wang J, Kiefer B, Haase A. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn Reson Med 2002;47(6):1202-1210.
10. Usman M, Vaillant G, Atkinson D, Schaeffter T, Prieto C. Compressive manifold learning: estimating one-dimensional respiratory motion directly from undersampled k-space data. Magn Reson Med 2014;72(4):1130-1140.
11. Poddar S, Jacob M. Dynamic MRI Using SmooThness Regularization on Manifolds (SToRM). IEEE Trans Med Imaging 2016;35(4):1106-1115.
12. Nakarmi U, Wang Y, Lyu J, Liang D, Ying L. A Kernel-Based Low-Rank (KLR) Model for Low-Dimensional Manifold Recovery in Highly Accelerated Dynamic MRI. IEEE Trans Med Imaging 2017;36(11):2297-2307.
13. Belkin M, Niyogi P. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 2003;15(6):1373-1396.
14. Hammernik K, Klatzer T, Kobler E, Recht MP, Sodickson DK, Pock T, Knoll F. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 2018;79(6):3055-3071.
15. Schlemper J, Caballero J, Hajnal JV, Price AN, Rueckert D. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans Med Imaging 2018;37(2):491-503.
16. Han Y, Yoo J, Kim HH, Shin HJ, Sung K, Ye JC. Deep learning with domain adaptation for accelerated projection-reconstruction MR. Magn Reson Med 2018.
17. Mardani M, Gong E, Cheng JY, Zaharchuk G, Vasanawala S, Pauly J. Generative Adversarial Networks for Compressed Sensing (GANCS) MRI. 2018; Pacific Grove, CA. Proceedings of the ISMRM Workshop on Machine Learning.
18. Lee D, Yoo J, Ye JC. Deep Artifact Learning for Compressed Sensing and Parallel MRI. 2017; Melbourne, Australia. IEEE International Symposium on Biomedical Imaging. p 15-18.
19. Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature 2018;555(7697):487-492.
20. Akcakaya M, Moeller S, Weingartner S, Ugurbil K. Scan-Specific Deep Learning with Robust Artifical-Neural-Networks for k-space Interpolation (RAKI) for Improved Parallel Imaging. 2018; Pacific Grove, CA. Proceedings of the ISMRM Workshop on Machine Learning.