This presentation will describe data-driven methods for image reconstruction, including adaptive dictionaries, sparsifying transforms, convolutional neural network (CNN) models, and deep learning techniques. It will also discuss limitations and challenges of such methods.
Image reconstruction is a key step in the magnetic resonance imaging (MRI) pipeline. Until recently, there have been two primary methods for image reconstruction: analytical and iterative. Analytical methods for image reconstruction use idealized mathematical models for the imaging system. Classical examples are the filtered back-projection method for tomography and the inverse Fourier transform used routinely in MRI. Typically the analytical methods consider only the sampling properties of the imaging system, and ignore the other aspects of the system physics and measurement noise. These reconstruction methods have been used extensively because they require modest computation.
Over the past two decades, image reconstruction has evolved from exclusive use of analytical methods to wider use of iterative or model-based methods that account for the physics of the imaging system, the statistical properties of the measurement noise, and prior models for the object being imaged. A key turning point in the field was the introduction of compressed sensing in about 2005 and its rapid illustration on real MRI applications in about 2007. This led to an explosion of research that finally led to FDA approval of compressed sensing MRI products in 2017 for some MRI vendors.
Until recently, the object priors that have been used for model-based image reconstruction and for compressed sensing have been developed "by hand" by algorithm designers, using standard mathematical image processing tools like wavelet transforms and total variation regularization.
The emerging trend in the field is to replace human-defined signal models with signal models that are learned from data. For example, in MRI there are numerous images available that were acquired with full k-space sampling. One can use machine learning techniques to learn signal models such as dictionaries from that training data and then use those signal models later to reconstruct images from under-sampled data. Another data-driven option is to learn a sparse signal model concurrently with the image reconstruction process, rather than relying on prior training data. This approach is called blind or adaptive dictionary (or transform) learning. These methods are a fairly radical departure from the previous 3+ decades of image reconstruction research where most regularizers were defined using math models and physics, not from data.
The latest trend in this field is to use existing data to train convolutional neural nets (CNNs) to use as a component in an image reconstruction algorithm.
This presentation will summarize some of the recent methods for using adaptive signal models in MR image reconstruction, including dictionary/transform/CNN models, and will discuss limitations and challenges associated with such methods.
S. Ravishankar, A. Lahiri, C. Blocker, and J. A. Fessler. “Deep dictionary-transform learning for image reconstruction.” Proc. IEEE Intl. Symp. Biomed. Imag. 2018, 1208–12.
M. T. McCann, K. H. Jin, and M. Unser. “Convolutional neural networks for inverse problems in imaging: A review.” IEEE Sig. Proc. Mag. 34.6 (Nov. 2017), 85–95.
B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen, and M. S. Rosen. “Image reconstruction by domain-transform manifold learning.” Nature 555 (Mar. 2018), 487–92.
T. M. Quan, T. Nguyen-Duc, and W. K. Jeong. “Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss.” IEEE Trans. Med. Imag. (2018).
G. Yang, S. Yu, H. Dong, G. Slabaugh, P. L. Dragotti, X. Ye, F. Liu, S. Arridge, J. Keegan, Y. Guo, and D. Firmin. “DAGAN: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction.” IEEE Trans. Med. Imag. (2018).
T. Eo, Y. Jun, T. Kim, J. Jang, H-J. Lee, and D. Hwang. “KIKI-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images.” Mag. Res. Med. (2018).
Y. Han, J. Yoo, H. H. Kim, H. J. Shin, K. Sung, and J. C. Ye. “Deep learning with domain adaptation for accelerated projection-reconstruction MR.” Mag. Res. Med. (2018).
K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll. “Learning a variational network for reconstruction of accelerated MRI data.” Mag. Res. Med. 79.6 (June 2018), 3055–71.
T. M. Quan, T. Nguyen-Duc, and W. K. Jeong. “Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss.” IEEE Trans. Med. Imag. (2018).