Machine Learning & Deep Learning: Technical Introduction

Florian Knoll¹

¹Department of Radiology, New York University, New York, NY, United States

Synopsis

This talk will provide an overview of the technical background of machine learning and deep learning in medical imaging. Common hurdles and pitfalls will be discussed via didactic examples from classification and reconstruction. Examples will be provided from a range of MRI applications, with special focus on body imaging.

Highlights

Understand the technical background and basic principles of machine learning and deep learning techniques that have recently been proposed in MRI.

Target Audience

Imaging researchers interested in machine learning.

Outcome/Objective

Provide imaging researchers the necessary background information and tools to assess recent developments in machine learning in Radiology, and put them in a position to start their own machine learning research projects.

Overview of educational talk

Recent developments in deep learning¹ have led to breakthrough improvements in areas as diverse as image classication² semantic labelling^3, optical flow⁴, image restauration⁵ or playing the game of Go⁶. Recently, attempts have been made to leverage neural networks for medical image reconstruction^{7,8,9,10,11,12,13}. The goal of this educational talk is to give the audience the necessary technical background to assess the potential of this technology for applications in Radiology, and to bridge the gap between having read research papers on deep learning, and applying these techniques to dedicated research projects in an imaging lab. Overlaps of medical imaging with general pattern recognition and computer vision will be discussed as well as the unique elements encountered in medical imaging. The talk will be organized around didactic examples, using well known datasets from pattern recognition¹⁴, computer vision¹⁵ and custom radiology data. Particular areas of emphasis that will be discussed in the talk are:

Basic principles of neural networks.
Understanding the optimization problem associated with the procedure of training a neural network.
Collection and organization of data.
Selection of an appropriate network architecture.
Selection of an implementation framework.
Selection and optimization of network hyper-parameters.
Training a model: Selecting a training algorithm, hyper-parameters and the loss function, monitoring of training performance.
Evaluation of a trained model: Training, validation and test error. Over- and underfitting.
Evaluation of robustness, generalization and transfer to related problems.

Conclusion

While deep learning and neural networks open up exciting possibilities in Radiology, translation of developments from computer vision and pattern recognition are sometimes not straight-forward. The development of approaches that are both robust and practical enough so that they can replace currently used clinical methods is still an open research topic.

Acknowledgements

NIH R01 EB024532, NIH P41 EB017183.

References

[1] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, 436–444 (2015).

[2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 1097–1105 (2012).

[3] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs,” in International Conference on Learning Representations (2015).

[4] A. Dosovitskiy, P. Fischer, E. Ilg, P. Häusser, C. Hazırbas ̧, V. Golkov, P. van der Smagt, D. Cremers, and T. Brox, “FlowNet: Learning Optical Flow with Convolutional Networks,” in IEEE International Conference on Computer Vision (ICCV), 2758–2766 (2015).

[5] Y. Chen, W. Yu, and T. Pock, “On learning optimized reaction diffusion processes for effective image restoration,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 5261–5269 (2015).

[6] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, “Mastering the game of Go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489 (2016).

[7] K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll, “Learning a Variational Network for Reconstruction of Accelerated MRI Data,” Magn. Reson. Med., in press (2017).

[8] S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng, D. Liang, “Accelerating magnetic resonance imaging via deep learning”, ISBI 514-517 (2016). [9] K.H. Jin, M.T. McCann, E. Froustey, M, Unser, “Deep Convolutional Neural Network for Inverse Problems in Imaging”, https://arxiv.org/abs/1611.03679 (2016).

[10] K. Kwon, D. Kim, H. Seo, J. Cho, B. Kim, H.W. Park, “Learning-based Reconstruction using Artificial Neural Network for Higher Acceleration”, in Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM), p1801 (2016).

[11] G. Wang, “Perspective on Deep Imaging”, IEEE Access 8914-8924 (2016).

[12] V Golkov, A Dosovitskiy, J.I. Sperl, M.I. Menzel, M. Czisch, P. Saemann, T. Brox, D. Cremers, “q-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI Scans”, IEEE TMI 35: 1344-1351 (2016).

[13] F Knoll, K Hammernik, E Garwood, A Hirschmann, L Rybak, M Bruno, T Block, J Babb, T Pock, DK Sodickson and MP Recht, “Accelerated knee imaging using a deep learning based reconstruction” in Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM) p645 (2017).

[14] R. A. Fisher. "The use of multiple measurements in taxonomic problems". Annals of Eugenics 7: 179–188 (1936).

[15] LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. “Gradient-based learning applied to document recognition”. Proceedings of the IEEE, 86, 2278–2324 (1998).

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)