Synopsis

This talk will provide a practical hands-on overview of how to get started in machine learning research from the point of view of an imaging lab. Common hurdles and pitfalls will be discussed via didactic examples from classification and reconstruction. The key differences of medical imaging data and computer vision applications will be highlighted. The talk will also discuss software frameworks and implementation, including code demos, which will be made available as open source.

Highlights

Overview of educational talk

Recent developments in deep learning¹ have led to breakthrough improvements in areas as diverse as image classication² semantic labelling³, optical flow⁴, image restauration⁵ or playing the game of Go⁶ . Recently, attempts have been made to leverage neural networks for medical image reconstruction^{7,8,9,10,11,12,13}. The goal of this educational talk is to help bridging the gap between having read research papers on deep learning, and applying these techniques to dedicated research projects in an imaging lab. Overlaps of medical imaging with general pattern recognition and computer vision will be discussed as well as the unique elements encountered in medical imaging. The talk will be organized around didactic examples, using well known datasets from pattern recognition¹⁴, computer vision¹⁵ and custom radiology data. Implementation of fully connected and convolutional neural networks will be discussed in general purpose computing environments like Matlab (The MathWorks, Natick, MA, USA) as well as specialized deep learning libraries like Tensorflow¹⁶ and high level interfaces like Keras¹⁷. Source code of all presented examples will be made available online. Particular areas of emphasis that will be discussed in the talk are:

• Collection and organization of data.
• Selection of an appropriate network architecture.
• Selection of an implementation framework.
• Selection and optimization of network hyper-parameters.
• Training a model: Selecting a training algorithm, hyper-parameters and the loss function, monitoring of training performance.
• Evaluation of a trained model: Training, validation and test error.
• Over- and underfitting. Evaluation of robustness, generalization and transfer to related problems.

Conclusion

Acknowledgements

References

[1] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, 436–444 (2015).

[2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 1097–1105 (2012).

[3] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs,” in International Conference on Learning Representations (2015).

[4] A. Dosovitskiy, P. Fischer, E. Ilg, P. Häusser, C. Hazırbas ̧, V. Golkov, P. van der Smagt, D. Cremers, and T. Brox, “FlowNet: Learning Optical Flow with Convolutional Networks,” in IEEE International Conference on Computer Vision (ICCV), 2758–2766 (2015).

[5] Y. Chen, W. Yu, and T. Pock, “On learning optimized reaction diffusion processes for effective image restoration,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 5261–5269 (2015).

[6] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, “Mastering the game of Go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489 (2016).

[7] K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll, “Learning a Variational Network for Reconstruction of Accelerated MRI Data,” Magn. Reson. Med., in press (2017).

[8] S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng, D. Liang, “Accelerating magnetic resonance imaging via deep learning”, ISBI 514-517 (2016). [9] K.H. Jin, M.T. McCann, E. Froustey, M, Unser, “Deep Convolutional Neural Network for Inverse Problems in Imaging”, https://arxiv.org/abs/1611.03679 (2016).

[10] K. Kwon, D. Kim, H. Seo, J. Cho, B. Kim, H.W. Park, “Learning-based Reconstruction using Artificial Neural Network for Higher Acceleration”, in Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM), p1801 (2016).

[11] G. Wang, “Perspective on Deep Imaging”, IEEE Access 8914-8924 (2016)

[12] V Golkov, A Dosovitskiy, J.I. Sperl, M.I. Menzel, M. Czisch, P. Saemann, T. Brox, D. Cremers, “q-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI Scans”, IEEE TMI 35: 1344-1351 (2016).

[13] F Knoll, K Hammernik, E Garwood, A Hirschmann, L Rybak, M Bruno, T Block, J Babb, T Pock, DK Sodickson and MP Recht, “Accelerated knee imaging using a deep learning based reconstruction” in Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM) p645 (2017).

[14] R. A. Fisher. "The use of multiple measurements in taxonomic problems". Annals of Eugenics 7: 179–188 (1936).

[15] LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. “Gradient-based learning applied to document recognition”. Proceedings of the IEEE, 86, 2278–2324 (1998).

[16] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, X. Zheng, and G. Brain, “TensorFlow: A System for Large-Scale Machine Learning,” in 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16), 265–284 (2016).

[17] François Chollet, “Keras” GitHub (2015).