Synopsis
We will give an overview of the current state-of-the-art in deep learning for medical imaging applications such as reconstruction, segmentation and classification. In particular, we will illustrate deep learning approaches based on Convolutional Neural Networks (CNN). We will focus on deep learning models taht use encoder-decoder networks and show these can be used for tasks such as image reconstruction and image segmentation. We show some applications of CNNs in the context of image classification. Finally, we will discuss some open challenges for deep learning approaches such as explainability and verification of deep learning.
Convolutional Neural Networks for Reconstruction, Super-Resolution and Segmentation
Recently, deep learning approaches have achieved significant success in applications such as image reconstruction, super-resolution and segmentation. We will illustrate deep learning approaches that are based on encoder-decoder CNN architectures. First, we will demonstrate how CNNs with encoder-decoder architectures can be used reconstruct MR images from undersampled k-space measurements [1]. Similar approaches can also be used to reconstruction high-resolution MR images from low-resolution input data, so-called super-resolution approaches [2]. Finally, we show how encoder-decoder CNN networks can be used for semantic image segmentation [3].Convolutional Neural Networks for Classification and Decision Support
In the second part of this talk, we will focus on how CNNs can be designed and employed for image classification. We will use an application exemplar of identifying fetal standard scan planes acquired from 2D ultrasound during pregnancy screening examinations: Here CNNs are used to detect automatically 13 fetal standard views in freehand 2D ultrasound data as well as provide a localisation of the fetal structures and organs via a bounding box [4]. An important contribution is that the neural network learns to localise the target anatomy using weak supervision based on image-level labels only. The network architecture is designed to operate in real-time while providing optimal output for the localisation task. We also show an application of deep
learning for cardiac motion analysis in the context of predicting human survival [5].Open challenges
In the final part of the talk we will discuss open challenges for using deep learning for medical images. In particular, we discuss the challenges of learning from small amounts of labelled data (few shot learning), leaning in the presence of domain shifts (transfer learning), analysis and verification of deep leaning approaches as well as explainability and/or interpretability of deep learning.Acknowledgements
We would like to thank the members of the BioMedIA group, Department of Computing, Imperial College London for their help.References
[1] J. Schlemper, J. Caballero, J. V. Hajnal,
A. N. Price and D. Rueckert. A Deep Cascade of Convolutional Neural Networks
for Dynamic MR Image Reconstruction. IEEE Transactions on Medical Imaging,
37(2): 491-503, 2018.
[2] O. Oktay, E. Ferrante, K. Kamnitsas, M. Heinrich, W. Bai, J. Caballero, S.
Cook, A. de Marvao, T. Dawes, D. O'Regan, B. Kainz, B. Glocker and D. Rueckert.
Anatomically
Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement
and Segmentation. IEEE Transactions on Medical Imaging, 37(2):384-395, 2018.
[3] K. Kamnitsas, C. Ledig, V. F. J. Newcombe, J. P. Simpson, A. D. Kane, D. K. Menon, D. Rueckert and B. Glocker. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis 36: 61-78, 2017.
[4] C. F. Baumgartner, K. Kamnitsas, J. Matthew, T. P. Fletcher, S. Smith, L. M. Koch, B. Kainz and D. Rueckert. SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound. IEEE Transactions on Medical Imaging, 36(11): 2204 – 2215, 2017.
[5] Deep learning cardiac motion analysis
for human survival prediction. G. A. Bello, T. J. W. Dawes, J. Duan, C. Biffi.
A. de Marvao, L. S. G. E. Howard, J. S. R. Gibbs, M. R. Wilkins, S. A. Cook, D.
Rueckert and D. P. O'Regan. Nature Machine Intelligence. 1:95-104, 2019.