Florian Knoll^{1}

This talk will provide a practical hands-on overview of how to get started in machine learning research from the point of view of an imaging lab. Common hurdles and pitfalls will be discussed via didactic examples from classification and reconstruction. The key differences of medical imaging data and computer vision applications will be highlighted. The talk will also discuss software frameworks and implementation, including code demos, which will be made available as open source.

Recent
developments in deep learning^{1}
have led to breakthrough improvements in areas as diverse as image
classication^{2}
semantic labelling^{3},
optical flow^{4},
image restauration^{5}
or playing the game of Go^{6}
. Recently, attempts have been made to leverage neural networks for
medical image reconstruction^{7,8,9,10,11,12,13}.
The goal of this educational talk is to help bridging the gap between
having read research papers on deep learning, and applying these
techniques to dedicated research projects in an imaging lab. Overlaps
of medical imaging with general pattern recognition and computer
vision will be discussed as well as the unique elements encountered
in medical imaging. The talk will be organized around didactic
examples, using well known datasets from pattern recognition^{14},
computer vision^{15}
and custom radiology data. Implementation of fully connected and
convolutional neural networks will be discussed
in general purpose
computing environments like Matlab
(The MathWorks, Natick, MA, USA) as well as specialized deep
learning libraries like Tensorflow^{16} and high
level interfaces like Keras^{17}. Source code of
all presented examples will be made available online. Particular
areas of emphasis that will be discussed in the talk are:

- Collection and organization of data.
- Selection of an appropriate network architecture.
- Selection of an implementation framework.
- Selection and optimization of network hyper-parameters.
- Training a model: Selecting a training algorithm, hyper-parameters and the loss function, monitoring of training performance.
- Evaluation of a trained model: Training, validation and test error.
- Over- and underfitting. Evaluation of robustness, generalization and transfer to related problems.

[1] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, 436–444 (2015).

[2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 1097–1105 (2012).

[3] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs,” in International Conference on Learning Representations (2015).

[4] A. Dosovitskiy, P. Fischer, E. Ilg, P. Häusser, C. Hazırbas ̧, V. Golkov, P. van der Smagt, D. Cremers, and T. Brox, “FlowNet: Learning Optical Flow with Convolutional Networks,” in IEEE International Conference on Computer Vision (ICCV), 2758–2766 (2015).

[5] Y. Chen, W. Yu, and T. Pock, “On learning optimized reaction diffusion processes for effective image restoration,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 5261–5269 (2015).

[6] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, “Mastering the game of Go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489 (2016).

[7] K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll, “Learning a Variational Network for Reconstruction of Accelerated MRI Data,” Magn. Reson. Med., in press (2017).

[8] S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng, D. Liang, “Accelerating magnetic resonance imaging via deep learning”, ISBI 514-517 (2016). [9] K.H. Jin, M.T. McCann, E. Froustey, M, Unser, “Deep Convolutional Neural Network for Inverse Problems in Imaging”, https://arxiv.org/abs/1611.03679 (2016).

[10] K. Kwon, D. Kim, H. Seo, J. Cho, B. Kim, H.W. Park, “Learning-based Reconstruction using Artificial Neural Network for Higher Acceleration”, in Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM), p1801 (2016).

[11] G. Wang, “Perspective on Deep Imaging”, IEEE Access 8914-8924 (2016)

[12] V Golkov, A Dosovitskiy, J.I. Sperl, M.I. Menzel, M. Czisch, P. Saemann, T. Brox, D. Cremers, “q-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI Scans”, IEEE TMI 35: 1344-1351 (2016).

[13] F Knoll, K Hammernik, E Garwood, A Hirschmann, L Rybak, M Bruno, T Block, J Babb, T Pock, DK Sodickson and MP Recht, “Accelerated knee imaging using a deep learning based reconstruction” in Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM) p645 (2017).

[14] R. A. Fisher. "The use of multiple measurements in taxonomic problems". Annals of Eugenics 7: 179–188 (1936).

[15] LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. “Gradient-based learning applied to document recognition”. Proceedings of the IEEE, 86, 2278–2324 (1998).

[16] M. Abadi, P. Barham, J.
Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving,
M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray,
B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, X.
Zheng, and G. Brain, “TensorFlow: A System for Large-Scale Machine
Learning,” in 12th USENIX Symposium on Operating Systems Design and
Implementation (OSDI ’16), 265–284 (2016).

[17] François Chollet, “Keras” GitHub (2015).