Nuts & Bolts of Machine Learning
Daniel Rueckert1

1Imperial College London, United Kingdom


This talk will give an overview of machine learning techniques for medical image analysis. We will describe both supervised and supervised machine learning approaches. A particular focus will be on deep learning approaches, including Convolutional Neural Networks (CNN) and how these can be used in cardiovascular MR imaging. We will demonstrate machine learning applications for the fast reconstruction of cardiac MR images from undersampled k-space data, image super-resolution as well segmentation of the cardiovascular anatomy in cine cardiac MRI.

Machine learning for image reconstruction and image quality assessment

Convolutional neural networks (CNN) have shown a great potential in image pattern recognition and segmentation for a variety of tasks. We will show some examples that make use of image-to-image neural networks for reconstructing dynamic sequences of 2D cardiac magnetic resonance (MR) images from undersampled data using a deep cascade of CNNs [1] in order to accelerate the data acquisition process. We show that such an approach cannot only be used to provide state-of-the-art image reconstruction but also enables very fast image reconstruction. We will also show that machine learning can be used to automate the assessment of image quality and to identify the potential occurrence of imaging artefacts such as cardiac and respiratory motion [2].

Machine learning for image super-resolution

CNN based approaches have also been used successfully for super-resolution of cardiac MR imaging. Even though 3D cardiac MR imaging enables accurate analysis of cardiac morphology and physiology, most clinical cardiac MR imaging is dominated by multi-slice 2D imaging. While the 2D imaging does not long acquisition and breath-hold times, it can hamper the visualization of anatomy and quantitative measurements as relatively thick slices are acquired. We show how high resolution 3D volumes can be reconstructed from 2D image stacks using CNNs [3]. The CNNs allows the use of multiple input data acquired from different viewing planes for improved performance. Experimental show that the CNN model outperforms state-of-the-art SR methods in terms of image quality while being computationally efficient.

Machine learning for image segmentation

We also demonstrate an automated analysis method for CMR images, which is based on a fully convolutional network (FCN) [4]. The network is trained and evaluated on a dataset of unprecedented size, consisting of 4,875 subjects with 93,500 pixelwise annotated images, which is by far the largest annotated CMR dataset. By combining FCN with a large-scale annotated dataset, we show that an automated method achieves a performance on par with human experts in analysing CMR images and deriving clinical measures. We anticipate this to be a starting point for automated and comprehensive CMR analysis with human-level performance, facilitated by machine learning. It is an important advance on the pathway towards computer-assisted CVD assessment.

Incorporating prior knowledge into machine learning models

Finally, we show that the incorporation of prior knowledge about organ shape and location is key to improve performance of image analysis approaches that use machine learning. In particular, priors can be useful in cases where images are corrupted and contain artefacts due to limitations in image acquisition. The highly constrained nature of anatomical objects can be well captured with learning based techniques. However, in most recent and promising techniques such as CNN based segmentation it is not obvious how to incorporate such prior knowledge. State-of-the-art methods operate as pixel-wise classifiers where the training objectives do not incorporate the structure and inter-dependencies of the output. To overcome this limitation, we show a generic training strategy that incorporates anatomical prior knowledge into CNNs through a new regularisation model, which is trained end-to-end [5]. The new framework encourages models to follow the global anatomical properties of the underlying anatomy (e.g. shape, label structure) via learned non-linear representations of the shape. We show that the this approach can be easily adapted to different analysis tasks (e.g. image enhancement, segmentation) and improve the prediction accuracy of the state-of-the-art models. The applicability of this approach is shown on multi-modal cardiac datasets and public benchmarks.


We would like to thank the members of the BioMedIA group, Department of Computing, Imperial College London for their help.


[1] J. Schlemper, J. Caballero, J. V. Hajnal, A. Price and D. Rueckert. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Transactions on Medical Imaging, in press, 2018.

[2] G. Tarroni, O. Oktay, W. Bai, A. Schuh, H. Suzuki, J. Passerat-Palmbach, B. Glocker, P. M. Matthews and D. Rueckert. Learning-Based Quality Control for Cardiac MR Images.

[3] O. Oktay, W. Bai, M. C. H. Lee, R. Guerrero, K. Kamnitsas, J. Caballero, A. de Marvao, S. A. Cook, D. P. O'Regan and D. Rueckert. Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks. Medical Image Computing and Computer Assisted Interventions (MICCAI): 246-254, 2016.

[4] W. Bai, M. Sinclair, G. Tarroni, O. Oktay, M. Rajchl, G. Vaillant, A. M. Lee, N. Aung, E. Lukaschuk, M. M. Sanghvi, F. Zemrak, K. Fung, J. M. Paiva, V. Carapella, Y. J. Kim, H. Suzuki, B. Kainz, P. M. Matthews, S. E. Petersen, S. K. Piechnik, S. Neubauer, B. Glocker and D. Rueckert. Human-level CMR image analysis with deep fully convolutional networks.

[5] O. Oktay, E. Ferrante, K. Kamnitsas, M. Heinrich, W. Bai, J. Caballero, S. Cook, A. de Marvao, T. Dawes, D. O'Regan, B. Kainz, B. Glocker and D. Rueckert. Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation. IEEE Transactions on Medical Imaging, in press, 2018.

Proc. Intl. Soc. Mag. Reson. Med. 26 (2018)