DL-POCS: Deep Learning Augmented POCS Reconstruction for Vastly Undersampled MR Data
Fang Liu1, Julia Velikina1, Richard Kijowski1, and Alexey Samsonov1

1Department of Radiology, University of Wisconsin-Madison, Madison, WI, United States


We introduced a novel reconstruction framework by combining deep learning (DL) neural network with the Projections Onto Convex Sets (POCS) algorithm, termed DL-POCS. The image restoration from undersampled images was first performed by a convolutional encoder-decoder network. Then the output from deep learning was used as initialization and extra constraints were imposed to promote the POCS reconstruction. We evaluated this approach on vastly undersampled knee MR data and found that this combined approach is superior to each of individual components alone. Our study suggests that deep learning regularized image reconstruction will have a substantial impact on data-driven accelerated MR imaging.


The Projections Onto Convex Sets (POCS) algorithm presents an efficient way to utilize a wide range of prior information in image restoration problems (1) such as restoration of partial k-space (2) and motion artifact correction(3). In our previous work, we adapted the POCS formalism for parallel MRI data reconstruction, where we showed utilization of prior information within the POCS framework is an efficient way to mitigate aliasing and noise amplification, thereby affording higher acceleration factors and improving image quality (4). In recent years, there have been many efforts to develop deep learning (DL) based methods for restoring undersampled image data for CT and MRI. These efforts lead to sophisticated, computationally intensive reconstruction pipelines where development of advanced deep learning networks with better performance is primary focus. In this study, we explore an alternative approach, termed DL-POCS, for the use of DL machinery for image reconstruction, in which DL-based pre-processing of undersampled data is applied only once to generate prior information for regularization and additional constraints of POCS framework. Evaluation of such framework was performed for reconstruction of highly undersampled knee MR images.


Deep Learning Network: A convolutional encoder-decoder (CED) network was designed to restore MR images from undersampled k-space data (Figure 1). The encoder consists of a set of convolution layers followed by batch normalization (BN) and rectified-linear unit (ReLU) activation. The decoder is a mirror network with the same structure with the convolution layers replaced by deconvolution layers. Symmetric shortcut connections between encoder and decoder layers were added by following the fully pre-activation residual network strategy to forward transfer image features. POCS Framework: The POCS processing pipeline follows POCSENSE (4) framework (Figure 2). Instead of using zero-filled image as POCSENSE initial input, we initialize it by prior image that is largely restored from learned latent aliases and noise patterns during the CED training process. Additionally, the CED network output can provide additional constraints such as object support in the POCS pipeline. Evaluation: The evaluation was performed on 10 retrospectively undersampled knee images from routine clinical scans. Images were acquired on 3T scanner (MR750, GE Healthcare, Waukesha, USA) using intermediate T2-weighted coronal fast spin-echo (TR/TE=2125/20ms, 420×448 matrix, 32 slices). We simulated 2D Cartesian parallel imaging sampling pattern with undersampling factors R=3 and 10. The central 8% of k-space data was fully sampled and used for calculating coil sensitivity. Eight knee images were used for training and image augmentation through 2D translation, rotation and shearing was used to generate three repeats. The network was trained using mean squared error as image loss and an adaptive gradient-based optimization algorithm (ADAM) with an initial learning rate of 0.0001 for a total of 100 epochs.


The total training phase took approximately 36 hours (computing hardware included an Intel Xeon W3520 quad-core CPU, 32 GB DDR3 RAM, and two Nvidia GeForce GTX 1080 Ti graphic cards with 7168 cores and 22GB GDDR5 RAM.). Generating corrected images for one subject took approximately 30 seconds for deep learning inference. Figure 3 demonstrates images reconstructed with several methods for R=3. While the CED network and POCS applied independently substantially reduced image artifacts, their combination within DL-POCS framework provided the best performance with almost complete image restoration. The normalized root mean squared error (nRMSE) for Zero Filling, DL, POCS and DL-POCS were 0.073, 0.045, 0.0023 and 0.0015, respectively. Figure 4 demonstrates the performance for R=10. At this extreme undersampling level, both DL and POCS retain visible image artifacts as indicated by the arrows. The DL-POCS significantly improves reconstruction by efficiently suppressing these artifacts. The nRMSE for Zero Filling, DL, POCS and DL-POCS were 0.095, 0.068, 0.078 and 0.056, respectively.


We have proposed a novel image reconstruction framework by combining deep learning convolutional neural network with POCS reconstruction for vastly undersampled MR data. We have demonstrated that this combined reconstruction strategy offers superior performance to the ones afforded by individual components. While most deep learning-based studies to date focus on development and implementation of better networks, our approach introduces a novel insight on integrating deep learning into traditional well-studied reconstruction frameworks. The synergy of group wise data-driven nature of deep learning and case specific data consistency of POCS amplify the reconstruction performance. Our study suggests that deep learning-regularized image reconstruction, such as DL-POCS, can have a substantial impact on data-driven accelerated MRI.


No acknowledgement found.


1. Youla DC, Webb H: Image Restoration by the Method of Convex Projections: Part 1 Theory. IEEE Trans Med Imaging 1982; 1:81–94.

2. ZP L, FE B, RT C, EM H, PC L, MR S: Constrained reconstruction methods in MR imaging. Rev Magn Reson Med 1992; 4:67–185.

3. Samsonov AA, Velikina J, Jung Y, Kholmovski EG, Johnson CR, Block WF: Pocs-enhanced correction of motion artifacts in parallel MRI. Magn Reson Med 2010; 63:1104–1110.

4. Samsonov AA, Kholmovski EG, Parker DL, Johnson CR: POCSENSE: POCS-based reconstruction for sensitivity encoded magnetic resonance imaging. Magn Reson Med 2004; 52:1397–1406.


Figure 1. Schematic illustration of convolutional encoder-decoder (CED) to restore undersampled MR images (represented as separate real and imaginary channels). This network consists of a combined encoder network and decoder network with multiple symmetrical shortcut connection (SC) between layers. Note there is no pooling and unpooling layers so this network works for any image size. The insertion of SC follows the strategy of full pre-activation deep residual network.

Figure 2. Schematic illustration of deep learning POCS pipeline. Deep learning provides initial images for the input of POCS and constrains the POCS reconstruction by extra object support.

Figure 3. Reconstruction performance for several methods at R=3. The DL method using CED network achieved a good image restoration with substantially reduced image artifacts. The POCS framework achieved even better performance. However, combined with CED output as initial guess, the DL-POCS has the best performance with almost complete image restoration.

Figure 4. Reconstruction performance for several methods at vastly undersampled data R=10. Despite largely recovered from the input aliased image, both DL and POCS remain subtle noticeable image artifacts as pointed by the arrows. The DL-POCS provides better reconstructed images with those artifacts substantially suppressed.

Proc. Intl. Soc. Mag. Reson. Med. 26 (2018)