4769

SANTIS: Sampling-Augmented Neural neTwork with Incoherent Structure for efficient and robust MR image reconstruction
Fang Liu1, Lihua Chen2,3, Richard Kijowski1, and Li Feng4

1Radiology, University of Wisconsin-Madison, Madison, WI, United States, 2Radiology, Southwest Hospital, Chongqing, China, 3Radiology, PLA 101st Hospital, Wuxi, China, 4Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY, United States

Synopsis

The purpose of this work was to develop and evaluate a new deep-learning based image reconstruction framework, termed as Sampling-Augmented Neural neTwork with Incoherent Structure (SANTIS) for MR image reconstruction. Our approach combines efficient end-to-end CNN mapping with k-space consistency using the concept of cyclic loss to enforce data fidelity. Adversarial training is implemented for maintaining high quality perceptional image structure and incoherent k-space sampling is used to improve reconstruction accuracy and robustness. The performance of SANTIS was demonstrated for reconstructing vast undersampled Cartesian knee images and golden-angle radial liver images. Our study demonstrated that the proposed SANTIS framework represents a promising approach for efficient and robust MR image reconstruction at vast acceleration rate.

Introduction

Deep learning has recently demonstrated great potentials for efficient image reconstruction from undersampled k-space measurements(1–5). For most of existing studies, a fixed undersampling pattern is typically employed for the network training, and the trained neural network is then applied to reconstruct new images acquired at the same sampling pattern. While this training strategy maintains good reconstruction accuracy, the robustness of the trained network could be degraded when an image to be reconstructed is acquired with an undersampling pattern different from the one used for the training(6). In this work, we hypothesized that deep learning-based reconstruction can actually benefit from extensively varying undersampling patterns during the training process, and this can potentially improve both the robustness and reconstruction performance of the trained neural network. Such a training strategy enforces sampling or k-space trajectory augmentation, representing a great candidate for applications that employ non-repeating k-space acquisitions, such as golden-angle radial MRI(7). This new framework, termed as Sampling-Augmented Neural neTwork with Incoherent Structure (SANTIS), was evaluated for retrospectively undersampled knee and liver MRI in this abstract.

Methods

(a) SANTIS Implementation (Fig. 1): Our network uses the Data-Cycle-Consistent GAN structure proposed in our previous study(8), which aims to solve a training objective consisting of three loss components, including i) an efficient end-to-end CNN mapping loss, ii) a data fidelity loss allowing CNN output image consistent with k-space measurements, and iii) an adversarial loss enforcing high perceptional quality of reconstructed images. Such structure is tailored from the general cycle-consistent GAN (CycleGAN) framework (9) and optimized for MR image reconstruction (8). A combination of U-net (for CNN mapping) and PatchGAN (for adversarial process) was selected for constructing the network, and was trained on an Nvidia GeForce GTX 1080Ti card using an adaptive gradient descent (ADAM) algorithm.

(b) Sampling Augmentation (Fig. 2): In SANTIS framework, the undersampling pattern used for training keeps varying at each network training iteration. As a result, the network has the capability to learn extensive undersampling structures for improved robustness. In other words, the trained network tries to learn various artifact structures and thus better generalizes towards new artifacts that might occur during the inference process. Moreover, the incoherence introduced by the varying sampling patterns can also potentially improve the reconstruction performance.

(c) Evaluation: The network was trained on 25 fully sampled Cartesian multislice knee data (sagittal proton-density weighted fast-spin-echo) and 50 post-contrast 3D golden-angle radial liver data. The trained network was then tested in 5 additional knee data and 8 liver data acquired using the same protocol. The undersampling mask used for testing was also randomly generated from the sampling pattern pool. For comparison, standard training using a fixed undersampling mask was also performed, and this network was used to reconstruct undersampled image with the same training mask (Matched) and with a different mask (Unmatched).

Results

Fig. 3 showed exemplary results for Cartesian knee imaging at R=3. SANTIS achieved better image quality than CNN-Fix, which used the same sampling pattern for training a network. CNN-Fix failed to reconstruct images reliably when the image to be reconstructed has a different undersampling pattern than the one used during training. In liver datasets (Fig. 4), SANTIS also achieved better image quality (green arrows) than CNN-Fix when the image to be reconstructed has changing undersampling patterns. SANTIS also outperformed conventional compressed sensing reconstruction (yellow arrows) with a sparsity constraint (wavelet transform). In Fig. 5, as shown with the green arrows, SANTIS outperformed both CNN-Fix-Match (consistent undersampling pattern during both training and inference) and CNN-Fix-Unmatch (different undersampling pattern between training and inference).However, we noted that CNN-Fix-Unmatch is better in liver imaging than knee imaging, potentially due to the more incoherent sampling behavior from the golden-angle radial sampling.

Discussion

The SANTIS framework enforces sampling or k-space trajectory augmentation with extensive variation of sampling patterns during the training process. Such a training strategy has the potential to improve the robustness of the network for better reconstruction performance. We have shown the performance of SANTIS in accelerated Cartesian and golden-angle radial imaging for knee and liver imaging, respectively. With a combination of efficient end-to-end CNN mapping, data fidelity reinforcement, adversarial training and incorporation of random training strategy, SANTIS allows rapid image reconstruction with high quality and good robustness at vast acceleration rate.

Acknowledgements

The authors thank the clinicians at the UW Hospital in Madison, USA for acquiring and sharing knee data with institutional IRB approval. The authors also thank the clinicians at the Southwest Hospital in Chongqing, China for acquiring and sharing the golden-angle radial liver data with institutional IRB approval.

References

1. Hammernik K, Klatzer T, Kobler E, Recht MP, Sodickson DK, Pock T, Knoll F. Learning a Variational Network for Reconstruction of Accelerated MRI Data. Magn. Reson. Med. [Internet] 2017;79:3055–3071. doi: 10.1002/mrm.26977.

2. Mardani M, Gong E, Cheng JY, Vasanawala SS, Zaharchuk G, Xing L, Pauly JM. Deep Generative Adversarial Neural Networks for Compressive Sensing (GANCS) MRI. IEEE Trans. Med. Imaging [Internet] 2018:1–1. doi: 10.1109/TMI.2018.2858752.

3. Wang S, Su Z, Ying L, Peng X, Zhu S, Liang F, Feng D, Liang D, Technologies I. ACCELERATING MAGNETIC RESONANCE IMAGING VIA DEEP LEARNING. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE; 2016. pp. 514–517. doi: 10.1109/ISBI.2016.7493320.

4. Schlemper J, Caballero J, Hajnal J V., Price A, Rueckert D. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans. Med. Imaging [Internet] 2017:1–1. doi: 10.1007/978-3-319-59050-9_51.

5. Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature [Internet] 2018;555:487–492. doi: 10.1038/nature25988.

6. Knoll F, Hammernik K, Kobler E, Pock T, Recht MP, Sodickson DK. Assessment of the generalization of learned image reconstruction and the potential for transfer learning. Magn. Reson. Med. [Internet] 2018. doi: 10.1002/mrm.27355.

7. Feng L, Grimm R, Block KT, Chandarana H, Kim S, Xu J, Axel L, Sodickson DK, Otazo R. Golden-angle radial sparse parallel MRI: Combination of compressed sensing, parallel imaging, and golden-angle radial sampling for fast and flexible dynamic volumetric MRI. Magn. Reson. Med. [Internet] 2014;72:707–717. doi: 10.1002/mrm.24980.

8. Liu F, Samsonov A. Data-Cycle-Consistent Adversarial Networks for High-Quality Reconstruction of Undersampled MRI Data. In: the ISMRM Machine Learning Workshop. ; 2018.

9. Zhu J-Y, Park T, Isola P, Efros AA. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In: 2017 IEEE International Conference on Computer Vision (ICCV). Vol. 2017–Octob. IEEE; 2017. pp. 2242–2251. doi: 10.1109/ICCV.2017.244.

Figures

Figure 1: A Data-Cycle-Consistent GAN algorithm, with a combination of residual learning U-Net (for CNN mapping) and PatchGAN (for adversarial process). There are three major components for a high efficient end-to-end CNN mapping with residual learning design, a data fidelity loss enforcing CNN output images to be consistent with k-space measurements and an adversarial loss ensuring high perceptional quality of reconstructed images.

Figure 2: The undersampling pattern used for training keeps varying in each network training iteration. The trained network tries to learn various artifact structures and thus better generalizes towards removing new artifacts that might occur during the inference process. The incoherence introduced by varying the sampling patterns can also potentially improve reconstruction performance.

Figure 3: SANTIS achieved better image quality than CNN-Fix, which used the same sampling pattern for training a network. CNN-Fix failed to reconstruct images reliably when the image to be reconstructed has a different undersampling pattern than the one used during training.

Figure 4: SANTIS also achieved better image quality (green arrows) than CNN-Fix when the image to be reconstructed has changing undersampling patterns. SANTIS also outperformed conventional compressed sensing reconstruction (yellow arrows) with a sparsity constraint (wavelet transform).

Figure 5: SANTIS outperformed both CNN-Fix-Match (consistent undersampling pattern during both training and inference) and CNN-Fix-Unmatch (different undersampling pattern between training and inference). However, CNN-Fix-Unmatch is better in liver imaging than knee imaging, potentially due to the more incoherent sampling behavior from the golden-angle radial sampling.

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)
4769