3482

PROJECTION GAN: Highly accelerated projection reconstruction using generative adversarial neural network
Tetiana Dadakova1, Jian Wu1, Hyun-Kyung Chung1, Brian Anhalt1, Dmitry Tkach1, Alexander Graff1, Natalie Marie Schenker-Ahmed1, David Karow1, and Christine Leon Swisher1
1Human Longevity, Inc., San Diego, CA, United States

Synopsis

Many clinical MRI applications in chest and abdomen require low sensitivity to motion. In addition, high acquisition speed is necessary for imaging in non-cooperative patients or those unable to perform breath holds. These applications would benefit from the highly accelerated radial acquisition. Deep learning has been shown to provide good results for image reconstruction from highly under-sampled k-space data. Here we introduce a Projection GAN - a generative adversarial neural network, which is trained to reconstruct highly accelerated MR images from uniformly rotated projections. Our results show that even with aggressive under-sampling the reconstruction has great overall performance.

Introduction

Cardiac, lung, breast, dynamic contrast enhanced abdominal exams, in particular, suffer from motion artifacts (respiratory and cardiac) and have an increased need for accelerated acquisition. To address this need, we developed an accelerated radial reconstruction approach.
Compared to Cartesian k-space trajectories, the major benefit of radial sampling1 is its tolerance to motion2, as the reconstructed image does not have structured ghosting (phase mis-mapping) from moving organs. In addition, the redundancy in the sampling of the center of k-space can be exploited to track motion using DC-navigator and correct it by binning the data into corresponding motion states3 (motion-resolved reconstruction). However, radial acquisitions require additional lines of k-space compared to Cartesian to satisfy the Nyquist sampling requirement (for the matrix size of M, radial acquisition requires M*π/2 spokes). Also, in the case of motion-resolved reconstruction, each motion state is reconstructed from only a fraction of the data. These are examples of when a fast and robust reconstruction method for the under-sampled radial data would be necessary.
Reconstruction from under-sampled k-space data using deep learning is substantially faster (sub-second) than iterative methods, because it allows to shift the time-consuming computations into training phase4. Commonly, the deep learning reconstruction from under-sampled data is focused on removing artifacts from images reconstructed from zero-filled k-space by Fourier transform. However, recent work showed a new approach5, where deep learning was used to reconstruct 3D computed tomography (CT) images of the lung from two orthogonal X-ray projection images.
Adapting this approach to MRI, we can leverage the property that the zero-frequency line or partition in k-space corresponds to the mean signal in image space in the corresponding direction for the 2D or 3D image, respectively (Figure 1 shows an example for 2D case). Such signal projection can be acquired at any angle; therefore, MRI allows us to use many more projections than CT, and as many projections necessary to achieve a balance between robust and accurate reconstruction of the image and acquisition time.
Here we present a novel Projection GAN – a generative adversarial neural network, which is trained to reconstruct MR images from a sub-Nyquist number of uniformly rotated projections.

Methods

A generative adversarial network DAGAN6 (Figure 2) was trained with an adversarial loss coupled with a content loss consisting of pixel-wise mean square error for both image and frequency domains (MSEs) and perceptual loss defined by pre-trained VGG network7. The discriminator had a convolutional neural network architecture and acted as a classifier. The generator network had a U-Net architecture and was used for inference with the acquired projections as an input.
A database8 of T1-weighted brain images was used to generate training, validation and test data (n=15400, n=100 and n=100, respectively). The projections were generated at multiple uniformly spaced angles by summing up all the pixel values along corresponding directions.
The input for the generator consisted of magnitude image of size 256x256 and the corresponding Np projections, of size 256x256xNp. The image, reconstructed by the generator was fed into a discriminator network, which decided whether image is real (from fully sampled k-space) or fake (from projections). Training of both generator and discriminator was performed adversarially until the discriminator could not distinguish image reconstructed by the generator from a real fully sampled image. The following evaluation metrics were used: normalized root-mean-square error (NMSE), peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

Results

Figure 3 shows a representative example image showing similar performance of fine detail in the fully sampled k-space (Figure 3A) and corresponding reconstructed image from only Np = 16 projections (corresponding to 16 times acceleration compared to Cartesian acquisition and 25 times compared to fully sampled radial; Figure 3B).
The following average quantitative results were achieved for the test data: NMSE = 0.20, PSNR = 28.06 dB and SSIM = 0.90.

Discussion and Conclusion

The results show that even with aggressive under-sampling, the reconstructed image has impressive performance in its similarity to the ground truth image. The difference in small details of the image would not be essential for some applications, for example functional lung imaging where the regional signal information and high temporal resolution are more important than accurate representation of small anatomical details. Increasing the number of projections would improve the accuracy of small details in the image.
Clinically, the proposed reconstruction method would be useful for applications, which require low sensitivity to motion and high acquisition speed. The examples of such applications include chest and abdominal MRI (pulmonary, cardiac, breast imaging, as well as MRI of pancreas or liver) in patients, who are unable to hold still or to perform breath holds (e.g. unconscious or pediatric patients).
Future research will include extending the Projection GAN to the 3D imaging, where each center partition will correspond to the 2D projection. As any k-space trajectory could be used within each kz=0 partition projection, an important application that could be explored is 3D ultra-short echo time (UTE) functional imaging of lungs9. In this case, under-sampling would allow for high temporal resolution, and the acquisition could be modified to acquire half projections starting in the center of k-space to achieve the UTE.

Acknowledgements

No acknowledgement found.

References

  1. Lauterbur PC. Image formation by induced local interactions: examples employing nuclear magnetic resonance. Nature 1973; 242:190-1
  2. Glover GH and Pauly JM. Projection reconstruction techniques for reduction of motion effects in MRI. Magn Reson Med. 1992;28(2):275-89
  3. Feng L, Axel L, Chandarana H, Block KT, Sodickson DK and Otazo R. XD-GRASP: Golden-angle radial MRI with reconstruction of extra motion-state dimensions using compressed sensing. Magn Reson Med. 2016;75(2):775-88
  4. Quan TM, Nguyen-Duc T and Jeong WK. Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging. 2018;37(6):1488-1497
  5. Ying, X., Guo, H., Ma, K., Wu, J., Weng, Z., Zheng, Y. X2CT-GAN: reconstructing CT from biplanar X-rays with generative adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019:10619–10628
  6. Yang G, Yu S, Dong H, Slabaugh G, Dragotti PL, Ye X, Liu F, Arridge S, Keegan J, Guo Y, Firmin D, Keegan J, Slabaugh G, Arridge S, Ye X, Guo Y, Yu S, Liu F, Firmin D, Dragotti PL, Yang G and Dong H. DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction. IEEE Trans Med Imaging. 2018 Jun;37(6):1310-1321
  7. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. Proc ICLR. 2015:1-14
  8. Sudlow C, Gallacher J, Allen N, Beral V, Burton P, Danesh J, Downey P, Elliott P, Green J, Landray M, Liu B, Matthews P, Ong G, Pell J, Silman A, Young A, Sprosen T, Peakman T, Collins R. UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 2015;12(3):e1001779
  9. Voskrebenzev A, Gutberlet M, Klimeš F, Kaireit TF, Schönfeld C, Rotärmel A, Wacker F, Vogel-Claussen J. Feasibility of quantitative regional ventilation and perfusion mapping with phase-resolved functional lung (PREFUL) MRI in healthy volunteers and COPD, CTEPH, and CF patients. Magn Reson Med. 2018 Apr;79(4):2306-2314. doi: 10.1002/mrm.26893. Epub 2017 Aug 30

Figures

Figure 1 Fully sampled k-space (top row), as well as the center line of k-space (bottom row) and the corresponding images. The right column shows the mean signal over the vertical direction of the image. Note that the curves are identical for both cases: fully sampled image and image from the only center k-space line

Figure 2 Schematic describing (A) how the model was trained and (B) how the generator was used during reconstruction of the under-sampled projection data

Figure 3 (A) Fully sampled image (ground truth) and (B) image reconstructed from Np = 16 projections, which corresponds to 16x acceleration compared to fully sampled Cartesian trajectory and 25x acceleration compared to fully sampled radial trajectory

Proc. Intl. Soc. Mag. Reson. Med. 28 (2020)
3482