2028

A Single-Image Super-Resolution Method for Late Gadolinium Enhancement CMR
Jin Zhu1, Guang Yang2,3, Tom Wong2,3, Raad Mohiaddin 2,3, David Firmin2,3, Jennifer Keegan 2,3, and Pietro Lio1

1Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom, 2National Heart and Lung Institute, Imperial College London, London, United Kingdom, 3Cardiovascular Research Centre, Royal Brompton Hospital, London, United Kingdom

Synopsis

3D late gadolinium enhanced (LGE) CMR is a useful imaging modality for detecting scar tissue in patients with atrial fibrillation. In order to visualize the thin-walled left atrium and scar tissue, high spatial resolution and contiguous coverage are required. However, increased spatial resolution requires markedly prolonged scanning time. In this paper, we propose a ROI focused single-image super-resolution (SISR) method based on the generative adversarial networks architecture to increase the apparent spatial resolution of 3D LGE data without increasing scan time. The proposed SISR method can boost the spatial resolution of the LGE CMR images while maintaining the perceptual quality.

Introduction:

Late gadolinium enhanced (LGE) CMR in patients with atrial fibrillation (AF) can show native and post-ablation treatment scar within the left atrium (LA) [1]. Although many studies have shown promising results, there are still on-going concerns regarding the accuracy of identifying scars using this technique [2], [3]. This is partially because the LA wall is very thin and the limited spatial resolution of the LGE CMR can lower its diagnostic value. The acquisition durations for 3D LGE imaging are long (typically 5–10 minutes) and increasing the acquired spatial resolution is not usually a practical option. Instead, super-resolution (SR) based post-processing has the potential to provide an inexpensive yet effective way to increase the spatial resolution of the LGE data. In this study, an advanced deep learning based single-image SR (SISR) method is proposed for LGE CMR.

Methods:

Our SISR method is based on a generative adversarial network (GAN) [4], which has shown promising results for natural images, e.g., SRGAN [5]. However, the vanilla GAN used in SR models suffers from the unstable training, collapsed mode and difficulties in hyper-parameters tuning. In our method, we incorporate Wasserstein GAN (WGAN) with gradient penalty [6], [7], which uses the Wasserstein-1 distance instead of the non-continuous divergence, to mitigate these problems.

In practice, the GAN based SR models are hard to train because distributions of the source and target domain images are in a high-dimensional space and their overlapping is rare, causing vanished gradient. In our SISR method, we tackle this problem by: (1) proposing a ROI focused SISR to enforce our network to super-resolve only the ROI (in our case the bounding box around the left atrium), before we apply the deep residual SR neural network (SRResNet [5]), and (2) proposing a multi-scale (MS) architecture to stabilize the optimization of the SR and to cope with higher magnifying factors (e.g., X4 magnification), instead of using all of the loss functions with a single scale. Figure 1 details our proposed SISR framework (named ROI-MS-WGAN).

With ethical approval, CMR data were collected from 20 patients presenting with longstanding persistent AF on a Siemens Magnetom Avanto 1.5T scanner. Transverse navigator-gated 3D LGE CMR [8], [9] was performed using an inversion prepared segmented gradient echo sequence ((1.4–1.5)×(1.4–1.5)×4mm3 reconstructed into (0.7–0.75)×(0.7–0.75)×2mm3) 15 minutes after gadolinium administration (Gadovist—gadobutrol, 0.1mmol/kg body weight) [10]. A dynamic inversion time (TI) was designed to null the signal from normal myocardium [11]. The 3D LGE data were acquired during free-breathing using CLAWS respiratory motion control to increase respiratory efficiency [2]. Navigator artefact was reduced by introducing a navigator-restore delay of 100 ms [8]. For the purposes of this study, the acquired data were considered to be the high resolution (HR) ground truth (GT) and were downsampled (by a factor of 4) to produce low resolution images (LR) for input to our ROI-MS-WGAN. Two-dimensional slices (N=743) from 20 3D LGE datasets were randomly divided into training/cross-validation (N=615) and independent testing (N=128) datasets. We compared the performance of our ROI-MS-WGAN method with high-resolution ground truth and with results obtained by bilinear interpolation, SRResNet and SRGAN with X4 magnification using peak SNR (dB) (PSNR), Structural SIMilarity (SSIM) index [13] and mean opinion score (MOS) [14].

Results:

Example results are shown in Figures 2 and 3. Boxplots summarizing the PSNR and SSIM results for all studies are shown in Figure 4. Wilcoxon rank-sum test showed that both SRResNet and ROI-MS-WGAN achieved superior PSNR compared to bilinear interpolation and SRGAN. Although ROI-MS-WGAN has obtained slightly lower SSIM, the mean opinion score (MOS) based evaluation showed that our ROI-MS-WGAN achieved the best performance (Figure 5).

Discussion:

Higher spatial resolution of the LGE CMR images is important for subsequent diagnosis, quantification and visualization of LA scar tissue. Our ROI-MS-WGAN method is ROI focused with multiscale architecture and the benefits are threefold: (a) the training cost can be reduced significantly by a huge reduction of network parameters; (b) SR with better perceptual qualities can be achieved by replacing the estimation of transformation in a lower dimensional space; and (c) possible artifacts can be suppressed by excluding features extracted from irrelevant regions outside the ROI. In addition, the multiscale strategy can further stabilize the training to achieve higher magnifying factors. Compared to high-resolution ground truth, our method has achieved SR results with improved fidelity and perceptual quality.

Conclusion:

In this study, we propose a ROI focused multiscale SISR method to improve the apparent spatial resolution of LGE CMR images. Compared to existing approaches, our ROI-MS-WGAN method can be applied to achieve SR with high magnifying factors while preserving the perceptual quality. This could improve visualization and quantification of the LA scar tissue with better diagnostic yield.

Acknowledgements

First author’s PhD research is partially funded by China Scholarship Council (Grant No.201708060173). This study was also funded by the British Heart Foundation Project Grant (Project Number: PG/16/78/32402). Jennifer Keegan and Pietro Lio are co-last authors.

References

[1] D. C. Peters et al., “Detection of pulmonary vein and left atrial scar after catheter ablation with three-dimensional navigator-gated delayed enhancement MR imaging: initial experience.,” Radiology, vol. 243, no. 3, pp. 690–695, 2007.

[2] C. McGann et al., “Atrial fibrillation ablation outcome is predicted by left atrial remodeling on MRI,” Circ. Arrhythmia Electrophysiol., vol. 7, no. 1, pp. 23–30, 2014.

[3] J. L. Harrison et al., “Repeat Left Atrial Catheter Ablation: Cardiac Magnetic Resonance Prediction of Endocardial Voltage and Gaps in Ablation Lesion Sets,” Circ. Arrhythmia Electrophysiol., vol. 8, no. 2, pp. 270–278, Apr. 2015.

[4] I. Goodfellow, J. Pouget-Abadie, and M. Mirza, “Generative Adversarial Networks,” arXiv Prepr. arXiv …, pp. 1–9, 2014.

[5] C. Ledig et al., “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” 2016.

[6] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein GAN,” Jan. 2017.

[7] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, “Improved Training of Wasserstein GANs,” in Advances in Neural Information Processing Systems, 2017, pp. 5767–5777.

[8] D. C. Peters et al., “Recurrence of Atrial Fibrillation Correlates With the Extent of Post-Procedural Late Gadolinium Enhancement. A Pilot Study,” JACC Cardiovasc. Imaging, vol. 2, no. 3, pp. 308–316, 2009.

[9] R. S. Oakes et al., “Detection and quantification of left atrial structural remodeling with delayed-enhancement magnetic resonance imaging in patients with atrial fibrillation,” Circulation, vol. 119, no. 13, pp. 1758–1767, 2009.

[10] J. Keegan, P. Jhooti, S. V. Babu-Narayan, P. Drivas, S. Ernst, and D. N. Firmin, “Improved respiratory efficiency of 3D late gadolinium enhancement imaging using the continuously adaptive windowing strategy (CLAWS),” Magn. Reson. Med., vol. 71, no. 3, pp. 1064–1074, 2014.

[11] J. Keegan, P. D. Gatehouse, S. Haldar, R. Wage, S. V Babu-Narayan, and D. N. Firmin, “Dynamic inversion time for improved 3D late gadolinium enhancement imaging in patients with atrial fibrillation.,” Magn. Reson. Med., vol. 73, no. 2, pp. 646–54, Feb. 2015.

[12] J. Keegan, P. Drivas, and D. N. Firmin, “Navigator artifact reduction in three-dimensional late gadolinium enhancement imaging of the atria,” Magn. Reson. Med., vol. 785, pp. 779–785, 2013.

[13] C.-Y. Yang, C. Ma, and M. Yang, “Single-Image Super-Resolution: A Benchmark,” in Computer Vision – ECCV 2014, 2014, pp. 372–386.

14] M. Seitzer et al., “Adversarial and Perceptual Refinement for Compressed Sensing MRI Reconstruction,” in Medical Image Computing and Computer Assisted Intervention (MICCAI 2018), 2018.

[15] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” Sep. 2014.

[16] G. Yang et al., “Fully Automatic Segmentation and Objective Assessment of Atrial Scars for Longstanding Persistent Atrial Fibrillation Patients Using Late Gadolinium-Enhanced MRI,” Med. Phys., vol. 45, no. 4, pp. 1562–1576, 2018.

Figures

Figure 1: The framework of our proposed ROI-MS-WGAN, containing a ROI detection neural network (ROI-NN), a multi-scale SISR neural network (MS-NN), and a discriminator D. The ROI-NN and MS-NN aim to achieve perceptually realistic SR output, and the D aims to distinguish them from the ground truth high-resolution images. All networks are trained together with pixel-wise mean-square-errors, GAN loss and a VGG [15] base perceptual loss, but only the ROI-NN and MS-NN are used for inference. (LR=low-resolution, HR=high-resolution, GT=ground truth, notice that the HR GT images were X4 downsampled to produce LR images for training our ROI-MS-WGAN).

Figure 2: Comparison of SR methods (X4 magnification). The ground truth (GT) segmentation of the LA is shown in yellow, and the detected ROI is framed in red. PSNR (in dB)/SSIM and MOS are also displayed. (The MOS and remarks were based on blinded scoring of the images by a CMR image processing physicist: S—Over-smooth; A—Motion and other kind of artefacts; U—Unrealistic textures; N—Too noisy or low SNR).

Figure 3: Comparison of SR methods (X4 magnification). SR results produced by different methods and high-resolution ground truth (GT, top row). Differences between the SR results and the GT (middle row). Calculated differences overlaid on the SR results (bottom row). Notice that the difference images were normalized and scaled into a range of [-0.3, 0.3] for visualization purpose. The color bar of the calculated differences is shown on the right hand side. (The MOS and remarks were displayed as the same as Figure 2: S—Over-smooth; A—Motion and other kind of artefacts; U—Unrealistic textures; N—Too noisy or low SNR).

Figure 4: Comparison of SR methods (X4 magnification). Boxplots comparing the calculated PSNR (a) and SSIM (b) of the ROI-MS-WGAN method with the bilinear interpolation, SRResNet and SRGAN methods. Wilcoxon rank-sum test showed that all the methods performed significantly different. It is of note that as previous studies [5], [14] our work also demonstrated that conventional quantitative metrics (PSNR/SSIM) may not be the best measure of SR results alone, but need to be considered with qualitative visual inspection. Wilcoxon rank-sum test showed significant statistical differences between the tested methods, e.g., ROI-MS-WGAN vs. bilinear interpolation (indicated by the *).

Figure 5: Perceptual quality measured by the MOS metric [14]. Thirty slices were randomly selected from the validation dataset for the MOS evaluation. For each slice, there was 1 ground truth image and 4 SR results corresponding to the 4 SR methods (150 images in total). A CMR image processing physicist (>3 years’ experience in LGE CMR) performed blinded scoring of the image quality of these 150 images on a Likert-type scale—1(Poor), 2(Fair), 3(Good), and 4(Excellent) [14], [16]. Wilcoxon rank-sum test showed no significant difference between our ROI-MS-WGAN and the ground truth. (S,A,U and N as defined in Figure 2).

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)
2028