0778
A ROI Focused Multi-Scale Super-Resolution Method for the Diffusion Tensor Cardiac Magnetic Resonance1Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom, 2National Heart and Lung Institute, Imperial College London, London, United Kingdom, 3Cardiovascular Research Centre, Royal Brompton Hospital, London, United Kingdom
Synopsis
Diffusion Tensor Cardiovascular Magnetic Resonance (DT-CMR) is a promising contrast-free and non-invasive technique to characterize the tissue integrity and microstructure of the myocardium. However, the complex acquisition protocol results in prolonged scan times and often results in images with low spatial resolution and relatively poor signal-to-noise ratio. Here a novel ROI focused multi-scale super-resolution approach is proposed to improve the apparent spatial resolution of in vivo DT-CMR. Based on simulation studies, our proposed method can achieve increases in apparent spatial resolution by a factor of 4 with preserved image quality and no obvious degradation in the derived DT-CMR parameters.
Introduction:
Diffusion Tensor Cardiovascular MR (DT-CMR) is an emerging contrast-free non-invasive technique providing rich information on myocardial microstructure [1,2]. Despite great efforts to drive DT-CMR towards a clinical utility, it is currently still limited by the low spatial resolution [3,4]. Here we use super-resolution (SR) based post-processing to provide a low-cost but effective way to retrospectively boost the spatial resolution of DT-CMR data. In this study, we apply an advanced deep learning based single-image SR (SISR) method, previously developed for high-resolution late gadolinium enhancement imaging (ISMRM submission 7022), to DT-CMR data.Methods:
With ethical approval, short-axis DT-CMR data were collected on a Siemens Skyra 3T scanner. All our DT-CMR data were acquired at peak-systole (N=133) or in diastasis (N=115) in healthy volunteers, using a breath hold STEAM-EPI sequence with diffusion encoded over 1 complete cardiac cycle [3]. The acquired spatial resolution was 2.8×2.8mm2, 1.4×1.4mm2 reconstructed, with 8mm slice thickness, repetition time 2 cardiac cycles, echo time 23–25ms, with SENSE factor of 2. Typically, 8 to 10 averages of each slice and diffusion encoding direction were acquired. Diffusion was encoded using 6 diffusion directions and diffusion weightings ranging from 150 to 600s/mm2.
Our SISR method is based on a generative adversarial network (GAN) [5], which can provide perceptually realistic SR, e.g., SRGAN [6]. Despite successful applications of the original GAN method in many scenarios, it suffers from the unstable training, collapsed mode and difficulties in tuning hyper-parameters. Thus, Wasserstein GAN (WGAN) with gradient penalty [7,8], which uses the Wasserstein-1 distance instead of the non-continuous divergence, is incorporated in our SISR.
In practice, GAN based model is notoriously hard to train because the two feature distributions are in high-dimensional manifolds that are rarely overlapped and their Jensen–Shannon divergence will always be a constant, causing the vanished gradient [9]. Thus, we propose two strategies to tackle this problem. Firstly, we propose a region of interest (ROI) focused network, which automatically detects and subsequently super-resolves only the detected ROI, in our case the bounding box region around the left ventricle, before the deep residual SR neural network (SRResNet [6]) is deployed. Secondly, all the loss functions are built with a single scale in the original SRResNet. It is very difficult to stabilize the optimization of the GAN based network with higher magnifying factors (e.g., X4 magnification). Therefore, we propose a multi-scale architecture to decompose this difficult problem into a series of simpler subproblems. Figure 1 shows the proposed SISR (dubbed ROI-MS-WGAN). The DT-CMR images were randomly split into training+cross-validation (N=208) and independent testing datasets (N=40, 20 diastole+20 systole), respectively. Images were downsampled by a factor of 4 as input into the training and testing. We compared the performance of our proposed method with bilinear interpolation, SRResNet and SRGAN with X4 magnification.
Results:
Results show that both SRResNet and ROI-MS-WGAN obtained promising SR results compared to the ground truth (Figure 2 and Figure 3). Bilinear interpolation showed over-smoothed results as expected. SRGAN achieved perceptually acceptable results but the difference image and peak SNR (dB)/Structural SIMilarity (PSNR/SSIM) [10] demonstrated large errors. Although for MD, FA and E2A, bilinear interpolation achieved relatively low RMSE, the inter-subject mean transmural HA line profile extracted showed that the results from SRResNet, SRGAN and ROI-MS-WGAN were more realistic (Figure 4). While ROI-MS-WGAN has slightly reduced PSNR/SSIM compared to SRResNet, the calculated DT-CMR parameter maps suggest that the results obtained by SRResNet are over-smooth (Figure 5).Discussion:
High-resolution DT-CMR is attractive, but often comes at the cost of prolonged acquisition durations. We propose a post-processing based SISR incorporating a ROI focused multi-scale architecture that has merits in: (a) reduced training cost via a reduction in network parameters; (b) SR with potentially better perceptual qualities can be achieved by replacing the estimation of transformation in a lower dimensional space; (c) the exclusion of artifacts from irrelevant regions outside of the ROI; (d) stabilized training that copes with higher magnifying factors enabled by the multi-scale architecture. Compared to existing methods, our RFMS-WGAN method has achieved SR results as good as any of the alternatives tested with potentially better perceptual quality. In theory, we can use the ROI-MS-WGAN to increase the apparent spatial resolution and avoid the need for parallel imaging and zero-filling reconstruction. Our study also demonstrated that conventional quantitative metrics (PSNR/SSIM/RMSE) may not be the best measure of SR results alone, and these measures need to be considered along with qualitative visual inspection.Conclusion:
Compared to existing approaches, our simulation-based study shows that the proposed ROI-MS-WGAN method can achieve reliable SR with high magnifying factors while preserving DT-CMR results. We envision a successful application of the proposed method for increasing the spatial resolution of DT-CMR images.Acknowledgements
First author’s PhD research is partially funded by China Scholarship Council (Grant No.201708060173). This study was also funded by the British Heart Foundation Project Grant (Project Number: PG/16/78/32402). Pietro Lio and David Firmin are co-last authors.References
1. Mekkaoui C, Reese TG, Jackowski MP, Bhat H, Sosnovik DE. Diffusion MRI in the heart. NMR Biomed. 2017;30.
2. Nielles-Vallespin S, Khalique Z, Ferreira PF, de Silva R, Scott AD, Kilner P, et al. Assessment of Myocardial Microstructural Dynamics by In Vivo Diffusion Tensor Cardiac Magnetic Resonance. J Am Coll Cardiol [Internet]. 2017;69:661–76. Available from: http://www.ncbi.nlm.nih.gov/pubmed/28183509
3. Nielles-Vallespin S, Mekkaoui C, Gatehouse P, Reese TG, Keegan J, Ferreira PF, et al. In vivo diffusion tensor MRI of the human heart: reproducibility of breath-hold and navigator-based approaches. Magn Reson Med. 2013;70:454–65.
4. Gorodezky M, Scott AD, Ferreira PF, Nielles-Vallespin S, Pennell DJ, Firmin DN. Diffusion tensor cardiovascular magnetic resonance with a spiral trajectory: An in vivo comparison of echo planar and spiral stimulated echo sequences. Magn Reson Med. 2018;80:648–54.
5. Goodfellow I, Pouget-Abadie J, Mirza M. Generative Adversarial Networks. arXiv Prepr arXiv … [Internet]. 2014;1–9. Available from: http://arxiv.org/abs/1406.2661
6. Ledig C, Theis L, Huszar F, Caballero J, Aitken A, Tejani A, et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. 2016; Available from: http://arxiv.org/abs/1609.04802
7. Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. 2017;
8. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A. Improved Training of Wasserstein GANs. Adv Neural Inf Process Syst [Internet]. 2017. p. 5767–5777. Available from: http://arxiv.org/abs/1704.00028
9. Wang J, He Z, Feng C, Zhu Z, Lin Q, Lv J, et al. Domain Confusion with Self Ensembling for Unsupervised Adaptation. 2018; Available from: http://arxiv.org/abs/1810.04472
10. Yang C-Y, Ma C, Yang M. Single-Image Super-Resolution: A Benchmark. Comput Vis – ECCV 2014 [Internet]. 2014. p. 372–86. Available from: http://link.springer.com/10.1007/978-3-319-10593-2_25
Figures
