2325
Rapid 4D-MRI reconstruction using a Deep RAdial ConvoLutionAl neural network: Dracula1Joint Department of Physics, The Institute of Cancer Research and The Royal Marsden NHS Foundation Trust, London, United Kingdom, 2CR UK Cancer Imaging Centre, The Institute of Cancer Research and The Royal Marsden NHS Foundation Trust, London, United Kingdom, 3Department of Radiotherapy, The Institute of Cancer Research and The Royal Marsden NHS Foundation Trust, London, United Kingdom, 4Department of Radiotherapy Related Research, The Christie NHS Foundation Trust, Manchester, United Kingdom, 5Medical Physics in Radiology, The German Cancer Research Center (DKFZ), Heidelberg, Germany
Synopsis
4D-MRI could inform online treatment plan adaptation on MRI guided radiotherapy systems, but long iterative reconstruction times (> 10 minutes) limit its use. A deep convolutional neural network was trained to learn the joint MoCo-HDTV algorithm and high-quality 4D-MRI (1.25x1.25x3.3 mm3, 16 respiratory phases) were reconstructed from gridded raw data in 27 seconds. Calculated 4D-MRI exhibited a high structural similarity index (0.97 ± 0.013) with the iteratively reconstructed test images and only a minor loss of fine details. Despite exclusively training the network on data from a diagnostic scanner, 4D-MRI were successfully reconstructed from raw data acquired on an MR-linac.
Purpose
To accelerate 4D-MRI reconstruction for online radiotherapy treatment plan adaptation and position verification on MRI guided radiotherapy (MRgRT) systems1. State-of-the-art compressed-sensing 4D-MRI reconstructions exploit spatiotemporal image sparsity to reconstruct artefact-free images from heavily undersampled data, but they are currently limited by long calculation times2-5. Deep convolutional neural networks (dCNNs) can accelerate streaking artefact reduction6 and variational networks were shown to outperform standard reconstruction algorithms7. Here, a 3D Deep RAdial ConvoLutionAl (Dracula) neural network was trained to learn the 4D joint MoCo-HDTV algorithm2 and applied to accelerate reconstruction of T1-weighted (T1w) 4D-MRI that were acquired on both diagnostic and MR-linac systems.Methods
A volumetric radial T1w stack-of-stars spoiled gradient echo sequence8 was employed on a 1.5 T diagnostic scanner (45 patient and 25 healthy volunteer scans) and on a 1.5 T MR-linac (2 healthy volunteers) (Table 1). Raw data were corrected with an adaptive gradient-delay compensation9 and then sorted into 20 overlapping respiratory phases using a self-gating signal5. 4D magnitude images were calculated for all data sets acquired on the diagnostic scanner with both a gridding reconstruction (Gridded) and the joint MoCo-HDTV algorithm2 (MoCo).
To comply with the dCNN’s architecture, respiratory phases 5, 10, 15 and 20 were discarded and Gridded and MoCo 4D-MRI were scaled on a subject-by-subject basis by 0.67*maximum Gridded intensity value. Afterwards, 4D-MRI were split into training (74 %), validation (17 %) and test sets (9 %). A 3D U-net architecture10 was implemented using Tensorflow11 to learn the joint MoCo-HDTV reconstruction (Figure 1). Network inputs and outputs were the scaled Gridded and MoCo 4D-MRI, respectively, with dimensions: 2D in-plane matrix-size x respiratory phase (256x256x16) and were looped in mini-batches over all slices and subjects. The Adam optimiser12 was chosen with mean square error as the loss function. Network hyper-parameters were optimised by monitoring training and validation error: learning rate = 10-5, drop-out rate= 0.25-0.5, epochs = 50 and batch-size = 2. Training and testing were performed on an NVIDIA Quadro P6000 GPU with 24 GB memory.
For validation purposes, the results of passing the scaled Gridded test-data through Dracula were compared to the corresponding MoCo test-data both visually and in terms of the structural similarity index (SSIM)13. The SSIM is scaled between 0 and 1, where the SSIM of two images = 1, if they are identical in terms of contrast, luminance and structure.
The feasibility of training Dracula on diagnostic data and then applying it to reconstruct MR-linac data was investigated. Gridded MR-linac 4D-MRI were reconstructed, scaled and used as input to Dracula as per the diagnostic data. The appearances of the Dracula-reconstructed 4D-MRI were then compared qualitatively to the input Gridded MR-linac 4D-MRI.
Results
Dracula was trained in ≈ 5 days and required < 27 seconds to reconstruct 4D-MRI (voxel size ≈ 1.25x1.25x3.3 mm3, 16 respiratory phases). Figure 2 shows an example comparison between Gridded, Dracula-reconstructed and MoCo diagnostic test-data. Dracula-reconstructed image appearance was qualitatively similar to the MoCo image but exhibited a slight loss of high frequency structures (e.g. small lung vessels) and minor residual streaking. For this example, tumour and healthy organs were similarly visible in the Dracula-reconstructed and MoCo images. Average SSIM between Dracula-reconstructed and MoCo images was 0.97 ± 0.013 which was greatly increased when compared to the corresponding Gridded and MoCo SSIM (0.72 ± 0.082). Dracula greatly reduced heavy streaking artefacts present in Gridded MR-linac test-data (Figure 3), being only trained on data from a diagnostic scanner.Discussion
We have presented a method to rapidly calculate high-quality 4D-T1w MRI by training a 3D dCNN to learn the joint MoCo-HDTV reconstruction. Dracula is faster (< 27 seconds) than state-of-the-art compressed-sensing reconstructions, such as GRASP3-4 (≈ 10 minutes) and joint MoCo-HDTV2 (unoptimised prototype ≈ 10 hours), making it an ideal candidate for applications where short reconstruction times are required. For instance, Dracula might be employed on MRgRT systems to obtain 4D-T1w MRI for online radiotherapy treatment plan adaptation and position verification. We demonstrated the feasibility of this scenario by successful application of Dracula to greatly reduce heavy streaking artefacts in MR-linac data.
The high SSIM between Dracula-reconstructed and MoCo images supports the hypothesis that the joint MoCo-HDTV reconstruction was well approximated. Remaining differences might be reduced by including information from adjacent slices.
Conclusion
High-quality 4D-T1w MRI were rapidly reconstructed using a 3D dCNN and could be employed on MRgRT systems for online radiotherapy treatment plan adaptation and position verification.Acknowledgements
We thank Siemens Healthineers GmbH, in particular Alto Stemmer, for providing us with the prototype radial MRI sequence. We also thank David Collins, Nina Tunariu and Dow-Mu Koh for their help with data acquisition and image analysis. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used for this research. We acknowledge NHS funding to the NIHR Biomedical Research Centre and the Clinical Research Facility at The Institute of Cancer Research and The Royal Marsden NHS Foundation Trust and the CR UK Cancer Imaging Centre grant C1060/A16464. We acknowledge funding from CR UK programme grants C33589/A19727, and project grants C347/A18365, C309/A20926 and C7224/A23275. Martin O. Leach is an NIHR Emeritus Senior Investigator.References
1. Bainbridge H, Salem A, Tijssen RHN, et al. Magnetic resonance imaging in precision radiation therapy for lung cancer. Transl Lung Cancer Res. 2017;6(6):689-707.
2. Rank CM, Heußer T, Buzan MT, et al. 4D respiratory motion-compensated image reconstruction of free-breathing radial MR data with very high undersampling. Magn Reson Med. 2017;77(3):1170–1183.
3. Feng L, Grimm R, Block KT, et al. Golden-angle radial sparse parallel MRI: Combination of compressed sensing, parallel imaging, and golden-angle radial sampling for fast and flexible dynamic volumetric MRI. Magn Reson Med. 2014;72(3):707–717.
4. Mickevicius NJ and Paulson ES. Investigation of undersampling and reconstruction algorithm dependence on respiratory correlated 4D-MRI for online MR-guided radiation therapy. Phys Med Biol. 2017;62(8):2910–2921.
5. Freedman JN, Collins DJ, Bainbridge H, et al. T2-weighted 4D magnetic resonance imaging for application in magnetic resonance – guided radiotherapy treatment planning. Invest Radiol. 2017;52(10):563–573.
6. Han Y, Yoo J, Kim H, et al. Deep learning with domain adaptation for accelerated projection-reconstruction MR. Magn Reson Med. 2018;80(3):1189–1205.
7. Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med. 2018;79(3):3055–3071.
8. Block KT, Chandarana H, Milla S, et al. Towards routine clinical use of radial stack‐of‐stars 3D gradient‐echo sequences for reducing motion sensitivity. J Korean Phys Soc. 2014;18(2):87-106.
9. Block K and Uecker M. Simple method for adaptive gradient-delay compensation in radial MRI. ISMRM 19th Annual Meeting & Exhibition. 2011;2816.
10. Ronneberger O, Fischer P and Brox T. U-net: Convolutional networks for biomedical image segmentation. MICCAI 18th Meeting. 2015;9351.
11. Abadi M, Agarwal A, Barham P, et al. Tensorflow: Large-scale machine learning on heterogeneous systems. 2015;Software available from tensorflow.org.
12. Kingma DP and Ba JA. A method for stochastic optimization. 2014;arXiv:1412.6980.
13. Z Wang, Bovik A, Sheikh HR, et al. Image Qualifty Assessment: From Error Visibility to Structural Similarity. IEEE Trans Image Process. 2004;13(4):600–612.
Figures
