1917
Reliability and test-retest reproducibility of advanced diffusion metrics in healthy older adults1Department of Neuroimaging, King's College London, London, United Kingdom, 2Department of Forensics and Neurodevelopmental Sciences, King's College London, London, United Kingdom, 3Janssen Research and Development, Janssen Pharmaceutica NV, Beerse, Belgium
Synopsis
Reliability and test-retest reproducibility of simple and advanced diffusion metrics were evaluated in healthy older adults. Advanced models such as Diffusion Kurtosis Imaging, White Matter Tract Integrity, Neurite Orientation Dispersion and Density Imaging and other multi-compartment methods have greater specificity but lower reliability and reproducibility than simple models such as DTI and spherical harmonic metrics. All models nevertheless had good or acceptable reproducibilty and reliability in certain cases. Particular care may be given when planning clinical research applications with advanced metrics, as greater sample sizes and/or improved data quality may be needed.
Introduction
Diffusion MRI metrics are used to study grey and white matter (WM) tissue properties in vivo and provide biomarkers for neurodegenerative and psychiatric disorders1,2. Diffusion tensor imaging3,4 (DTI) metrics are widely applied but limited by a lack of biological specificity. Microstructure imaging aims to develop diffusion models with increased specificity, however, for these advanced metrics to be effective in neuroscience and clinical studies, it is important to understand how reliable and reproducible the metrics are.In this study we compared reliability and reproducibility of conventional Diffusion Tensor Imaging (DTI) and commonly-used advanced approaches: Free Water Elimination (FWE) DTI5, diffusion kurtosis imaging6,7, (DKI), White Matter Tract Integrity8 (WMTI) and Neurite Orientation Dispersion and Density Imaging9, NODDI).
As baseline for anisotropy, diffusivity and multi-compartment volume fractions, we also included single-shell spherical harmonics-derived (SH) metrics and a “simplified” multicompartmental model (MCsimple). The intra-class correlation coefficient10 (ICC) and within-subjects coefficient of variation (CVws) were used to assess reliability and reproducibility in white and grey matter (GM).
Methods
Participants. Nineteen healthy adults (7 females) aged 52-74 years were scanned three times at 0, 1 and 4 weeks. Participants gave written informed consent. Approved by the local research ethics subcommittee (ethics number HR-17/18-5720).MRI Acquisition. A 3T MR750 GE scanner with 32-channel Nova coil was used to acquire multi-band DWI spin echo EPI data with in-plane acceleration ARC=2, multi-slice acceleration MB=3, TE/TR=66.5/4000 ms, 2 mm isotropic voxels and 12/12/48/48 volumes/directions with b=0/500/1000/2000 s/mm2.
Preprocessing. Data were denoised10, corrected for Gibbs ringing11, motion, eddy current and magnetic susceptibility distortion artefacts (FSL topup12,13 and eddy14).
Modelling. See Table 1.
Analysis. Maps were registered to MNI space using ANTs SyN registration21. Regions of interest (ROI) of WM tracts and cortical GM were defined using Megatrack Atlas (megatrackatlas.org) and the Harvard-Oxford cortical atlas22 (Figure 2). ICC and CVws were calculated from the mean of each ROI for each metric using a neuroimaging ICC toolbox23.
Results
Reliability. Maps of each metric are shown in Figure 2. All anisotropy metrics had high reliability (ICC>0.8) for most tracts (Figure 3). Cortical GM scored lower, with ICC between 0.6-0.8 for FAFWE, FADKI and AP.Among the diffusivity measures, MDDTI, ADCSH, ADDTI, ADFWE and RDDTI performed well for most tracts (ICC>0.8). The remaining FWE-DTI and DKI diffusivity metrics were less reliable and had greater variability between tracts (ICC between 0.6-0.9). The cortex was highly reliable across the diffusivity metrics (ICC>0.8).
Kurtosis metrics had a good ICC between 0.6-0.9 for MK and AK, but 0.2-0.8 for RK in most tracts. Kurtosis metrics were less reliable in the cortex, with ICC<0.5.
WMTI metrics and fast-diffusion volume fractions (FWFFWE, FASTMCsimple and ISOVFNODDI) showed large variability in the reliability between ROIs (ICC between 0.2-0.8) for most tracts. Restricted-diffusion fractions in WM ROIs were the most reliable (RESTMCsimple, ICC>0.8; ICVFNODDI, ICC>0.7). In contrast to WM, cortical measurements were very reliable for the fast-diffusion fractions (ICC>0.8), and not reliable for restricted-diffusion, with ICC≈0.5. The fibre organisation metrics performed well for most ROIS (EAS-TORTDKI, ICC>0.7; ODNODDI, ICC>0.8).
Outlier tracts with low reliability included the cingulum and CC, while the fornix had higher reliability across most of the metrics.
Reproducibility. All anisotropy, diffusivity, WMTI, kurtosis and fibre organisation metrics had high reproducibility (CVws<10%), for most WM tracts, while multi-compartment volume fractions were less reproducible (Figure 4). ICVFNODDI had CVws<15%, FWFFWE, SLOWMCsimple and RESTMCsimple , CVws<30%, FASTMCsimple CVws<10-40% and ISOVFNODDI was the least reproducible, with CVws for WM (excluding the CC and fornix) between 30-80%. The CC, fornix and cortical GM regions were relatively reproducible for fast diffusion fractions and less reproducible for SLOWMCsimple, restricted-diffusion fractions and ODNODDI.
Discussion
Our results show that simple diffusion models have better reliability and reproducibility than complex models. Anisotropy measures performed well in most WM tracts for all models, with FADTIand AP showing the highest reliability and reproducibility. For the diffusivity metrics however, the DTI model for MD, AD and RD performed noticeably better than the FWE-DTI and DKI models. The results for DTI metrics are supported by previous literature on reliability and reproducibility24. The DKI kurtosis and multi-compartment diffusivity metrics had relatively low reliability compared with the anisotropy and standard diffusivity metrics suggesting data quality can be quickly become a critical element of any protocol design using advanced diffusion models. Increased SNR and/or a larger number of diffusion weighted volumes are needed compared to traditional DTI and single shell metrics.As expected from the diffusion characteristics of WM and GM tissue, fast diffusion volume fractions were highly reliable and had good reproducibility for GM and performed less well for WM tracts (particularly ISOVFNODDI, while the opposite was true for reliability of restricted-diffusion fractions, with ICVFNODDI very reproducible. Fibre organization metrics EAS-TORTDKI and particularly ODNODDI performed well for most ROIs.
Conclusion
Advanced models provide more independent metrics to describe tissue characteristics but at the cost of reliability and reproducibility, compared with simple DTI. This result should inform the use of complex diffusion models in neuroscience applications. Simple models may be better suited for applications without a strong a-priori hypothesis and/or limited data-acquisition protocols. A stronger hypothesis, larger sample size and/or improved data quality for increased statistical power is advised for advanced diffusion metrics.Acknowledgements
Thanks to Janssen Pharmaceutica and the BRC Neuroimaging Chapter for supporting this research.References
1. Goveas J, O’Dwyer L, Mascalchi M, et al. Diffusion-MRI in neurodegenerative disorders. Magn Reson Imaging. 2015;33(7):853-876. doi:10.1016/J.MRI.2015.04.006
2. Raja R, Rosenberg G, Caprihan A. Review of diffusion MRI studies in chronic white matter diseases. Neurosci Lett. 2019;694:198-207. doi:10.1016/J.NEULET.2018.12.007
3. Basser PJ, Mattiello J, LeBihan D. MR diffusion tensor spectroscopy and imaging. Biophys J. 1994;66(1):259-267. doi:10.1016/S0006-3495(94)80775-1
4. Basser PJ, Pierpaoli C. Microstructural and physiological features of tissues elucidated by quantitative-diffusion-tensor MRI. J Magn Reson Ser B. 1996;111(3):209-219. http://www.sciencedirect.com/science/article/pii/S1064186696900862.
5. Pasternak O, Sochen N, Gur Y, Intrator N, Assaf Y. Free water elimination and mapping from diffusion MRI. Magn Reson Med. 2009;62(3):717-730. doi:10.1002/MRM.22055
6. Jensen JH, Helpern JA, Ramani A, Lu H, Kaczynski K. Diffusional kurtosis imaging: The quantification of non-gaussian water diffusion by means of magnetic resonance imaging. Magn Reson Med. 2005;53(6):1432-1440. doi:10.1002/MRM.20508
7. Lu H, Jensen JH, Ramani A, Helpern JA. Three-dimensional characterization of non-gaussian water diffusion in humans using diffusion kurtosis imaging. NMR Biomed. 2006;19(2):236-247. doi:10.1002/NBM.1020
8. Fieremans E, Jensen JH, Helpern JA. NeuroImage White matter characterization with diffusional kurtosis imaging. Neuroimage. 2011;58(1):177-188. doi:10.1016/j.neuroimage.2011.06.006
9. Zhang H, Schneider T, Wheeler-Kingshott CA, Alexander DC. NODDI: practical in vivo neurite orientation dispersion and density imaging of the human brain. Neuroimage. 2012;61(4):1000-1016. doi:10.1016/J.NEUROIMAGE.2012.03.072
10. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420-428. http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=18839484&retmode=ref&cmd=prlinks.
11. Veraart J, Novikov DS, Christiaens D, Ades-aron B, Sijbers J, Fieremans E. Denoising of diffusion MRI using random matrix theory. Neuroimage. 2016;142:394-406. doi:10.1016/j.neuroimage.2016.08.016
12. Kellner E, Dhital B, Kiselev VG, Reisert M. Gibbs-ringing artifact removal based on local subvoxel-shifts. Magn Reson Med. 2016;76(5):1574-1581. doi:10.1002/mrm.26054
13. Andersson JLR, Skare S, Ashburner J. How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. Neuroimage. 2003;20(2):870-888. doi:10.1016/S1053-8119(03)00336-7
14. Smith, Stephen M, Jenkinson, Mark, Woolrich, Mark W, et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage. 2004;23(0):S208--S219. doi:10.1016/j.neuroimage.2004.07.051
15. Andersson JLR, Sotiropoulos SN. An integrated approach to correction for off-resonance effects and subject movement in diffusion MR imaging. Neuroimage. 2016;125:1063-1078. doi:10.1016/j.neuroimage.2015.10.019
16. Pierpaoli C, Walker L. TORTOISE: an integrated software package for processing of diffusion MRI data. ISMRM 18th Annu Meet Stock Sweden. 2010;51(1):2010. www.tortoisedti.org. Accessed November 8, 2021.
17. Irfanoglu MO, Nayak A, Jenkins J, Pierpaoli C. TORTOISE v3 : Improvements and New Features of the NIH Diffusion MRI Processing Pipeline motion & eddy currents distortion is to register the DWIs to a b = 0 s / mm 2 image with a mutual information metric and a parsimonious quadratic / cubic eddy current. ISMRM 25th Annu Meet Honolulu, HI. 2018:9-11. doi:10.1016/j.neuroimage.2016.08.016.5.
18. Veraart J, Sijbers J, Sunaert S, Leemans A, Jeurissen B. Weighted linear least squares estimation of diffusion MRI parameters: strengths, limitations, and pitfalls. Neuroimage. 2013;81:335-346. doi:10.1016/J.NEUROIMAGE.2013.05.028
19. Daducci A, Canales-Rodríguez EJ, Zhang H, Dyrby TB, Alexander DC, Thiran JP. Accelerated Microstructure Imaging via Convex Optimization (AMICO) from diffusion MRI data. Neuroimage. 2015;105:32-44. doi:10.1016/J.NEUROIMAGE.2014.10.026
20. Dell’Acqua F, Lacerda L, Catani M, Simmons A. Anisotropic Power Maps: A diffusion contrast to reveal low anisotropy tissues from HARDI data. In: Proceedings of the 22nd Annual Meeting of ISMRM. Vol 22. Milan, Italy; 2014:29960-29967. doi:10.1021/acsami.6b09809
21. Avants BB, Epstein CL, Grossman M, Gee JC. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal. 2008;12(1):26-41. doi:10.1016/J.MEDIA.2007.06.004
22. Desikan RS, Ségonne F, Fischl B, et al. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage. 2006;31(3):968-980. doi:10.1016/J.NEUROIMAGE.2006.01.021
23. Caceres A, Hall DL, Zelaya FO, Williams SCR, Mehta MA. Measuring fMRI reliability with the intra-class correlation coefficient. Neuroimage. 2009;45(3):758-768. doi:10.1016/j.neuroimage.2008.12.035
24. Luque Laguna PA, Combes AJE, Streffer J, et al. Reproducibility, reliability and variability of FA and MD in the older healthy population: A test-retest multiparametric analysis. NeuroImage Clin. 2020;26(January):102168. doi:10.1016/j.nicl.2020.102168
Figures
Table 1
Models and methods used to calculate diffusion metrics.