What is the best method for robust statistical inference on connectomic graph metrics?
Mark Drakesmith1,2, David Linden2, Anthony S David3, and Derek K Jones1,2

1CUBRIC, Cardiff University, Cardiff, United Kingdom, 2Neuroscience and Mental health Research Institute, Cardiff University, Cardiff, United Kingdom, 3Institute of Psychiatry, Psychology and Neurosceince, Kings College London, London, United Kingdom

Synopsis

Connectomic network analyses, while powerful, suffer from high instability, which is problematic for robust statistical inference. The area under the curve (AUC) across thresholds is a common approach, but lacks robustness to this instability. A superior approach is multi-threshold permutation correction (MTPC), but this is computationally expensive. Smoothed AUCs (smAUCs) are less costly and theoretically can achieve the same level of sensitivity as MTPC. smAUC was tested and compared with MTPC in a virtual patient-control comparison. Results show that smAUC sensitivity is not consistently comparable to MTPC and that exhaustive searching across the threshold space is required for robust inference.

Introduction

Network, or graph theoretical, analyses are a powerful and increasingly utilised tool for studying brain connectomics (1). However, graph metrics are highly sensitive to various parameters and as such are highly unstable (2–4). Large phase transitions in topology can lead to instability and regions of high variance across thresholds (fig. 1) (4,5). A common approach to analysing graph metrics is to compute the area under the curve (AUC) across thresholds, e.g. (6). However, this can dilute inferred effects if true effects only manifest in narrow threshold ranges, which is often the case (4). A more recent development is multi-threshold permutation correction (MTPC) (4), which agnostically identifies limited but sustained effects across different threshold levels. However, this method is computationally expensive. A potentially less costly approach is to compute smoothed AUCs (smAUCs). Theoretically, this will smooth out narrow effects across a wider range of thresholds while also reducing the high variance around phase transitions. However, this approach is yet to be tested.

Aims

To test and compare the sensitivity of smAUCs with different smoothing kernels with the MTPC method in a virtual patient-control comparison.

Method

Connectomes for 248 subjects were constructed with deterministic tractography with the damped Lucy-Richardson algorithm (7) (step size = 1mm, angle threshold = 45°, seed point resolution = 2mm3, length threshold=20–500mm, FOD threshold = 0.05, β =1.77, λ = 0.0019, η = 0.04), from 60 direction HARDI acquisition (cardiac gated, b=1200s/mm2, TE=87ms, 60 slices, FoV=230×230mm, acquisition matrix=96×96, 2.4mm3 isotropic resolution). Connectivity was inferred between cortical regions of the automated atlas labelling (AAL) atlas (8), with additional a priori anatomical constraints to eliminate spurious streamlines. The datasets were randomly assigned to a 'healthy' or a 'lesioned' group. In the lesioned group a proportion of inter-hemispheric streamlines were removed from the connection (Gaussian distribution with standard deviation ξ, ranged from 0:0.05:0.3). The larger ξ is, the larger the effect size. Four graph metrics were tested: global efficiency, mean clustering coefficient, mean betweenness and smallworldness. Graph metrics were computed across streamline counts of 0:1:30. smAUCs were computed with Gaussian smoothing kernels with FWHMs of σ = 0, 1, 2, 4, 8, 16 and 32 (fig. 2). Permutation corrected p-values (pcorr) were obtained from 500 randomisations. MTPC was also performed across the same threshold range and the same randomisation indices. The sensitivity of each method was assessed from the minimum value for ξ at which pcorr<0.05.

Results

In most cases, the MTPC method was sensitive to smaller ξ than the smAUC method (fig. 3, 4). The only exceptions is mean clustering coefficient where the smAUC with σ=32 was more sensitive. Mean betweeness showed no sensitivity to group effects using smAUC. In most cases, the smoothing kernel made no improvement compared to the standard AUC method, although it does have an effect of increased sensitivity for global efficiency.

Conclusion

MTPC provides robust testing of statistical effect on graph metrics, allowing the identification of sustained but restricted effects across thresholds. The smAUC theoretically does the same smoothing localised effect across larger ranges of thresholds but without the computational demands of MTPC. However, smAUC does not prevent dilution of the true effect by regions where true effects are hidden due to high variance and does not provide consistently comparable performance compared to MTPC. smAUC shows comparable performance with Global efficiency. This may be explained by this metric being less susceptible to phase transition effects. This indicates that the effect of phase transitions cannot be effectively overcome with smoothing and a more exhaustive search of the threshold space is still required. These results have implications for statistical connectomic network analyses of neuroimaging data, especially in comparison where effects on network topology are expected to be subtle.

Acknowledgements

The MRI data used in this study was funded by the Wellcome Trust and the Medical Research Council, UK.

References

1. Rubinov M, Sporns O. Complex network measures of brain connectivity: uses and interpretations. Neuroimage 2010;52:1059–69. doi: 10.1016/j.neuroimage.2009.10.003.

2. van Wijk BCM, Stam CJ, Daffertshofer A. Comparing brain networks of different size and connectivity density using graph theory. Sporns O, editor. PLoS One 2010;5:e13701. doi: 10.1371/journal.pone.0013701.

3. Fornito A, Zalesky A, Breakspear M. Graph analysis of the human connectome: promise, progress, and pitfalls. Neuroimage 2013;80:426–44. doi: 10.1016/j.neuroimage.2013.04.087.

4. Drakesmith M, Caeyenberghs K, Dutt A, Lewis G, David AS, Jones DK. Overcoming the effects of false positives and threshold bias in graph theoretical analyses of neuroimaging data. Neuroimage 2015;118:313–333. doi: doi:10.1016/j.neuroimage.2015.05.011.

5. Ginestet CE, Nichols TE, Bullmore ET, Simmons A. Brain network analysis: Separating cost from topology using Cost-Integration. PLoS One 2011;6:e21570. doi: 10.1371/journal.pone.0021570.

6. Hosseini SMH, Hoeft F, Kesler SR. GAT: a graph-theoretical analysis toolbox for analyzing between-group differences in large-scale structural and functional brain networks. PLoS One 2012;7:e40709. doi: 10.1371/journal.pone.0040709.

7. Dell’acqua F, Scifo P, Rizzo G, Catani M, Simmons A, Scotti G, Fazio F. A modified damped Richardson-Lucy algorithm to reduce isotropic background effects in spherical deconvolution. Neuroimage 2010;49:1446–58. doi: 10.1016/j.neuroimage.2009.09.033.

8. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 2002;15:273–89. doi: 10.1006/nimg.2001.0978.

Figures

Fig. 1. Toy example of the effect of phase transitions ϕ on inferential statistics across thresholds (τ). Around such transition, the graph metric f(τ) has high variance where as away from ϕ, variance is low and group effect size (denoted SH) is detectable.

Fig. 2. Examples of smoothing kernels (with FWHM of σ) applied to graph metrics across thresholds.

Fig. 3. pcorr values across ξ for the four graph metrics and inference methods (smAUC and MPTC) tested. the significance threshold of pcorr<0.05 is indicated in grey.

Fig 4. Sensitvity as measured by the minimum detectable ξ (at pcorr<0.05) for the smAUCs across smoothing kernels and MTPC. For any comparisons where no threshold satisfies pcorr<0.05, sensitivity is treated as infinite.



Proc. Intl. Soc. Mag. Reson. Med. 24 (2016)
2046