Mark Drakesmith1,2, David Linden2, Anthony S David3, and Derek K Jones1,2
1CUBRIC, Cardiff University, Cardiff, United Kingdom, 2Neuroscience and Mental health Research Institute, Cardiff University, Cardiff, United Kingdom, 3Institute of Psychiatry, Psychology and Neurosceince, Kings College London, London, United Kingdom
Synopsis
Connectomic network analyses,
while powerful, suffer from high instability, which is problematic for robust
statistical inference. The area under the curve (AUC) across thresholds is a
common approach, but lacks robustness to this instability. A superior approach
is multi-threshold permutation correction (MTPC), but this is computationally
expensive. Smoothed AUCs (smAUCs) are less costly and theoretically can achieve
the same level of sensitivity as MTPC. smAUC was tested and compared with MTPC
in a virtual patient-control comparison. Results show that smAUC sensitivity
is not consistently comparable to MTPC and that exhaustive searching across the
threshold space is required for robust inference. Introduction
Network, or graph theoretical, analyses are a powerful and
increasingly utilised tool for studying brain connectomics (1). However, graph metrics are
highly sensitive to various parameters and as such are highly unstable (2–4). Large phase transitions in topology can lead
to instability and regions of high variance across thresholds (fig. 1) (4,5). A common approach to analysing graph metrics
is to compute the area under the curve (AUC) across thresholds, e.g. (6). However, this can dilute
inferred effects if true effects only manifest in narrow threshold ranges,
which is often the case (4). A more recent development is
multi-threshold permutation correction (MTPC) (4), which agnostically
identifies limited but sustained effects across different threshold
levels. However, this method is computationally
expensive. A potentially less costly approach is to compute smoothed AUCs
(smAUCs). Theoretically, this will smooth out narrow effects across a wider
range of thresholds while also reducing the high variance around phase
transitions. However, this approach is
yet to be tested.
Aims
To test and compare the sensitivity of smAUCs with different
smoothing kernels with the MTPC method in a virtual patient-control comparison.
Method
Connectomes for 248 subjects were constructed with
deterministic tractography with the damped Lucy-Richardson algorithm (7) (step size = 1mm, angle threshold
= 45°, seed point resolution = 2mm
3, length threshold=20–500mm, FOD threshold =
0.05, β =1.77, λ = 0.0019, η = 0.04), from 60 direction HARDI acquisition (cardiac
gated, b=1200s/mm
2, TE=87ms, 60 slices, FoV=230×230mm, acquisition matrix=96×96,
2.4mm
3 isotropic resolution). Connectivity was inferred between cortical
regions of the automated atlas labelling (AAL) atlas (8), with additional a priori
anatomical constraints to eliminate spurious streamlines. The datasets were
randomly assigned to a 'healthy' or a 'lesioned' group. In the lesioned group a
proportion of inter-hemispheric streamlines were removed from the connection
(Gaussian distribution with standard deviation ξ, ranged from 0:0.05:0.3). The
larger ξ is, the larger the effect size. Four graph metrics were tested: global
efficiency, mean clustering coefficient, mean betweenness and
smallworldness. Graph metrics were
computed across streamline counts of 0:1:30. smAUCs were computed with Gaussian
smoothing kernels with FWHMs of σ = 0, 1, 2, 4, 8, 16 and 32 (fig. 2). Permutation
corrected p-values (p
corr) were obtained from 500 randomisations.
MTPC was also performed across the same threshold range and the same randomisation
indices. The sensitivity of each method was assessed from the minimum value for
ξ at which p
corr<0.05.
Results
In most cases, the MTPC method was sensitive to smaller ξ
than the smAUC method (fig. 3, 4). The only exceptions is mean clustering
coefficient where the smAUC with σ=32
was more sensitive. Mean betweeness showed no sensitivity to group effects using smAUC. In most cases, the smoothing kernel made no improvement
compared to the standard AUC method, although it does have an effect of
increased sensitivity for global efficiency.
Conclusion
MTPC provides robust testing of statistical effect on graph
metrics, allowing the identification of sustained but restricted effects across
thresholds. The smAUC theoretically does the same smoothing localised effect
across larger ranges of thresholds but without the computational demands of
MTPC. However, smAUC does not prevent dilution of the true effect by regions
where true effects are hidden due to high variance and does not provide
consistently comparable performance compared to MTPC. smAUC shows comparable
performance with Global efficiency. This may be explained by this metric being
less susceptible to phase transition effects. This indicates that the effect of
phase transitions cannot be effectively overcome with smoothing and a more
exhaustive search of the threshold space is still required. These results have
implications for statistical connectomic network analyses of neuroimaging data,
especially in comparison where effects on network topology are expected to
be subtle.
Acknowledgements
The MRI data used in this study was funded by the Wellcome Trust and the Medical Research Council, UK.References
1. Rubinov M, Sporns O. Complex network measures of
brain connectivity: uses and interpretations. Neuroimage 2010;52:1059–69. doi:
10.1016/j.neuroimage.2009.10.003.
2. van Wijk BCM, Stam CJ, Daffertshofer A. Comparing brain
networks of different size and connectivity density using graph theory. Sporns
O, editor. PLoS One 2010;5:e13701. doi: 10.1371/journal.pone.0013701.
3. Fornito A, Zalesky A, Breakspear M. Graph analysis of the
human connectome: promise, progress, and pitfalls. Neuroimage 2013;80:426–44.
doi: 10.1016/j.neuroimage.2013.04.087.
4. Drakesmith M, Caeyenberghs K, Dutt A, Lewis G, David AS,
Jones DK. Overcoming the effects of false positives and threshold bias in graph
theoretical analyses of neuroimaging data. Neuroimage 2015;118:313–333. doi:
doi:10.1016/j.neuroimage.2015.05.011.
5. Ginestet CE, Nichols TE, Bullmore ET, Simmons A. Brain
network analysis: Separating cost from topology using Cost-Integration. PLoS
One 2011;6:e21570. doi: 10.1371/journal.pone.0021570.
6. Hosseini SMH, Hoeft F, Kesler SR. GAT: a graph-theoretical
analysis toolbox for analyzing between-group differences in large-scale
structural and functional brain networks. PLoS One 2012;7:e40709. doi:
10.1371/journal.pone.0040709.
7. Dell’acqua F, Scifo P, Rizzo G, Catani M, Simmons A,
Scotti G, Fazio F. A modified damped Richardson-Lucy algorithm to reduce
isotropic background effects in spherical deconvolution. Neuroimage
2010;49:1446–58. doi: 10.1016/j.neuroimage.2009.09.033.
8. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello
F, Etard O, Delcroix N, Mazoyer B, Joliot M. Automated anatomical labeling of
activations in SPM using a macroscopic anatomical parcellation of the MNI MRI
single-subject brain. Neuroimage 2002;15:273–89. doi: 10.1006/nimg.2001.0978.