Improving power in functional magnetic resonance imaging by moving beyond cluster-level inference

Significance

Localizing cognitive function to distinct brain areas has been a mainstay of human brain research since early reports that focal injuries produce changes in behavior. Yet, accumulating evidence shows that areas do not act in isolation. Here, we evaluate the practical implications of the localizationist perspective by comparing the performance of localizing versus broad-scale statistical procedures in real connectome data (1,000 subjects performing 7 tasks). We find that popular localizing procedures miss substantially more true effects than simple broad-scale procedures. By highlighting the power of simple alternatives, we argue that moving beyond localization is viable and can help unlock opportunities for human neuroscience discovery.

Abstract

Inference in neuroimaging typically occurs at the level of focal brain areas or circuits. Yet, increasingly, well-powered studies paint a much richer picture of broad-scale effects distributed throughout the brain, suggesting that many focal reports may only reflect the tip of the iceberg of underlying effects. How focal versus broad-scale perspectives influence the inferences we make has not yet been comprehensively evaluated using real data. Here, we compare sensitivity and specificity across procedures representing multiple levels of inference using an empirical benchmarking procedure that resamples task-based connectomes from the Human Connectome Project dataset (∼1,000 subjects, 7 tasks, 3 resampling group sizes, 7 inferential procedures). Only broad-scale (network and whole brain) procedures obtained the traditional 80% statistical power level to detect an average effect, reflecting >20% more statistical power than focal (edge and cluster) procedures. Power also increased substantially for false discovery rate– compared with familywise error rate–controlling procedures. The downsides are fairly limited; the loss in specificity for broad-scale and FDR procedures was relatively modest compared to the gains in power. Furthermore, the broad-scale methods we introduce are simple, fast, and easy to use, providing a straightforward starting point for researchers. This also points to the promise of more sophisticated broad-scale methods for not only functional connectivity but also related fields, including task-based activation. Altogether, this work demonstrates that shifting the scale of inference and choosing FDR control are both immediately attainable and can help remedy the issues with statistical power plaguing typical studies in the field.