Abstract
This article summarizes arguments against the use of power to analyze data, and illustrates a key pitfall: Lack of statistical significance (e.g., p > .05) combined with high power (e.g., 90%) can occur even if the data support the alternative more than the null. This problem arises via selective choice of parameters at which power is calculated, but can also arise if one computes power at a prespecified alternative. As noted by earlier authors, power computed using sample estimates ("observed power") replaces this problem with even more counterintuitive behavior, because observed power effectively double counts the data and increases as the P value declines. Use of power to analyze and interpret data thus needs more extensive discouragement.
Keywords
CounterintuitiveNull hypothesisStatistical powerNull (SQL)Power (physics)Sample size determinationValue (mathematics)StatisticsMedicinePower analysisEconometricsStatistical significancep-valueData miningComputer scienceMathematicsThermodynamicsPhysics
MeSH Terms
Data InterpretationStatisticalEpidemiologic Methods
Affiliated Institutions
Related Publications
Testing a Point Null Hypothesis: The Irreconcilability of<i>P</i>Values and Evidence
Abstract The problem of testing a point null hypothesis (or a "small interval" null hypothesis) is considered. Of interest is the relationship between the P value (or observed s...
Significance, Errors, Power, and Sample Size: The Blocking and Tackling of Statistics
Inferential statistics relies heavily on the central limit theorem and the related law of large numbers. According to the central limit theorem, regardless of the distribution o...
Some Methods for Strengthening the Common χ 2 Tests
Since the x2 tests of goodness of fit and of association in contingency tables are presented in many courses on statistical methods for beginners in the subject, it is not surpr...
Testing Precise Hypotheses
Testing of precise (point or small interval) hypotheses is reviewed, with special emphasis placed on exploring the dramatic conflict between conditional measures (Bayes factors ...
Wald's Test as Applied to Hypotheses in Logit Analysis
Abstract For tests of a single parameter in the binomial logit model, Wald's test is shown to behave in an aberrant manner. In particular, the test statistic decreases to zero a...
Publication Info
- Year
- 2012
- Type
- article
- Volume
- 22
- Issue
- 5
- Pages
- 364-368
- Citations
- 90
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
90
OpenAlex
3
Influential
56
CrossRef
Cite This
Sander Greenland
(2012).
Nonsignificance Plus High Power Does Not Imply Support for the Null Over the Alternative.
Annals of Epidemiology
, 22
(5)
, 364-368.
https://doi.org/10.1016/j.annepidem.2012.02.007
Identifiers
- DOI
- 10.1016/j.annepidem.2012.02.007
- PMID
- 22391267