Accuracy of Effect Size Estimates from Published Psychological Research

Abstract

A Monte-Carlo simulation was used to model the biasing of effect sizes in published studies. The findings from the simulation indicate that, when a predominant bias to publish studies with statistically significant results is coupled with inadequate statistical power, there will be an overestimation of effect sizes. The consequences such an effect size overestimation will then have on meta-analyses and power analyses are highlighted and discussed along with measures which can be taken to reduce the problem.

Keywords

Monte Carlo methodStatistical powerSample size determinationPublication biasStatisticsPublicationMeta-analysisEconometricsComputer sciencePsychologyMathematicsMedicine

MeSH Terms

HumansPsychologyPublishingResearch

Affiliated Institutions

Related Publications

PLS, Small Sample Size, and Statistical Power in MIS Research

Dale L. Goodhue , William W. Lewis , Ron Thompson

There is a pervasive belief in the Management Information Systems (MIS) field that Partial Least Squares (PLS) has special abilities that make it more appropriate than other tec...

2006 299 citations

Modelling multiple sources of dissemination bias in meta‐analysis

Jack Bowden , Dan Jackson , Simon G. Thompson

Abstract Asymmetry in the funnel plot for a meta‐analysis suggests the presence of dissemination bias. This may be caused by publication bias through the decisions of journal ed...

2010 Statistics in Medicine 16 citations

How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power

Linda K. Muthén , Bengt Muthén

Abstract A common question asked by researchers is, "What sample size do I need for my study?" Over the years, several rules of thumb have been proposed. In reality there is no ...

2002 Structural Equation Modeling A Multid... 2142 citations

Effect size, confidence interval and statistical significance: a practical guide for biologists

Shinichi Nakagawa , Innes C. Cuthill

Abstract Null hypothesis significance testing (NHST) is the dominant statistical approach in biology, although it has many, frequently unappreciated, problems. Most importantly,...

2007 Biological reviews/Biological reviews... 3646 citations

A robust <i>P</i>‐value for treatment effect in meta‐analysis with publication bias

J. B. Copas , Paul Malley

Abstract Publication bias is a major and intractable problem in meta‐analysis. There have been several attempts in the literature to adapt methods to allow for such bias, but th...

2008 Statistics in Medicine 36 citations

Publication Info

Year: 2008
Type: article
Volume: 106
Issue: 2
Pages: 645-649
Citations: 54
Access: Closed

External Links

Download PDF (Free) View on DOI.org PubMed Semantic Scholar

Social Impact

Altmetric

Accuracy of Effect Size Estimates from Published Psychological Research

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

OpenAlex

Influential

CrossRef

Cite This

APA Style

                            
                                
                                    Andrew Brand, 
                                
                                    M. T. Bradley, 
                                
                                    Lisa A. Best
                                
                                et al.
                            
                            (2008). 
                            Accuracy of Effect Size Estimates from Published Psychological Research. 
                            Perceptual and Motor Skills
                            , 106
                            (2)
                            , 645-649.
                            https://doi.org/10.2466/pms.106.2.645-649
                        

Identifiers

DOI: 10.2466/pms.106.2.645-649
PMID: 18556917

Data Quality

Data completeness: 81%