The Perils of Misinterpreting and Misusing “Publication Bias” in Meta-analyses: An Education Review on Funnel Plot-Based Methods

AbstractPublication bias refers to a systematic deviation from the truth in the results of a meta-analysis due to the higher likelihood for published studies to be included in meta-analyses than unpublished studies. Publication bias can lead to misleading recommendations for decision and policy making. In this education review, we introduce, explain, and provide solutions to the pervasive misuses and misinterpretations of publication bias that afflict evidence syntheses in sport and exercise medicine, with a focus on the commonly used funnel-plot based methods. Publication bias is more routinely assessed by visually inspecting funnel plot asymmetry, although it has been consistently deemed unreliable, leading to the development of statistical tests to assess publication bias. However, most statistical tests of publication bias (i) cannot rule out alternative explanations for funnel plot asymmetry (e.g., between-study heterogeneity, choice of metric, chance) and (ii) are grossly underpowered, even when using an arbitrary minimum threshold of ten or more studies. We performed a cross-sectional meta-research investigation of how publication bias was assessed in systematic reviews with meta-analyses published in the top two sport and exercise medicine journals throughout 2021. This analysis highlights that publication bias is frequently misused and misinterpreted, even in top tier journals. Because of conceptual and methodological problems when assessing and interpreting publicat...
Source: Sports Medicine - Category: Sports Medicine Source Type: research