The frequent insignificance of a “significant” p‐value

We describe diagnostic tests, the prosecutor's fallacy in the courtroom, and NHST, which involve inter-related conditional probabilities, to help clarify the meaning ofp-values, and discuss the enormous sampling variability, or unreliability, ofp-values. Finally, we use a cardiac surgical database and simulations to explore further issues involvingp-values. In clinical studies,p-values provide a poor summary of the observed treatment effect, whereas the three-number summary provided by effect estimates and confidence intervals is more informative and minimizes over-interpretation of a “significant” result.p-values are an unreliable measure of the strength of evidence; if used at all they give only, at best, a very rough guide to decision making. Researchers should adopt Open Science practices to improve the trustworthiness of research and, where possible, use estimation (three-number summaries) or other better techniques.
Source: Journal of Cardiac Surgery - Category: Cardiovascular & Thoracic Surgery Authors: Tags: REVIEW Source Type: research