What's the Matter with Significance?

Discussions about statistical significance are not usually found in newspapers, but the  Associated Press  recently hadsuch a discussion about the results of a clinical trial involving a heart drug. Statistical significance refers to whether a study finds a “real” effect or whether any differences measured are a result of chance. For example, in the case of the heart drug study, the authors attempt to measure whether the drug reduces patients’ mortality by comparing the mortality of patients on the drug to people not on the drug. The statistical significance reflects the authors’ confidence that the difference (reduced mortality) they find is not a fluke. As the article correctly states, “Significance is reflected in a calculation that produces something called a p-value. Usually, if this produces a p-value of less than 0.05, the study findings are considered significant. If not, the study has failed the test [i.e., the findings cannot be differentiated from random chance].”The heart drug study had a p-value of .059, meaning that the study ’s authors are 94.1 percent confident that the apparent lower mortality they found for patients on the drug than for those not on the drug is real. By the standard .05 p-value (95 percent confidence level) criterion, the study’s findings are not considered statistically significant.In all scientific research (including clinical trials), scientists must make a tradeoff between the likelihood of accepting a false finding bec...
Source: Cato-at-liberty - Category: American Health Authors: Source Type: blogs