Why p values can ’ t tell you what you need to know and what to do about it

Conclusions in the 2017 paper).  If you were willing to assume a 50:50 prior chance of there being a real effect the p = 0.005 would correspond to FPR50 = 0.034, which sounds satisfactory (from Table, above, or web calculator). But if, for example, you are testing a hypothesis about teleportation or mind-reading or homeopathy then you probably wouldn’t be willing to give a prior of 50% to that being right before the experiment. If the prior  probability of there being a real effect were 0.1, rather than 0.5, the Table above shows that observation of p = 0.005 would suggest, in my example, FPR = 0.24 and a 24%  risk of a false positive would still be disastrous.  In this case you would have to have observed p = 0.00043 in order to reduce the false positive risk to 0.05.  So no fixed p value threshold will cope adequately with every problem. Links For up-to-date links to the web calculator, and to papers, start at http://www.onemol.org.uk/?page_id=456 Colquhoun, 2014, An investigation of the false discovery rate and themisinterpretation of p-valueshttps://royalsocietypublishing.org/doi/full/10.1098/rsos.140216 Colquhoun, 2017, The reproducibility of research and the misinterpretationof p-values https://royalsocietypublishing.org/doi/10.1098/rsos.171085 Colquhoun, 2019, The False Positive Risk: A Proposal Concerning What to Do About p-Valueshttps://www.tandfonline.com/doi/full/10.1080/00031305.2018.1529622 Benjamin & Berger, Thre...
Source: DC's goodscience - Category: Science Authors: Tags: Uncategorized P values seminars statistics Source Type: blogs