The use of p-values in applied research: Interpretation and new trends

Donata Marasini, Piero Quatto, Enrico Ripamonti

Abstract


In this paper we consider a controversy on the use and interpretation of p-values in applied research. In recent years several applied and theoretical journals have started to discuss on the appropriate use of p-values in research fields such as Psychology, Ecology, and Medicine. First, the notion of p-value has some intrinsic limitations, which have been already highlighted in the statistical literature, but are far from being recognized in applied research. Second, it has emerged the so-called practice of p-hacking, which consists in analyzing and re-analyzing data until obtaining a significant result in terms of a p-value less than 0.05. In the light of these problems, we review two alternative theoretical frameworks, given by the use of Bayes factor and a recent proposal that leads to evaluate statistical hypotheses in terms of a priori and a posteriori odds ratios.

Keywords


p-value; Neyman-Pearson; Bayes factor; odds ratio; p-hacking

Full Text:

PDF (English)

References


M. BAYARRI, D. J. BENJAMIN, J. O. BERGER, T. M. SELLKE (2016). Rejection odds and rejection ratios: A proposal for statistical practice in testing hypotheses. Journal of Mathematical Psychology, 72, pp. 90-103.

M. BAYARRI, J. O. BERGER (1999). Quantifying surprise in the data and model verification. Bayesian Statistics, 6, pp. 53-82.

D. BENJAMIN, J. BERGER (2016). A simple alternative to p-values. The American Statistician, Online supplement, 70.

Y. BENJAMINI (2016). It's not the p-values fault. The American Statistician, Online supplement, 70.

K. S. BUTTON, J. P. IOANNIDIS, C. MOKRYSZ, B. A. NOSEK, J. FLINT, E. S. ROBINSON, M. R. MUNAFÒ (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, pp. 365-376.

J. COHEN (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

D. COLQUHOUN (2014). An investigation of the false discovery rate and the misinterpretation of p-values. Royal Society Open Science, 1, pp. 1-16.

G. Cumming (2013). The new statistics why and how. Psychological Science, 25, pp. 7-29.

S. GOODMAN (2008). A dirty dozen: twelve p-value misconceptions. Seminars in Hematology, 45, pp. 135-140.

S. GOODMAN (2016). The next questions: Who, what, when, where and why? The American Statistician, Online supplement, 70.

S. N. GOODMAN (1999). Toward evidence-based medical statistics. 2: The Bayes factor. Annals of Internal Medicine, 130, pp. 1005-1013.

S. N. GOODMAN (2001). Of p-values and bayes: a modest proposal. Epidemiology, 12, pp. 295-297.

S. GREENLAND, S. J. SENN, K. J. ROTHMAN, J. B. CARLIN, C. POOLE, S. N. GOODMAN, D. G. ALTMAN (2016). Statistical tests, p values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31, pp. 1-14.

J. P. IOANNIDIS (2014). Discussion: why an estimate of the science-wise false discovery rate and application to the top medical literature" is false. Biostatistics, 15, pp. 28-36.

V. JOHNSON (2016). Comments on the ASA statement on statistical significance and p-values and marginally significant p-values. The American Statistician, Online supplement, 70.

H. J. MOTULSKY (2015). Common misconceptions about data analysis and statistics. British Journal of Pharmacology, 172, pp. 2126-2132.

P. A. MURTAUGH (2014). In defense of p-values. Ecology, 95, pp. 611-617.

T. SELLKE, M. BAYARRI, J. O. BERGER (2001). Calibration of values for testing precise null hypotheses. The American Statistician, 55, pp. 62-71.

D. TRAFIMOW, M. MARKS (2015). Editorial. Basic and Applied Social Psychology, 37, pp. 1-2.

D. TRAFIMOW, M. MARKS (2016). Editorial. Basic and Applied Social Psychology, 38, pp. 1-2.

B. VIDGEN, T. YASSERI (2016). P-values: misunderstood and misused. pre-print Cornell University.

R. L. WASSERSTEIN, N. A. LAZAR (2016). The ASA's statement on p-values: context, process, and purpose. The American Statistician, 70, pp. 129-133.




DOI: 10.6092/issn.1973-2201/6439

Creative Commons License 

ISSN 1973-2201

Registration at Bologna Court Law no. 2476 on 24th February 1955 (in progress)

The journal is hosted and mantained by ABIS-AlmaDL [privacy]