Please use this identifier to cite or link to this item:
Full metadata record
DC FieldValueLanguage
dc.rights.licenseCC-BY-SA 4.0en_US
dc.contributor.authorErdfelder, Edgar-
dc.contributor.authorHeck, Daniel W.-
dc.identifier.citationErdfelder, E., & Heck, D. W. (2019, March 14). Detecting Evidential Value and P-Hacking With the P-curve tool: A Word of Caution. ZPID (Leibniz Institute for Psychology Information).
dc.description.abstractSimonsohn, Nelson, and Simmons (2014a) proposed p-curve – the distribution of statistically significant p-values for a set of studies – as a tool to assess the evidential value of these studies. They argued that, whereas right-skewed p-curves indicate true underlying effects, left-skewed p-curves indicate selective reporting of significant results from a much larger set of tests conducted on the same data when there is no true effect (“p-hacking”). We first review research that criticized the first claim by showing that null effects may indeed produce right-skewed p-curves under some conditions. We then question the second claim by showing that not only selective reporting but also selective non-reporting of significant results (e.g., of an ANCOVA for randomized 2-groups designs) due to a significant outcome of a more popular alternative test of the same hypothesis (e.g., a two-group t-test) may produce left-skewed p-curves, even if all studies included in a p-curve reflect true effects. Thus, although it is true that left-skewed p-curves indicate selection bias, it is possible that the bias is due to studies excluded from the p-curve rather than to those included in it. Hence, just as right-skewed p-curves do not necessarily imply evidential value, left-skewed p-curves do not necessarily imply p-hacking and absence of true effects in the studies involved.en_US
dc.publisherZPID (Leibniz Institute for Psychology Information)en_US
dc.relation.ispartofOpen Science 2019, Trier, Germanyen_US
dc.titleDetecting Evidential Value and P-Hacking With the P-curve tool: A Word of Cautionen_US
Appears in Collections:Conference Object

Files in This Item:
File Description SizeFormat 
m_1_Erdfelder Open Science Conference Trier 2019.pdfConference Talk2,04 MBAdobe PDF Preview PDF Download

This item is licensed under a Creative Commons License Creative Commons