Article Version of Record

Does preregistration improve the credibility of research findings?

Author(s) / Creator(s)

Rubin, Mark

Abstract / Description

Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document before they undertake their data collection and analyses. This document is then made available with the published research report to allow readers to identify discrepancies between what the researchers originally planned to do and what they actually ended up doing. This historical transparency is supposed to facilitate judgments about the credibility of the research 1ndings. The present article provides a critical review of 17 of the reasons behind this argument. The article covers issues such as HARKing, multiple testing, p-hacking, forking paths, optional stopping, researchers’ biases, selective reporting, test severity, publication bias, and replication rates. It is concluded that preregistration’s historical transparency does not facilitate judgments about the credibility of research 1ndings when researchers provide contemporary transparency in the form of (a) clear rationales for current hypotheses and analytical approaches, (b) public access to research data, materials, and code, and (c) demonstrations of the robustness of research conclusions to alternative interpretations and analytical approaches.

Keyword(s)

preregistration forking paths HARKing multiple testing optional stopping p-hacking publication bias replication crisis

Persistent Identifier

Date of first publication

2020-09-18

Journal title

The Quantitative Methods for Psychology

Volume

16

Issue

4

Page numbers

376-390

Publisher

PsychArchives

Publication status

publishedVersion

Review status

peerReviewed

Citation

Rubin, M. (2020). Does preregistration improve the credibility of research findings? The Quantitative Methods for Psychology, 16(4), 376–390. http://dx.doi.org/10.23668/psycharchives.4839
  • Author(s) / Creator(s)
    Rubin, Mark
  • PsychArchives acquisition timestamp
    2021-05-17T07:03:25Z
  • Made available on
    2021-05-17T07:03:25Z
  • Date of first publication
    2020-09-18
  • Abstract / Description
    Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document before they undertake their data collection and analyses. This document is then made available with the published research report to allow readers to identify discrepancies between what the researchers originally planned to do and what they actually ended up doing. This historical transparency is supposed to facilitate judgments about the credibility of the research 1ndings. The present article provides a critical review of 17 of the reasons behind this argument. The article covers issues such as HARKing, multiple testing, p-hacking, forking paths, optional stopping, researchers’ biases, selective reporting, test severity, publication bias, and replication rates. It is concluded that preregistration’s historical transparency does not facilitate judgments about the credibility of research 1ndings when researchers provide contemporary transparency in the form of (a) clear rationales for current hypotheses and analytical approaches, (b) public access to research data, materials, and code, and (c) demonstrations of the robustness of research conclusions to alternative interpretations and analytical approaches.
    en
  • Publication status
    publishedVersion
    en
  • Review status
    peerReviewed
    en
  • External description on another website
    https://sites.google.com/site/markrubinsocialpsychresearch/replication-crisis/preregistration-criticisms
  • Citation
    Rubin, M. (2020). Does preregistration improve the credibility of research findings? The Quantitative Methods for Psychology, 16(4), 376–390. http://dx.doi.org/10.23668/psycharchives.4839
    en
  • ISSN
    2292-1354
  • Persistent Identifier
    https://hdl.handle.net/20.500.12034/4276
  • Persistent Identifier
    https://doi.org/10.23668/psycharchives.4839
  • Language of content
    eng
  • Publisher
    PsychArchives
    en
  • Keyword(s)
    preregistration
    en
  • Keyword(s)
    forking paths
    en
  • Keyword(s)
    HARKing
    en
  • Keyword(s)
    multiple testing
    en
  • Keyword(s)
    optional stopping
    en
  • Keyword(s)
    p-hacking
    en
  • Keyword(s)
    publication bias
    en
  • Keyword(s)
    replication crisis
    en
  • Dewey Decimal Classification number(s)
    150
  • Title
    Does preregistration improve the credibility of research findings?
    en
  • DRO type
    article
    en
  • Leibniz subject classification
    Psychologie
    de_DE
  • Issue
    4
  • Journal title
    The Quantitative Methods for Psychology
    en
  • Page numbers
    376-390
  • Volume
    16
  • Visible tag(s)
    Version of Record
  • Visible tag(s)
    preregistration
    en
  • Visible tag(s)
    forking paths
    en
  • Visible tag(s)
    HARKing
    en
  • Visible tag(s)
    multiple testing
    en
  • Visible tag(s)
    optional stopping
    en
  • Visible tag(s)
    p-hacking
    en
  • Visible tag(s)
    publication bias
    en
  • Visible tag(s)
    replication crisis
    en