Reproducibility-related Reporting Practices in Meta-analyses on the Effectiveness of Psychological Interventions
Author(s) / Creator(s)
Lopez-Nicolas, Ruben
Sanchez-Meca, Julio
Lopez-Lopez, Jose Antonio
Rubio-Aparicio, Maria
Abstract / Description
Background: The last decade has revealed significant problems in terms of reproducibility in psychological research. The scientific psychology community has detected various errors and questionable research practices in the process of studying psychological phenomena. In this context, different proposals have been put forward in order to improve transparency and reproducibility practices. In the present study, we focus on meta-analytic methodology. Meta-analysis is a powerful and important tool to synthesize the literature about a research topic. Like other kinds of research, meta-analyses must be reproducible to be compliant with the principles of the scientific method. Furthermore, reproducible meta-analyses can be easily updated with new data or reanalysed applying new and more refined analysis techniques. Objetives: We attempted to empirically assess the prevalence of reproducibility-related methodological reporting practices in published meta-analyses of psychological interventions. Our purpose was to identify the key points that could be improved with the aim to provide some recommendations to carry out reproducible meta-analyses. Research question: Are the meta-analytical reports comprehensive enough to allow for potential reproduction attempts? What are the key points where there is a lack of reporting? Method: We conducted a meta-review of meta-analyses of psychological interventions published between 2000 and 2020. We searched PubMed, PsycInfo and Web of Science databases. Then, we selected a random sample of 100 meta-analyses to examine and extract a range of transparency and reproducibility-related indicators using an ad hoc checklist based on existing meta-analysis guidelines. We collected data concerning the systematic review methods (identification and selection of studies and data collection process) and the meta-analysis methods (effect measures handling and synthesis and analysis methods). Results: Major issues concerning completely reproducible search procedures report, specification of exact methods to compute effect sizes and specification of certain elements of the model used were found. Conclusion and implications: We will use the findings from our meta-review to articulate some recommendations intended to improve the transparency, openness and reproducibility-related reporting practices of meta-analyses in clinical psychology and related areas.
Keyword(s)
Replicability Meta-analysis Transparency Reproducibility Meta-sciencePersistent Identifier
Date of first publication
2021-05-21
Is part of
Research Synthesis & Big Data, 2021, online
Publisher
ZPID (Leibniz Institute for Psychology)
Citation
Lopez-Nicolas, R., Sanchez-Meca, J., Lopez-Lopez, J. A., & Rubio-Aparicio, M. (2021). Reproducibility-related Reporting Practices in Meta-analyses on the Effectiveness of Psychological Interventions. ZPID (Leibniz Institute for Psychology). https://doi.org/10.23668/PSYCHARCHIVES.4823
-
Lopez-Nicolas_RSC_slides.pdfAdobe PDF - 482.5KBMD5: 4b614825a146a0ef98a9993e023f93b7Description: Conference paper slides
-
There are no other versions of this object.
-
Author(s) / Creator(s)Lopez-Nicolas, Ruben
-
Author(s) / Creator(s)Sanchez-Meca, Julio
-
Author(s) / Creator(s)Lopez-Lopez, Jose Antonio
-
Author(s) / Creator(s)Rubio-Aparicio, Maria
-
PsychArchives acquisition timestamp2021-05-14T11:36:44Z
-
Made available on2021-05-14T11:36:44Z
-
Date of first publication2021-05-21
-
Abstract / DescriptionBackground: The last decade has revealed significant problems in terms of reproducibility in psychological research. The scientific psychology community has detected various errors and questionable research practices in the process of studying psychological phenomena. In this context, different proposals have been put forward in order to improve transparency and reproducibility practices. In the present study, we focus on meta-analytic methodology. Meta-analysis is a powerful and important tool to synthesize the literature about a research topic. Like other kinds of research, meta-analyses must be reproducible to be compliant with the principles of the scientific method. Furthermore, reproducible meta-analyses can be easily updated with new data or reanalysed applying new and more refined analysis techniques. Objetives: We attempted to empirically assess the prevalence of reproducibility-related methodological reporting practices in published meta-analyses of psychological interventions. Our purpose was to identify the key points that could be improved with the aim to provide some recommendations to carry out reproducible meta-analyses. Research question: Are the meta-analytical reports comprehensive enough to allow for potential reproduction attempts? What are the key points where there is a lack of reporting? Method: We conducted a meta-review of meta-analyses of psychological interventions published between 2000 and 2020. We searched PubMed, PsycInfo and Web of Science databases. Then, we selected a random sample of 100 meta-analyses to examine and extract a range of transparency and reproducibility-related indicators using an ad hoc checklist based on existing meta-analysis guidelines. We collected data concerning the systematic review methods (identification and selection of studies and data collection process) and the meta-analysis methods (effect measures handling and synthesis and analysis methods). Results: Major issues concerning completely reproducible search procedures report, specification of exact methods to compute effect sizes and specification of certain elements of the model used were found. Conclusion and implications: We will use the findings from our meta-review to articulate some recommendations intended to improve the transparency, openness and reproducibility-related reporting practices of meta-analyses in clinical psychology and related areas.en
-
Publication statusunknownen
-
Review statusunknownen
-
SponsorshipThis research has been funded with a grant from the Ministerio de Ciencia e Innovación and by FEDER funds (Project nº PID2019-104080GB-I00).en
-
CitationLopez-Nicolas, R., Sanchez-Meca, J., Lopez-Lopez, J. A., & Rubio-Aparicio, M. (2021). Reproducibility-related Reporting Practices in Meta-analyses on the Effectiveness of Psychological Interventions. ZPID (Leibniz Institute for Psychology). https://doi.org/10.23668/PSYCHARCHIVES.4823en
-
Persistent Identifierhttps://hdl.handle.net/20.500.12034/4260
-
Persistent Identifierhttps://doi.org/10.23668/psycharchives.4823
-
Language of contenteng
-
PublisherZPID (Leibniz Institute for Psychology)en
-
Is part ofResearch Synthesis & Big Data, 2021, onlineen
-
Keyword(s)Replicabilityen
-
Keyword(s)Meta-analysisen
-
Keyword(s)Transparencyen
-
Keyword(s)Reproducibilityen
-
Keyword(s)Meta-scienceen
-
Dewey Decimal Classification number(s)150
-
TitleReproducibility-related Reporting Practices in Meta-analyses on the Effectiveness of Psychological Interventionsen
-
DRO typeconferenceObjecten
-
Visible tag(s)ZPID Conferences and Workshops