MetaLab and metalabR: Facilitating dynamic meta-analyses in developmental psychology

Background: Developmental psychologists often make statements of the form “babies learn to do X at age Y”. Yet summaries made on the basis of one or a few studies can misrepresent a messy and complex evidence base. True results may also not be generalizable outside of the specific testing context for theoretically important reasons including the language or age of the infants. Other factors that are typically considered "noise" might also impede generalizability, including the lab where testing takes place, the type of stimuli, or the methods that were used. Systematic reviews and meta-analyses help bring coherence to a complex evidence base, control for confounds, and increase confidence in results. However, meta-analyses are underused in developmental research. Objective: To facilitate developmental researchers’ access to current meta-analyses, we created MetaLab (metalab.stanford.edu), a platform for open, dynamic meta-analytic datasets. In 6 years, the site has grown to 30 meta-analyses with data from 45,000 infants and children. A key feature is the standardized data storage format, which allows a unified framework for analysis and visualization. This facilitates the addition of new data points, resulting in community-augmented meta-analyses (CAMAs; Tsuji, Bergmann, & Cristia, 2014), which provide the most up-to-date summary of the body of literature. Use of this standardized format is facilitated by tailored documentation and our metalabR package, as follows. Method: The MetaLab website hosts tutorials for conducting a systematic review and quantitative synthesis. In progress are tutorials for researchers who are planning on conducting, or who have conducted, a study on a topic where a MetaLab CAMA already exists. These tutorials will help researchers decide on their methodology and sample size, based on the collated body of evidence, and to add their results to the CAMA when their study is complete. Currently in development, our new R package, metalabR, facilitates and standardizes the process of conducting and integrating meta-analyses with the MetaLab platform. Existing key features focus on ensuring adherence to our data format by providing functions for reading, validating, and cleaning new datasets and added data points, and calculating standardized effect sizes. Multiple datasets can be read into one dataframe to allow for meta-meta-analysis across different topics. MetalabR helps one access existing MetaLab functionalities for quantitative analysis, building on metafor (Viechtbauer, 2007) to run inverse-variance weighted multivariate meta-analytic models with random effects. The package also includes functions for data visualization, building on ggplot2 (Wickham, 2016) to create scatter, forest, funnel and violin plots, all with a standardized theme and colour scheme. In progress is a function for generating a summary report of results of random effects models appropriate for meta-analyses in experimental developmental psychology. In addition to using the metalabR package in R, various shinyapps can be accessed from the MetaLab website, which have a user-friendly interface and require no knowledge of R or coding. These are tools for visualizing MetaLab data, conducting power analyses and power simulations, and data validation tools to ensure a reviewer’s planned meta-analysis fits the MetaLab structure and format, so their data can eventually be integrated into MetaLab. Outlook: MetaLab facilitates the adoption of transparent, reproducible, and dynamic meta-analyses in developmental psychology along multiple dimensions. It contains tools for conducting a new meta-analysis, planning a study, contributing to an existing meta-analysis, and conducting a meta-meta-analysis across multiple meta-analyses. Our functionalities and interfaces are also easily adaptable for use outside of the field of developmental psychology, with one spinoff having been created for evidence synthesis of vocal patterns in neuropsychiatric conditions (Nyholm Jensen & Dwenger, 2020, metavoice.au.dk). We will continue to develop MetaLab and metalabR to accommodate different types of data, and interface with standards such as Brain Imaging Data Structure (BIDS, Gorgolewski et al., 2016) for neuroimaging datasets, thereby further broadening the scope and utility of MetaLab.