Appraisal of the methodological quality of systematic reviews on pharmacological and psychological interventions for major depression in adults using the AMSTAR 2

Trial registration number: International Prospective Register of Systematic Reviews (PROSPERO) registration number: CRD42018110214. Background: Major depression is a common mental disorder with high prevalence and mortality. There is a high need for reliable and summarized information for clinicians as well as policy makers in the field. Whereas systematic reviews should provide a comprehensive and objective appraisal of evidence, poor reporting and flaws in methodological quality are often and impair the reliability of conclusions. Several standards have been developed to assess methodological quality of SR [2], widely used is the AMSTAR (A Measurement Tool to Assess SR, published in 2007) with 11 items. Recently, an updated version of AMSTAR - AMSTAR 2 [1] has been published, which allows a more detailed evaluation of SR in 16 items and the rating of the overall confidence in the results of the review. Objectives: The present study will determine the methodological quality of SR in the treatment of adult major depression using the new AMSTAR 2 and identify potential predictive factors associated with the quality. To reflect the current quality we focus on SR published in the years of 2012 to 2017. Methods: We conducted electronic searches in August 2017 in the bibliographic databases MEDLINE, EMBASE and the Cochrane Database of SR. We used a combination of Mesh terms and keywords to identify SR from 2012 to 2017 referring to the topic “Major Depression” and did not apply any restrictions on language or countries. Two authors independently screened the titles, abstracts and full texts of the retrieved literature to assess their eligibility according the a priori defined criteria and coded the bibliographic characteristics (e.g. corresponding author’s original region, number of authors, Journal impact factor at year of publication) onto a data collection template in EXCEL. All selected SR were appraised after a calibration phase with AMSTAR 2 by four independent evaluators. Each evaluator appraised 30 SR. A consensus for each of the 16 items was reached with majority rule. Furthermore, the rating of the overall confidence in the results of the review was performed with the critical domains as recommended by Shea et al. 2017 [1]. This was done by two evaluators independently. Any discrepancies were resolved through discussions. To assess whether the intervention (pharmacological or psychological interventions), the type of review (Cochrane vs. non-Cochrane reviews), and/or Open Access status (yes vs. no) are associated with AMSTAR 2 scores, a sub-analysis of AMSTAR 2 scores will be performed. The associations between bibliographical characteristics and scoring on AMSTAR 2 items will be analysed using multivariate logistic regression or multi-nominal logistic regression depending on the scaling of the dependent variable. Results: The electronic literature search detected 1,524 citations. 72 SR comprising 30 SRs with psychological and 42 SRs with pharmacological interventions met our eligibility criteria. 30 out of 42 pharmacological SRs were randomly drawn and served together with the identified 30 psychotherapeutic SRs as sample for this study. Of the 60 SR evaluated in our sample, 42 SR included only randomized trials and 18 SR additionally non-randomized studies. Four out of the 60 SR were Cochrane Reviews. In four domains of AMSTAR 2 (item 1, 6, 14, 16) the majority (more than 50%) of the SR scores “yes”. The results according to all AMSTAR 2 domains are shown in figure 1. Figure 1: Methodological quality of 60 SR according to the 16 items of AMSTAR 2; yes: white colour, partially yes: light grey, no: dark grey, no meta-analysis (MA) conducted: striped. In rating overall confidence in the results of the SR only four reviews were considered as “high” (three of them Cochrane Reviews), two as “moderate”, one as “low” and 53 as “critically low”. The analysis of subgroups and the evaluation of the predictors is currently in progress. Conclusions and implications: According to AMSTAR 2 the overall methodological quality of our current and representative sample of SR on pharmacological and psychological interventions for major depression in adults is disappointing. In almost 90% of the sample of SR, overall confidence in the results of the SR was considered to be “critically low”, so the use of findings of these reviews should be limited. Although there is a high need for reliable and summarized information for clinicians as well as policy makers in the field of mental disorders, this work demonstrates the need to critically assess SR before using their findings. Possible suggestions for the improvement of the quality of SR are the following: Authors of future SR should establish review methods prior to the conduct of the review, give an explanation for study designs included in the review, use a satisfactory technique for assessing the risk of bias as well as publication bias, use appropriate meta-analytic methods, and consider the scientific quality when formulating conclusions. References: [1] Shea et al. (2017). BMJ, 358, j4008. [2] Zeng et al. (2015). J of Evidence‐Based Medicine, 8: 2-10.