Conference Object

What the Future Holds: Predicting Psychotherapy Outcome with Machine Learning

Author(s) / Creator(s)

Taubitz, Friedrich-Samuel
Büdenbender, Björn
Alpers, Georg W.

Other kind(s) of contributor

University of Mannheim

Abstract / Description

(a) Background: Mental disorders cause substantial costs for the individual and our society. The effectiveness of psychotherapy to lessen these patients suffering is well documented but, unfortunately, not all patients benefit to the same degree. Machine learning (ML) may be useful to predict therapy outcomes for individual patients (i.e., a useful practical perspective) and help to identify relevant predictors for successful treatments (i.e., this is an important theoretical perspective). ML could thus contribute further to our understanding of the mechanisms of effective psychotherapy. However, so far, there are only very few applications of ML in psychotherapy research. These are typically either based on small sample sizes or focused on specific diagnoses (e.g., schizophrenia). (b) Objectives: We aimed to examine ML models' ability to predict the outcome of cognitive behavioral therapy (CBT) on an individual level using a sufficiently large and heterogeneous sample with a variety of diagnoses in an ecologically valid outpatient setting. (c) Research questions: Of particular interest was identifying the ML algorithm that is best suited to predict the outcome of psychotherapy in a naturalistic setting and identifying variables that are most important for the prediction. (d) Method/Approach: We used data from N = 685 patients with diverse disorders from an outpatient center. Available were data on sociodemographic and clinical baseline information from routine assessments at the outset of CBT. We trained several ML models (Decision-Trees, Random-Forest & Gradient Boosting Machines [GBM]) to predict treatment success. Success was defined as a clinical significant change (CSC) in the perceived symptom severity after treatment, measured with the Brief-Symptom-Checklist. To identify the most relevant predictor variables, we combined internal feature selection approaches with the BORUTA algorithm. (e) Results/Findings: All models significantly outperformed the no-information rate on previously unseen validation data. The best performing model (GBM) achieved a balanced accuracy of 70%. Out of 405 variables, we identified the 16 most important predictors, which were sufficient to predict CSC with 67% balanced accuracy. Exploratively, the predictors can be summarized into six substantive factors: high baseline severity, paranoid thinking or ideations, prior mental disorders, functionality, somatization, and passive-aggressive traits. (f) Conclusions and implications: Our study demonstrates that data which is typically available at the outset of therapy can predict whether an individual will substantially benefit from the intervention to a clinically relevant degree. Some of these predictors were predicted (e.g., level of functioning), but other factors need further validation (e.g., job-impairment). The 16 most important predictors almost perfectly replicated the fully specified model's predictive performance. The comparably small set of predictors' predictive power and the fact that these predictors were already identified in different studies underline the importance of data-driven approaches. ML is clearly an attractive extension to more established psychotherapy research methodology from a theoretical and practical perspective.

Persistent Identifier

Date of first publication

2021-05-18

Is part of

Research Synthesis & Big Data, 2021, online

Publisher

ZPID (Leibniz Institute for Psychology)

Citation

Taubitz, F.-S., Büdenbender, B., & Alpers, G. W. (2021). What the Future Holds: Predicting Psychotherapy Outcome with Machine Learning. ZPID (Leibniz Institute for Psychology). https://doi.org/10.23668/PSYCHARCHIVES.4842
  • Author(s) / Creator(s)
    Taubitz, Friedrich-Samuel
  • Author(s) / Creator(s)
    Büdenbender, Björn
  • Author(s) / Creator(s)
    Alpers, Georg W.
  • Other kind(s) of contributor
    University of Mannheim
    en
  • PsychArchives acquisition timestamp
    2021-05-17T14:34:06Z
  • Made available on
    2021-05-17T14:34:06Z
  • Date of first publication
    2021-05-18
  • Abstract / Description
    (a) Background: Mental disorders cause substantial costs for the individual and our society. The effectiveness of psychotherapy to lessen these patients suffering is well documented but, unfortunately, not all patients benefit to the same degree. Machine learning (ML) may be useful to predict therapy outcomes for individual patients (i.e., a useful practical perspective) and help to identify relevant predictors for successful treatments (i.e., this is an important theoretical perspective). ML could thus contribute further to our understanding of the mechanisms of effective psychotherapy. However, so far, there are only very few applications of ML in psychotherapy research. These are typically either based on small sample sizes or focused on specific diagnoses (e.g., schizophrenia). (b) Objectives: We aimed to examine ML models' ability to predict the outcome of cognitive behavioral therapy (CBT) on an individual level using a sufficiently large and heterogeneous sample with a variety of diagnoses in an ecologically valid outpatient setting. (c) Research questions: Of particular interest was identifying the ML algorithm that is best suited to predict the outcome of psychotherapy in a naturalistic setting and identifying variables that are most important for the prediction. (d) Method/Approach: We used data from N = 685 patients with diverse disorders from an outpatient center. Available were data on sociodemographic and clinical baseline information from routine assessments at the outset of CBT. We trained several ML models (Decision-Trees, Random-Forest & Gradient Boosting Machines [GBM]) to predict treatment success. Success was defined as a clinical significant change (CSC) in the perceived symptom severity after treatment, measured with the Brief-Symptom-Checklist. To identify the most relevant predictor variables, we combined internal feature selection approaches with the BORUTA algorithm. (e) Results/Findings: All models significantly outperformed the no-information rate on previously unseen validation data. The best performing model (GBM) achieved a balanced accuracy of 70%. Out of 405 variables, we identified the 16 most important predictors, which were sufficient to predict CSC with 67% balanced accuracy. Exploratively, the predictors can be summarized into six substantive factors: high baseline severity, paranoid thinking or ideations, prior mental disorders, functionality, somatization, and passive-aggressive traits. (f) Conclusions and implications: Our study demonstrates that data which is typically available at the outset of therapy can predict whether an individual will substantially benefit from the intervention to a clinically relevant degree. Some of these predictors were predicted (e.g., level of functioning), but other factors need further validation (e.g., job-impairment). The 16 most important predictors almost perfectly replicated the fully specified model's predictive performance. The comparably small set of predictors' predictive power and the fact that these predictors were already identified in different studies underline the importance of data-driven approaches. ML is clearly an attractive extension to more established psychotherapy research methodology from a theoretical and practical perspective.
    en
  • Publication status
    unknown
    en
  • Review status
    unknown
    en
  • Citation
    Taubitz, F.-S., Büdenbender, B., & Alpers, G. W. (2021). What the Future Holds: Predicting Psychotherapy Outcome with Machine Learning. ZPID (Leibniz Institute for Psychology). https://doi.org/10.23668/PSYCHARCHIVES.4842
    en
  • Persistent Identifier
    https://hdl.handle.net/20.500.12034/4279
  • Persistent Identifier
    https://doi.org/10.23668/psycharchives.4842
  • Language of content
    eng
  • Publisher
    ZPID (Leibniz Institute for Psychology)
    en
  • Is part of
    Research Synthesis & Big Data, 2021, online
    en
  • Dewey Decimal Classification number(s)
    150
  • Title
    What the Future Holds: Predicting Psychotherapy Outcome with Machine Learning
    en
  • DRO type
    conferenceObject
    en
  • Visible tag(s)
    ZPID Conferences and Workshops