Conference Object

Estimating the Performance of Predictive Models with Resampling Methods

Author(s) / Creator(s)

Pargent, Florian

Abstract / Description

Machine learning methodology is gaining popularity in psychology and other social sciences. One core element is how to estimate the accuracy of individual predictions when applying a predictive model to new observations in practice. In this talk, Dr. Florian Pargent will give an introduction on how such an evaluation of predictive performance is achieved by resampling methods like k-fold cross-validation. Specifically, he will demonstrate why such out-of-sample estimates of predictive performance are necessary when working with machine learning algorithms (e.g. Random Forests). Then, it will be discussed whether resampling methods should be more frequently used when evaluating linear regression models which are predominantly used in psychological science. Finally, Dr. Florian Pargent will outline common pitfalls in evaluating model performance, that can lead to gross overestimates of predictive accuracy in the literature when preprocessing steps like variable selection are not adequately combined with the resampling scheme.
Presentation within the frame of the ZPID Colloquium, 16 January 2019

Persistent Identifier

Date of first publication

2019-01-16

Is part of

ZPID-Kolloquium 2019, Trier, Germany

Publisher

ZPID (Leibniz Institute for Psychology Information)

Is version of

Citation

Pargent, F. (2019). Estimating the Performance of Predictive Models with Resampling Methods. PsychArchives. https://doi.org/10.23668/psycharchives.2354
  • Author(s) / Creator(s)
    Pargent, Florian
  • PsychArchives acquisition timestamp
    2019-01-22T12:56:56Z
  • Made available on
    2019-01-22T12:56:56Z
  • Date of first publication
    2019-01-16
  • Abstract / Description
    Machine learning methodology is gaining popularity in psychology and other social sciences. One core element is how to estimate the accuracy of individual predictions when applying a predictive model to new observations in practice. In this talk, Dr. Florian Pargent will give an introduction on how such an evaluation of predictive performance is achieved by resampling methods like k-fold cross-validation. Specifically, he will demonstrate why such out-of-sample estimates of predictive performance are necessary when working with machine learning algorithms (e.g. Random Forests). Then, it will be discussed whether resampling methods should be more frequently used when evaluating linear regression models which are predominantly used in psychological science. Finally, Dr. Florian Pargent will outline common pitfalls in evaluating model performance, that can lead to gross overestimates of predictive accuracy in the literature when preprocessing steps like variable selection are not adequately combined with the resampling scheme.
  • Abstract / Description
    Presentation within the frame of the ZPID Colloquium, 16 January 2019
  • Citation
    Pargent, F. (2019). Estimating the Performance of Predictive Models with Resampling Methods. PsychArchives. https://doi.org/10.23668/psycharchives.2354
    en
  • Persistent Identifier
    https://hdl.handle.net/20.500.12034/1986
  • Persistent Identifier
    https://doi.org/10.23668/psycharchives.2354
  • Language of content
    eng
    en_US
  • Publisher
    ZPID (Leibniz Institute for Psychology Information)
    en_US
  • Is part of
    ZPID-Kolloquium 2019, Trier, Germany
    en_US
  • Is version of
    https://osf.io/a8qbt/
  • Is related to
    https://doi.org/10.23668/psycharchives.2421
  • Dewey Decimal Classification number(s)
    150
  • Title
    Estimating the Performance of Predictive Models with Resampling Methods
    en_US
  • DRO type
    conferenceObject
    en_US
  • Visible tag(s)
    ZPID Conferences and Workshops