Validating (easy) measures to combine speed and accuracy

• Applicants: Professor Dr. Markus Janczyk; Privatdozent Dr. Heinrich René Liesefeld
• Subject Area: General, Cognitive and Mathematical Psychology
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 523708317

The major share of experimental psychological studies produces two performance measures: the speed (mean response times, mean RTs) and accuracy (percentage correct, PC) of pressing one of two buttons (two-alternative forced choice tasks, 2AFC). It is not always clear, which of the two dependent measures is most relevant and should therefore be entered into statistical analyses and whether it is safe to treat the respective other measure as incidental. Problematically, there exists a strong, non-arbitrary relationship between mean RTs and PCs in that increasing speed usually comes with a corresponding decrease in accuracy. Combined measures aim to control for this speed-accuracy tradeoff (SAT), in order to reflect “true” performance. Several measures that combine both performance aspects have been suggested in the past and are heavily used in research. The present project aims to test their validity and scope. This will be done via large-scale simulations using established computational decision models and purpose-collected empirical data with experimental manipulations of performance and SAT levels. The aim is to identify (and further develop) those measures that effectively control for SATs and best reflect overall performance on 2AFC tasks and thereby to provide guidance for future experimental psychological studies with regard to the choice of dependent measures. One potential outcome is that none of the examined measures proofs suitable for this purpose. WP 1 will focus on pairwise within-participants comparisons on data simulated with the drift-diffusion model (because between-participants designs were covered by our already published study). WP 2 will use various models and model versions that implement various mechanisms that have been suggested to produce SATs to examine whether our conclusions generalize to other data-producing algorithms. WP 3 will develop experimental approaches to induced desired levels of SATs empirically and produce a large-scale benchmark data set. WP 4 will take a closer look on difficulties that have emerged in earlier WPs and try to resolve these. If applicable, WP 4 will also further develop any promising combined performance measures.

• DFG Programme: Research Grants