Test-taking engagement and test-taking behavior: Modeling the processes underlying item nonresponse and guessing

• Applicants: Professorin Dr. Steffi Pohl; Dr. Jochen Ranger
• Subject Area: Personality Psychology, Clinical and Medical Psychology, Methodology
• Term: from 2015 to 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 288472689

Disengaged test-taking behavior poses a severe threat to the validity of results obtained from cognitive assessments since test scores may be confounded with the level of engagement. Typical forms of disengaged test-taking behavior are item nonresponse and guessing and researchers have made use of response times in addition to responses to identify these test-taking behaviors. Different forms of item nonresponse, which differ in their missing mechanism, have been identified. These include not reached items due to time limits, not reached items due to quitting on the test, and item omissions. In previous research, we have proposed models that allow modeling these different mechanisms. These models, however, have not, yet, been combined and make rather strong assumptions on stationarity, conditional independence, and response time distributions. In the current project, we aim for weakening the assumptions made in these models and for combining the models. This allows for application of the models in empirical studies. There has been a vast amount of literature on guessing. However, to date, the research on item nonresponse and guessing is rather distinct, although both may be indicators of the same motivational or cognitive status. Relying on our own preliminary work, we aim for combining the modeling of item nonresponse and guessing. Specifically, we will make use of cognitive models to gain a more thorough understanding of the process of disengaged responding and for distinguishing between engaged and disengaged item nonresponse and guessing. With this we extent previous approaches, in which item nonresponse and guessing have mainly been considered as disengaged responding. The parameters obtained in our models will be validated on external measures of engagement in an empirical study. Our work has implications for a) test construction, b) scoring of item responses, and c) for drawing conclusions about the ability of examinees.

• DFG Programme: Research Grants
• International Connection: Austria, USA
• Co-Investigator: Dr. Esther Ulitzsch
• Cooperation Partners: Professor Dr. Matthias von Davier; Privatdozent Dr. Timo Gnambs