Standard

Model meets reality : Validating a new behavioral measure for test-taking effort. / Ulitzsch, Esther; Penk, Christiane; von Davier, Matthias; Pohl, Steffi.

In: Educational Assessment, Vol. 26, No. 2, 05.2021, p. 104-124.

Research output: Contribution to journalJournal articleResearchpeer-review

Harvard

Ulitzsch, E, Penk, C, von Davier, M & Pohl, S 2021, 'Model meets reality: Validating a new behavioral measure for test-taking effort', Educational Assessment, vol. 26, no. 2, pp. 104-124. https://doi.org/10.1080/10627197.2020.1858786

APA

Ulitzsch, E., Penk, C., von Davier, M., & Pohl, S. (2021). Model meets reality: Validating a new behavioral measure for test-taking effort. Educational Assessment, 26(2), 104-124. https://doi.org/10.1080/10627197.2020.1858786

Vancouver

Ulitzsch E, Penk C, von Davier M, Pohl S. Model meets reality: Validating a new behavioral measure for test-taking effort. Educational Assessment. 2021 May;26(2):104-124. https://doi.org/10.1080/10627197.2020.1858786

Author

Ulitzsch, Esther ; Penk, Christiane ; von Davier, Matthias ; Pohl, Steffi. / Model meets reality : Validating a new behavioral measure for test-taking effort. In: Educational Assessment. 2021 ; Vol. 26, No. 2. pp. 104-124.

BibTeX

@article{ff11d5818f96407dbbe14df31d29bcf4,
title = "Model meets reality: Validating a new behavioral measure for test-taking effort",
abstract = "Identifying and considering test-taking effort is of utmost importance for drawing valid inferences on examinee competency in low-stakes tests. Different approaches exist for doing so. The speed-accuracy+engagement model aims at identifying non-effortful test-taking behavior in terms of nonresponse and rapid guessing based on responses and response times. The model allows for identifying rapid-guessing behavior on the item-by-examinee level whilst jointly modeling the processes underlying rapid guessing and effortful responding. To assess whether the model indeed provides a valid measure of test-taking effort, we investigate (1) convergent validity with previously developed behavioral as well as self-report measures on guessing behavior and effort, (2) fit within the nomological network of test-taking motivation derived from expectancy-value theory, and (3) ability to detect differences between groups that can be assumed to differ in test-taking effort. Results suggest that the model captures central aspects of non-effortful test-taking behavior. While it does not cover the whole spectrum of non-effortful test-taking behavior, it provides a measure for some aspects of it, in a manner that is less subjective than self-reports. The article concludes with a discussion of implications for the development of behavioral measures of non-effortful test-taking behavior.",
author = "Esther Ulitzsch and Christiane Penk and {von Davier}, Matthias and Steffi Pohl",
year = "2021",
month = may,
doi = "10.1080/10627197.2020.1858786",
language = "English",
volume = "26",
pages = "104--124",
journal = "Educational Assessment",
issn = "1062-7197",
publisher = "Routledge",
number = "2",

}

RIS

TY - JOUR

T1 - Model meets reality

T2 - Validating a new behavioral measure for test-taking effort

AU - Ulitzsch, Esther

AU - Penk, Christiane

AU - von Davier, Matthias

AU - Pohl, Steffi

PY - 2021/5

Y1 - 2021/5

N2 - Identifying and considering test-taking effort is of utmost importance for drawing valid inferences on examinee competency in low-stakes tests. Different approaches exist for doing so. The speed-accuracy+engagement model aims at identifying non-effortful test-taking behavior in terms of nonresponse and rapid guessing based on responses and response times. The model allows for identifying rapid-guessing behavior on the item-by-examinee level whilst jointly modeling the processes underlying rapid guessing and effortful responding. To assess whether the model indeed provides a valid measure of test-taking effort, we investigate (1) convergent validity with previously developed behavioral as well as self-report measures on guessing behavior and effort, (2) fit within the nomological network of test-taking motivation derived from expectancy-value theory, and (3) ability to detect differences between groups that can be assumed to differ in test-taking effort. Results suggest that the model captures central aspects of non-effortful test-taking behavior. While it does not cover the whole spectrum of non-effortful test-taking behavior, it provides a measure for some aspects of it, in a manner that is less subjective than self-reports. The article concludes with a discussion of implications for the development of behavioral measures of non-effortful test-taking behavior.

AB - Identifying and considering test-taking effort is of utmost importance for drawing valid inferences on examinee competency in low-stakes tests. Different approaches exist for doing so. The speed-accuracy+engagement model aims at identifying non-effortful test-taking behavior in terms of nonresponse and rapid guessing based on responses and response times. The model allows for identifying rapid-guessing behavior on the item-by-examinee level whilst jointly modeling the processes underlying rapid guessing and effortful responding. To assess whether the model indeed provides a valid measure of test-taking effort, we investigate (1) convergent validity with previously developed behavioral as well as self-report measures on guessing behavior and effort, (2) fit within the nomological network of test-taking motivation derived from expectancy-value theory, and (3) ability to detect differences between groups that can be assumed to differ in test-taking effort. Results suggest that the model captures central aspects of non-effortful test-taking behavior. While it does not cover the whole spectrum of non-effortful test-taking behavior, it provides a measure for some aspects of it, in a manner that is less subjective than self-reports. The article concludes with a discussion of implications for the development of behavioral measures of non-effortful test-taking behavior.

U2 - 10.1080/10627197.2020.1858786

DO - 10.1080/10627197.2020.1858786

M3 - Journal article

VL - 26

SP - 104

EP - 124

JO - Educational Assessment

JF - Educational Assessment

SN - 1062-7197

IS - 2

ER -

ID: 1440924