TY - JOUR

T1 - Is a long essay always a good essay?

T2 - The effect of text length on writing assessment

AU - Fleckenstein, Johanna

AU - Meyer, Jennifer

AU - Jansen, Thorben

AU - Keller, Stefan

AU - Köller, Olaf

PY - 2020/9/25

Y1 - 2020/9/25

N2 - The assessment of text quality is a transdisciplinary issue concerning the research areas of educational assessment, language technology, and classroom instruction. Text length has been found to strongly influence human judgment of text quality. The question of whether text length is a construct-relevant aspect of writing competence or a source of judgment bias has been discussed controversially. This paper used both a correlational and an experimental approach to investigate this question. Secondary analyses were performed on a large-scale dataset with highly trained raters, showing an effect of text length beyond language proficiency. Furthermore, an experimental study found that pre-service teachers tended to undervalue text length when compared to professional ratings. The findings are discussed with respect to the role of training and context in writing assessment.

AB - The assessment of text quality is a transdisciplinary issue concerning the research areas of educational assessment, language technology, and classroom instruction. Text length has been found to strongly influence human judgment of text quality. The question of whether text length is a construct-relevant aspect of writing competence or a source of judgment bias has been discussed controversially. This paper used both a correlational and an experimental approach to investigate this question. Secondary analyses were performed on a large-scale dataset with highly trained raters, showing an effect of text length beyond language proficiency. Furthermore, an experimental study found that pre-service teachers tended to undervalue text length when compared to professional ratings. The findings are discussed with respect to the role of training and context in writing assessment.

KW - text length

KW - writing assessment

KW - text quality

KW - judgment bias

KW - English as a foreign language

KW - human raters

KW - pre-service teachers

U2 - 10.3389/fpsyg.2020.562462

DO - 10.3389/fpsyg.2020.562462

M3 - Journal article

VL - 11

JO - Frontiers in Psychology

JF - Frontiers in Psychology

SN - 1664-1078

M1 - 562462

ER -