Rater Cognition in L2 Speaking Assessment: A Review of the Literature

Abstract

This literature review surveys representative studies within the context of L2 speaking assessment that have contributed to the conceptualization of rater cognition. Two types of studies are looked at: 1) studies that examine how raters differ (and sometimes agree) in their cognitive processes and rating behaviors, in terms of their focus and feature attention, approaches to scoring, and treatment of the scoring criteria and non-criteria relevant aspects and features of the speaking performance; 2) studies that explore why raters differ, through the analysis of the interactions between several rater background factors (i.e., rater language background, rater experience and rater training) and their rating behaviors and decision-making processes. The two types of studies have improved our understanding of the nature and the causes of rater variability in their perception and evaluation of L2 speech. However, few studies has drawn on existing theories of human information processing and research on strategy use, which can explain on a cognitive-processing (Purpura, 2014) level what goes on in raters’ mind during assessment. It is argued as a final conclusion that only based on established frameworks of human information processing and research on (meta)cognitive strategy use can rater cognition be explored with more depth and breadth.