Writing Task Performance and First Language Background on an ESL Placement Exam A Many-Facets Rasch Analysis of Facet Main Effects and Differential Facet Functioning

Abstract

First Language (L1) has been assumed to play a role in Second Language ability (Bachman & Palmer, 1996). However, the interplay between them across skill, task, or scoring criteria is more complex (Hamp-Lyons & Davies, 2008). Using Many-Facets Rasch Measurement, this study investigates the main effects of examinee ability, rater severity, task difficulty, and rubric scale difficulty and functionality on the writing section of an English as a Second Language program's placement test, then compares performance among L1 Spanish and Japanese examinees to discern the presence of bias across facet. The results for examinee ability and rater severity suggest score variability not expected by the model. Regarding task difficulty and scale difficulty and functionality, it can be concluded that an argumentative essay genre was more difficult than a customer review, or that rater assessed rubric criteria for Content, Organization, and Language more harshly for the former. A bias analysis among L1 Spanish and Japanese examinees revealed that the customer review displayed a bias against Japanese examinees, particularly for Organization, while the argumentative essay displayed bias for them, particularly for Organization and Language. These results demonstrate how placement testing could inform curricula in language programs with linguistically diverse student populations.