The aim of this research project was to conduct an empirically-evidenced linking study to align the General English Proficiency Test (GEPT) writing tests to the Common European Framework of Reference (CEFR).
The study closely followed the suggested methods and procedures set out in the Manual on Relating language examinations to the Common European Framework of Reference for Languages: learning, teaching, assessment (Council of Europe, 2009) and therefore involved the following four stages: familiarization, specification, standardization and validation. The standard-setting study used a twin panel design, with a group of standard-setting judges in Australia and a group in Taiwan. Two standard setting methods were employed: the borderline method and the contrasting groups method. The standard setting process was conducted entirely online. Participants were provided online materials that they were able to access and interact with at times convenient to them.
The findings from the specification phase showed that, in terms of test content and task design, the different GEPT writing sub-tests aligned well with the intended CEFR levels. The results from the two standard-setting panels also supported alignment with the CEFR. Cut-scores resulting from the analysis of the panelists' judgments, however, indicate that the existing GEPT pass score for the sub-test may need to be set slightly lower in order to reflect the relevant CEFR level benchmarks. However, it should be noted that writing in the GEPT takes place under testing conditions in which 'task requirement' is a crucial criterion for evaluating candidates' performance, whereas 'task requirement' is not included in the CEFR can-do descriptors. This distinction may have influenced the resulting cut- scores. Results from the two panels and the two standard-setting methods were similar, adding validity to the findings. The final report has been published and is available below.