General English Proficiency Test

We work on projects that focus on linking GEPT writing tests to the Common European Framework of Reference (CEFR)

Project 2: Linking the GEPT Writing Sub-test   (Part 1) to the Common European Framework of Reference (CEFR)

2019-2020

This study aims to link GEPT writing subtest (Part 1) to the Common European Framework of Reference and explore the challenges in linking translation tasks to the CEFR levels

Person writing

Previous research has aligned the GEPT reading, listening and writing (Part 2) subtests to the Common European Framework of Reference (CEFR). However, no attempt has been made to link Part 1 of the GEPT writing subtest (Chinese-English translation) to the CEFR levels. In view of the recent publication of the illustrative descriptors of mediation in the companion volume to the CEFR (Council of Europe, 2018), this proposed study is aimed at linking Part 1 of the GEPT writing subtest at intermediate and high-intermediate levels to the CEFR, following the four stages of familiarisation, specification, standardisation, and validation set out in the CEFR manual (Council of Europe, 2009).

Two examinee-centred standards-setting methods (ie the ‘Contrasting Groups’ and ‘Borderline Group’ methods) will be used in combination in the alignment process. In addition, this study also aims to explore the potential challenges in linking translation tasks to the CEFR levels through a think aloud protocols (TAP) study which enables us to understand the mental processes that the participating panelists engage with when linking test takers’ translation performance to the CEFR levels. Findings of this study will provide further empirical evidence concerning the alignment between the GEPT and the CEFR, thus facilitating test score interpretations and test use. In addition, this study will also shed light on aligning translation tests with the CEFR levels, as research of such a nature is scarce in the field.

Project 1: Linking the GEPT Writing Sub-test to the Common European Framework of Reference (CEFR)

2014-2016

The aim of this research project was to conduct an empirically-evidenced linking study to align the General English Proficiency Test (GEPT) writing tests to the CEFR

CEFR levels

The study closely followed the suggested methods and procedures set out in the Manual on Relating language examinations to the Common European Framework of Reference for Languages: learning, teaching, assessment (Council of Europe, 2009) and therefore involved the following four stages: familiarisation, specification, standardisation and validation. The standard-setting study used a twin panel design, with a group of standard-setting judges in Australia and a group in Taiwan. Two standard setting methods were employed: the borderline method and the contrasting groups method. The standard setting process was conducted entirely online. Participants were provided online materials that they were able to access and interact with at times convenient to them.

The findings from the specification phase showed that, in terms of test content and task design, the different GEPT writing sub-tests aligned well with the intended CEFR levels. The results from the two standard-setting panels also supported alignment with the CEFR. Cut-scores resulting from the analysis of the panelists’ judgments, however, indicate that the existing GEPT pass score for the sub-test may need to be set slightly lower in order to reflect the relevant CEFR level benchmarks. However, it should be noted that writing in the GEPT takes place under testing conditions in which ‘task requirement’ is a crucial criterion for evaluating candidates’ performance, whereas ‘task requirement’ is not included in the CEFR can-do descriptors. This distinction may have influenced the resulting cut- scores. Results from the two panels and the two standard-setting methods were similar, adding validity to the findings. The final report has been published and is available below.

LTTC-GEPT Research Reports

Project 2

Sponsor / Grant scheme

Language Testing and Training Center (LTTC), Taiwan / LTTC Teaching and Research Grants 2019

Project team members

Dr Jason Fan
Associate Professor Ute Knoch

Contact

Dr Jason Fan

Project 1

Sponsor / Grant scheme

Language Testing and Training Center (LTTC), Taiwan / LTTC-GEPT Research Grants Program 2014-2016

Project team members

Associate Professor Ute Knoch
Dr Kellie Frost

Contact

Associate Professor Ute Knoch