Multiword phrase use in Aptis speaking across proficiency levels and L1s
A British Council Assessment Research Grant project.
2023-2025
The main aim of this study was to analyse performance on the Aptis speaking test across test-taker proficiency levels and L1s in terms of the multiword phrases used by test takers.
The main aim of this project was to analyse performances on Aptis speaking tasks using the British Council Lancaster Aptis Corpus (BCLAC) across different proficiency levels in terms of the multiword phrases used by test takers. More specifically, it explored differences across test-taker English proficiency (CEFR levels B1, B2, and C) and L1 (Arabic, Chinese, Spanish). The BCLAC is a corpus consisting of 664,056 words and spoken test performances from all 866 speakers are used. Aptis speaking transcripts from BCLAC are analysed with the aid of corpus linguistics, to identify two-word multiword phrases and their use.
The project was anticipated to not only provide empirically supported evidence of multiword phrase use (density and type) that distinguishes between these variables, but practically, it will also serve as validity evidence for the Aptis test to improve understanding of how multiword phrases related to Aptis speaking descriptors. The findings mainly related to evaluation inference/scoring validity in that it should be possible to distinguish multiword phrase use across proficiency levels. In addition, findings relating to L1 would affect the generalisation inference and test-taker characteristics. If there are major differences in multiword use between L1s, this may be an important aspect to consider in relation to test fairness and to English language learning and teaching.
The final report was completed in late 2024.