Occupational English Test (OET) test development and validation

The Language Testing Research Centre (LTRC) has been engaged in research on the Occupational English Test (OET) for over three decades. The OET is a test to evaluate the English-language competency of qualified medical and health professionals who wish to practise in an English-language context.

Project: A research study examining the communication needs and challenges of personal care attendants (PCAs) working with the elderly in residential aged-care settings or providing in-home services.

2021-2022

In Australia, as in many other western countries, the aged care labour force is heavily dependent on migrants, the vast majority of these coming from culturally and linguistically diverse (CALD) backgrounds.

With an aging population, the number of Australians living in residential aged care facilities or those requiring in-home services is likely to increase substantially over the coming years. As a result, the workforce attending to the elderly will also need to grow. In Australia, as in many other western countries, the aged care labour force is heavily dependent on migrants, the vast majority of these coming from culturally and linguistically diverse (CALD) backgrounds.

Personal care attendants (PCAs), as the name suggests, spend more hours than any other professions working with residents in aged care facilities. Their day-to-day tasks differ from those of nurses in that they are employed to assist with the personal care and daily activities of those living in aged care settings or receiving in-home care. PCAs differ from nurses in that the profession is not regulated – there is no professional registration authority assessing qualification or language proficiency. PCAs generally study a Certificate III (although this is not mandatory) before being able to enter the workforce.
A recent Royal Commission in Australia (Royal Commission into Aged Care Quality and Safety, 2021) showed that there were concerns about the language proficiency and communication skills of PCAs. As a result, the Royal Commission recommended that regulation of personal care workers is introduced, with a national registration scheme, minimum qualification requirements and minimum levels of English language proficiency (among other requirements) (Royal Commission into Aged Care Quality and Safety, 2021, p. 126).

The Occupational English Test is well-placed to develop a language proficiency test for PCAs, already offering testing modules for twelve health professions. To develop a test for PCAs, however, a detailed domain analysis is required to underpin the validity of such an instrument (Knoch & Macqueen, 2020). There is a limited literature on the communication demands of PCAs – this domain has received much less attention compared to other professions, such as nurses and doctors. The communicative tasks and demands are likely to be somewhat related to that of nurses, but will differ in key aspects. To gain a deeper understanding of PCA’s daily communication tasks and possible communicative challenges encountered by CALD PCAs, this research proposes a multi-method project.
The project aims are as follows:

• to gain an understanding of the daily communication tasks of PCAs
• to gain insights into the communication challenges faced by CALD PCAs in the Australian workforce
• to identify the communicative events that are particular barriers to successful communication with colleagues, other health professionals, residents and family
• to explore which particular difficulties may lead to risks to care and adverse outcomes
• to create, based on the findings of the domain analysis, a design blue-print for an OET module for PCAs.

Past projects:

Project: A research study exploring the employment outcomes and work experiences of Internationally Qualified Nurses (IQN’s) in Australia who have taken OET.

2020-2021

This study aimed to address three key concerns about the English-language
communication skills of IQNs applying for registration in Australia. It aimed to
get a better sense of the employment outcomes of IQNs taking the OET test.

IQNs are required to pass an English language screening test before being able to apply for registration and one of the challenges for the test designers is ensuring these tests are fit for purpose. In Australia, the Occupational English Test (OET) is one of the tests accepted by registration authorities for registration. The other tests that are commonly used by applicants are the IELTS and Pearson PTE tests, which are developed to assess academic readiness for university study, rather than nursing-specific communication skills.
This study was designed to address three key concerns about the English-language communication skills of IQNs applying for registration in Australia. It aimed to get a better sense of the employment outcomes of IQNs taking the OET test and to establish where nurses who have not yet passed the test are employed (anecdotal evidence suggests many work in aged care settings as personal care assistants, or PCAs). The study also set out to establish from those IQNs already working as registered nurses and as PCAs in Australian workplaces what day-to-day communication tasks they encounter and which they perceive as difficult. This information can be used to establish how well the OET test materials capture these kinds of challenging tasks. Finally, we set out to gather further information about the impact of the OET on IQNs in Australia. The study draws on a combined survey and interview design to gain detailed understanding of issues outlined above.
The study is the first to link IQNs working in Australian health care settings with the design of language tests. This group of informants has largely been disregarded as informants in test design in this setting. The study elicits important information from nurses and this information can improve our theoretical understanding of health communication for overseas-trained health professionals as well as guide design decisions in language testing for specific purposes. The final report of the study completed in 2020.

Scoping study of professional to professional communication across health professions

This two-phase study investigated the types of professional to professional oral communication engaged in by healthcare professionals across a range of professions with a view to informing revisions to the existing speaking section of the OET to ensure the test is well-aligned with current workplace communication practices. In the first phase of the study, interviews were conducted with a range of health professionals and health professional educators to gain insights into the nature of current inter- and intra-professional communication practices.

Based on the interview data, the LTRC designed an online survey to gather insights from a larger number of health professionals. Findings showed that oral communication practices amongst professionals are diverse and, in some aspects, profession-specific, and that there was support for revising the existing OET speaking section to better reflect the communication demands of these practices. Study outcomes have formed the basis of profession-specific recommendations in relation to OET speaking task development.

Verification of checklist indicators in OET speaking test data

This study aimed to establish, through detailed discourse analyses, the extent to which criteria specified in a revised checklist of indicators of effective oral communication were elicited by the current OET speaking tasks. The data set consisted of 160 audio recorded role play performances across eight health professions (medicine, dentistry, nursing, physiotherapy, dietetics, radiography, vet science, and pharmacy) from live OET administrations.

Findings showed that existing role play tasks elicit some, but not all aspects of behaviours described in the revised checklist, with considerable variation across professions. The range of behaviours elicited were found to be contingent, at least in part, on the nature of role play scenarios and instructions, which in some tasks appeared to constrain interactions more than required. Study outcomes led to recommendations to develop rating scale criteria to incorporate behaviours in the checklist that were elicited across professions, and to revise task design to enable a broadening of the construct.

Trialling of new OET listening and reading tests

As part of continued test validation efforts, the LTRC conducted trials of newly developed versions of the OET listening and reading tests on behalf of Cambridge English and the Occupation English Test Centre.