Seminar: Exploring context and construct in rating scale design

Rating criteria designed for operational rating, regardless of whether they are related to a framework document such as the CEFR, have traditionally been divided into two groups, (1) those designed drawing on intuitive methods and (2) those created using a data-driven design (Fulcher, 2003; COE, 2001). It has been argued that the latter of these methods is more sensitive to the local testing context, as criteria are designed on the basis of real learner performances, while the former method is less context-sensitive. It is clear from this description that the issue of context is highly relevant to the design of rating scale criteria.
In this talk A/Prof. Knoch will explore the issue of construct definition and context in the design of rating criteria. She will discuss different sources of test construct and aspects of context that are relevant to this, and explore how these may come into play both in the design and adaption of operational scale criteria. A/Prof. Knoch will do this by drawing both on the increasing literature on rating scale design and on a range of projects she has worked on, in a variety of contexts.She will argue that context-sensitive criteria can both increase the precision of the inferences that can be made based on the test scores or threaten generalisability, depending on what contextual issues are considered.
Venue: Room 407, Babel Building (139), University of Melbourne, Parkville.