Testing oral ability

CONTENTS REPRESENTATIVE TASK ELICIT A VALID SAMPLE OF ORAL ABILITY ENSURE VALID AND RELIABLE SCORING CONCLUSION WHAT IS THE ORAL DEFINITION The oral exam is a practice in many schools and disciplines in which an examiner poses questions to the student in spoken form The student has to answer the question in such a way as to demonstrate sufficient knowledge of the subject to pass the exam COMPREHENSION ORAL ABILITY INTERACT SUCCESSFULLY PRODUCTION We want to set tasks that form a representative sample of the population of oral tasks that we except candidates to able to perform THE BASIC PROBLEMS IN TESTING ORAL ABILITY The tasks should elicit behavior which truly represents the candidate’s ability The samples of behavior can and will be scored validly and reliably REPRESENTATIVE TASKS • Specify all possible content • Include a representative sample of the specified content when setting tasks SPECIFY ALL POSSIBLE CONTENT OPERATIONS Content of the Cambridge CCSE Test of Oral Interaction + Expressing: like/dislike, complain, opinions,… + Directing: advising, persuading, instructing,… + Describing: actions, events,… + Eliciting: information, directions,… + Narration: sequence of events + Reporting: description, comment,… SKILLS Informational skills + Provide personal or non personal information + Narrate events + Expressing personal opinions, emotions, requirements, ect + Narrate, analyzes, summaries what they have said Interactional skills + Express purpose, agreement, disagreement + Elicit opinions, information, clarification + Recognize other speaker’s purpose + Justify or support statements or opinions of other speakers + Respond to requests for clarification + Correct themselves or others + Indicate understanding or uncertainty, ect Skills in managing interactions + Initiate interactions + Change the topic of an interaction + Give turns to other speakers + Come to a decision 2.2.PLAN AND STRUCTURE THE TEST CAREFULLY Time of the test: as long as feasible Plan the test Give as many “fresh starts” as possible Avoid reminding candidates when they are in test Use a second tester for interview Set tasks and topics that cause no difficulty in the candidates’s first language Carry out the test in a quiet room Put the candidates at their ease so that they can show what they are capable of Collect enough relevant information 10 Do not talk too much 11 Select interviewers carefully & train them TEST’S STAGE BACKGROUND AND OVERVIEW ASSIGNING CANDIDATES TO LEVELS CONDUCTING INTERVIEWS ASSESSMENT TEST’S STAGE BACKGROUND AND OVERVIEW • Trainees are given background on the interview • Trainees are given a copy of the handbook and taken through its contents • The structure of the interview is described • A video of a typical interview is shown • Trainees are asked to study handbook before the second stage of the training TEST’S STAGE ASSIGNING CANDIDATES TO LEVELS • Queries arising from reading the handbook are answered • A set of calibrated video is shown • After each video, trainees are asked to write down the levels to which they assign the candidate according to the level descriptions and the analytic scale, and to complete a questionnaire on the task A discussion follows • All papers completed by trainees during this stage are kept as a record of their performance TEST’S STAGE CONDUCTING INTERVIEWS • Pairs of trainees conduct interviews, which are videoed • The other trainees watch the interview on a monitor in another room • After each interview, all trainees assign the candidate to a level and complete a questionnaire These are then discussed • Each trainee will complete interviews TEST’S STAGE ASSESSMENT • Procedures will be as in stage 3, except that the performance of trainees will not be watched by other trainees Nor will there be any discussion after each interview ENSURE VALID AND RELIABLE SCORING 3.1 CREATE APPROPRIATE SCALES FOR SCORING ACCURACY Pronunciation APPROPRIACY The use of language RANGE Vocabulary, language available to the candidate FLEXIBILITY SIZE Adaptation to changes at the conversation Capability of making length and complex contributions where appropriate PROFICIENCY DESCRIPTION (ILR speaking levels) ENSURE VALID AND RELIABLE SCORING 3.2 Calibrate the scale to be used • Generally the same procedures are followed in calibrating speaking scales as were described for writing scales • However, the obvious difference that video-recordings are used rather than pieces of written work ENSURE VALID AND RELIABLE SCORING 3.3 Trains scorers (as opposed to interviewers) • The training of interviewers has already been outlined • Where raters are used to score interviews without acting as interviewers themselves, or are involved in the rating of responses to audio-or- video-recorded stimuli, the same methods can be used as for the training of raters of written work ENSURE VALID AND RELIABLE SCORING 3.4 Follow acceptable scoring procedures • Ignore personal qualities of the candidates that are irrelevant to an assessment of their language ability • The accurate measurement of oral ability is not easy CONCLUSION • However, where a test is high stakes, or backwash is an important consideration, the investment of such time and effort may be considered necessary • Readers are reminded that the appropriateness of content, of rating scales levels, and of elicitation techniques used in oral testing will depend upon the needs of individual institutions or organizations ...CONTENTS REPRESENTATIVE TASK ELICIT A VALID SAMPLE OF ORAL ABILITY ENSURE VALID AND RELIABLE SCORING CONCLUSION WHAT IS THE ORAL DEFINITION The oral exam is a practice in many schools and disciplines... candidates to able to perform THE BASIC PROBLEMS IN TESTING ORAL ABILITY The tasks should elicit behavior which truly represents the candidate’s ability The samples of behavior can and will be scored... subject to pass the exam COMPREHENSION ORAL ABILITY INTERACT SUCCESSFULLY PRODUCTION We want to set tasks that form a representative sample of the population of oral tasks that we except candidates