Assessing efl speaking skills in vietnamese tertiary education

Assessing EFL Speaking Skills in Vietnamese Tertiary Education by Thanh Nam Lam BA (EFL Pedagogy), MA (TESOL) A dissertation submitted to the School of Education, the University of Newcastle, Australia in fulfilment of the requirements for the Degree of Doctor of Philosophy (Education) October 2018 STATEMENT OF ORIGINALITY I hereby certify that the work embodied in the thesis is my own work, conducted under normal supervision The thesis contains no material which has been accepted, or is being examined, for the award of any other degree or diploma in any university or other tertiary institution and, to the best of my knowledge and belief, contains no material previously published or written by another person, except where due reference has been made I give consent to the final version of my thesis being made available worldwide when deposited in the University’s Digital Repository, subject to the provisions of the Copyright Act 1968 and any approved embargo Thanh Nam Lam Signature: ………………………… Date: …… October, 2018 ii ACKNOWLEDGEMENTS After an intensive period of more than four years, today is the day: writing this note of thanks is the finishing touch on my thesis It has been a period of rewarding learning for me, not only in the scientific arena, but also on a personal level I would like to reflect on the people who have supported and helped me so much throughout my academic journey First and foremost, I would like to express my sincere gratitude to my wonderful supervisors Professor James Albright, Professor John Fischetti, and Professor Greg Preston for their continuous support of my PhD study, for their patience, responsibility, enthusiasm, and immense knowledge I am fortunate to have had them as my supervisors Without their dedicated guidance, this research project would never have been possible I acknowledge my debt to the Government of Vietnam through the Ministry of Education and Training and the University of Newcastle, Australia, for awarding me a VIED-TUIT scholarship to support my doctoral studies I am very thankful to the University of Newcastle for providing such a rich research resource and excellent student services and, in particular, to the friendly and supportive academic staff of the School of Education I must say a special thank you to Vietnam Aviation Academy, where I am working, for facilitating my study with the most favourable conditions they can My profound gratitude goes to Professor Max Smith, Professor Allyson Holbrook, and Associate Professor James Ladwig, who gave me interesting lectures and useful research skills in my coursework at the University of Newcastle I wish also to express my great appreciation to Associate Professor Kylie Shaw, Associate Professor Mitchell O’Toole, Dr Maura Sellars, Dr Rachel Burke, Ms Helen Thursby, Ms Helen Hopcroft, and Mr Nicholas Collier for their suggestions, interest, language assistance and constructive feedback on my very first manuscript My additional thanks goes to Ms Ruth McHugh for helping me with the tedious and long running job of reading proof iii I would like to express my heartfelt thanks to Dr Ho Thanh My Phuong As my former supervisor, she has taught me and motivated me more than I could ever give her credit for here I would also like to take this opportunity to thank my former instructor, Dr Vu Thi Phuong Anh She inspired me with research ideas about language testing and assessment She has shown me, by her example, what a good scientist (and person) should be My deepest respect and appreciation go to Heads of the EFL Departments at the tertiary institutions in Vietnam for their generosity giving consent to my data collection at their EFL classes I would like to thank the EFL teacher and student participants in Vietnam Their information and ideas constituted the core of my study Many of the participants set aside considerable amounts of their time to provide me with a profound understanding about their experiences and perceptions of oral assessment The EFL experts’ insights enriched my research results and contributed enormously to the eventual conclusions from the investigation I am greatly appreciative of their enthusiastic cooperation in my study I wish to thank my family for their unconditional love and encouragement during my time of studying away from home They have been and still are ever ready to assist me in my various endeavours Finally, there are my lovely schoolmates here in Callaghan campus We were not only able to support each other by deliberating over our study problems and findings, but also happily by talking about things other than just our papers I cannot forget my friends from Vietnam and the US Their messages and emails gave me motivational strength to complete my study Newcastle, 30 October 2018 Nam Lam iv TABLE OF CONTENTS STATEMENT OF ORIGINALITY ii ACKNOWLEDGEMENTS iii TABLE OF CONTENTS v LIST OF ABBREVIATIONS x LIST OF TABLES xi LIST OF FIGURES xiv LIST OF APPENDICES xv GLOSSARY xvii ABSTRACT xviii CHAPTER ONE: INTRODUCTION 21 1.1 Background of the study 21 1.1.1 A brief history of testing L2 speaking 22 1.1.2 Trends in language testing 26 1.1.3 Context of the research 29 1.2 Research questions 44 1.3 Significance of the research 47 1.4 Organisation of the thesis 49 CHAPTER TWO: LITERATURE REVIEW 51 2.1 Introduction 51 2.2 Key issues in designing spoken language tests 51 2.1.1 Construct validity in oral language testing 56 2.1.2 Content aspect of construct validity 59 2.1.3 Reliability 60 2.3 Conceptual framework for validating speaking tests 63 2.4 Formats of speaking tests 65 2.5 Technological applications in oral assessment 68 v 2.6 Factors affecting test validity and reliability 69 2.6.1 Assessment criteria 69 2.6.2 Rating scales 70 2.6.3 Test tasks 71 2.7 Washback of oral language assessment 74 2.8 Summary 76 CHAPTER THREE: METHODOLOGY 78 3.1 Introduction 78 3.2 Rationale for the research design 78 3.2.1 Adoption of a mixed methods approach 78 3.2.2 Using a convergent design 80 3.3 Research setting 81 3.3.1 Research sites 82 3.3.2 Research participants 83 3.4 Data sources 87 3.4.1 Test room observation 89 3.4.2 Questionnaire surveys 90 3.4.3 Interviews 95 3.4.4 Expert judgements 98 3.4.5 Documents 99 3.5 Data collection procedures 100 3.6 Methods of data analysis 105 3.6.1 Quantitative data processing 105 3.6.2 Qualitative data processing 106 3.7 Presenting data analysis 112 3.8 Research ethics and compliance 113 3.9 Assuring research quality 115 3.10 Summary 117 CHAPTER FOUR: RESULTS: TEST TAKER CHARACTERISTICS AND TEST ADMINISTRATION 119 4.1 Introduction 119 vi 4.2 Test taker characteristics 120 4.3 Speaking test administration across institutions 135 4.4 Candidates’ and raters’ perceptions of the oral test administration 145 4.4.1 Candidates’ perceptions of the oral test administration 145 4.4.2 Test raters’ perceptions of the oral test administration 150 4.5 Summary 155 CHAPTER FIVE: RESULTS: CONTENT RELEVANCE OF SPEAKING TEST QUESTIONS 157 5.1 Introduction 157 5.2 Defining test constructs 158 5.3 Designing the content judgement protocol 161 5.4 Selecting approaches to data analysis 164 5.5 Relevance of test contents 165 5.5.1 EFL experts’ judgements on test content relevance 166 5.5.2 Linking expert opinions with other data sources 180 5.6 Summary 190 CHAPTER SIX: RESULTS: SPEAKING TEST TASKS 192 6.1 Introduction 192 6.2 A comparative analysis of speaking test tasks across institutions 193 6.2.1 Response format 193 6.2.2 Task purposes 206 6.2.3 Time constraints 225 6.2.4 Channels of communication 230 6.3 Raters’ and candidates’ perceptions of the test tasks 235 6.3.1 Teachers’ perceptions of the test tasks 235 6.3.2 Candidates’ perceptions of the test tasks 238 6.4 Summary 240 CHAPTER SEVEN: RESULTS: RATER CHARACTERISTICS AND RATING ORAL SKILLS 242 7.1 Introduction 242 7.2 Test rater characteristics 243 vii 7.3 Rating and scoring 246 7.3.1 Oral assessment criteria 247 7.3.2 Rating scales 248 7.3.3 Oral rating process 251 7.4 Raters’ consistency in oral rating 256 7.4.1 Scoring methods 256 7.4.2 Aspects of rating in oral assessment 258 7.4.3 Giving bonus points 261 7.4.4 Familiarity between the rater and candidates 263 7.5 Test raters’ and candidates’ perceptions about the practice of rating and scoring 266 7.5.1 Test raters’ perceptions of the rating process 266 7.5.2 Candidates’ perceptions of the rating and scoring 269 7.6 Test score analysis 272 7.6.1 Distribution of test scores 273 7.6.2 Inter-rater reliability in direct scoring between pairs of raters 275 7.6.3 Inter-rater reliability in semi-direct scoring between across pairs of raters 278 7.7 Summary 283 raters…… ….… CHAPTER EIGHT: RESULTS: IMPACT OF ORAL TESTING 285 8.1 Introduction 285 8.2 Impact of the oral test from candidates’ perspectives 286 8.2.1 Impact of test scores 287 8.2.2 Learning activities candidates found useful for the oral test 290 8.2.3 Candidates’ perceptions of the test impact on EFL learning 292 8.3 Impact of the oral test on teaching from teacher raters’ perspectives 298 8.3.1 Major desired changes in teaching speaking skills 298 8.3.2 Teaching activities to prepare learners for the oral test 300 8.3.3 Teachers’ perceptions of the test impacts on teaching EFL speaking skills 304 8.3.4 Implementing a new method of assessing speaking skills 314 8.4 Summary 316 CHAPTER NINE: SUMMARY AND CONCLUSIONS 317 9.1 Introduction 317 viii 9.2 Summary of research results 318 9.2.1 Issues in test administration affecting test fairness and candidates’ speaking performance 319 9.2.2 Test content relevance and inequality in test questions’ degree of difficulty 323 9.2.3 Diversity in test tasks elicited different speech patterns for assessment 324 9.2.4 Inconsistency in rating and scoring spoken language performance 326 9.2.5 Impact of oral testing on EFL teaching and learning 327 9.3 Limitations of the study and recommendations for future research 335 9.4 Implications 340 9.4.1 Implications for speaking test administrators 340 9.4.2 Implications for speaking test designers 342 9.4.3 Implications for oral test raters and scorers 343 9.4.4 Implications for oral test takers 344 9.4.5 Implications for educational policy makers 345 9.4.6 Implications for language tesing researchers 347 9.5 Conclusions 348 REFERENCES 352 APPENDICES 385 ix LIST OF ABBREVIATIONS BA Bachelor of Arts BEC Business English Certificate CA Conversation analysis CBT Computer-based testing CEFR Common European framework of reference for languages CLT Communicative language teaching EFL English as a foreign language ELT English language teaching ESL English as a second language ESP English for specific purposes ETS Educational Testing Service IELTS International English Language Testing System L1 First language L2 Second/Foreign Language MOET Ministry of Education and Training (Vietnam) OPI Oral Proficiency Interview SBA School-based assessment SE Spoken English SPSS Statistical Package for Social Sciences TESOL Teaching English to Speakers of Other Languages TOEFL Test of English as a Foreign Language TOEIC Test of English for International Communication UCLES University of Cambridge Local Examinations Syndicate x Appendix F.4 Occurence of language functions in speech samples Language functions Informational functions Providing personal information Past Present Future Expressing opinions Elaborating Justifying opinions Comparing Speculating Describing Sequence of events Scene Summarising Suggesting Expressing preferences Interactional functions Agreeing Disagreeing Modifying Asking for opinions Persuading Asking for information Conversational repair Negotiating meaning Check meaning Understanding Common ground Asking clarification Respond to required clarification Correcting utterance Managing interaction Initiating Changing Reciprocating Deciding Uni A Task1 x x x x x x x x x x x Uni B Task1 Task2 Task x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x Uni C Task2 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x Example (University, candidate) When I was a child, er I used to be a very quiet person (U3.C8) When I buy a product, I care about price (U3.C12) They’ll find the land of freedom (U1.C28) In my opinion, when we are satisfied with our achievement… (U1.C3) It also convenient for people who don’t have time to go to the market or supermarket to buy their fresh food (U2.C10) I think taking a gap (year) in Vietnam is not popular because they have er an entrance examination to university (U2.C9) The whole team become a great team, become more fit to work (U1.C14) the copyright laws unclear maybe most in Vietnam and some Asian some… in Asian uh country (U3.C12) He was also standing with, uh, two hands spreading out (U1.C24) First of all this is fifty kilometre, and then they lower thirty kilometre, but they still don’t go (U1.C6) It’s a very high mountains and clouds (U1.C23) In conclusion, in order to be a successful person we have to have er a lot of characters (U2.C17) We should just use it uhm uh like a method… (U1.C11) To me I want to make change every day to better myself even if I am in the current situation (U1.C4) x x x Yes, I agree with you that smart phone, uh, beside its advantages, it also has some disadvantages… (U1.C11) Uhm no, I don’t think so (U3.C4) I just now to play to reduce stress or play with my friends to have fun, not to any tournament any more (U3.C24) How about you? Are there just soft skills? (U2.C17) It give us not only experience… from our real life, but… during the gap year, we also have a chance to know that what our passion is (U2.C10) Do you understand what does it mean ‘free’? (U1.C27) I want to tell you a story…when I was young, uh when I was a child, I used to be a very quiet person (U3.C8) Making change doesn’t mean that you are careless, (or) you are reckless to get into the change (U1.C4) What you mean by ‘warranty’? (U3.R2) I understand your point (U1.C4) I also think that when you take risk you have to run into a lot of problems or obstacles (U1.C27) Control us? (U1.C12) Don’t, not let them control us (U1.C11) x They don’t think about their themselves in the future (U3.C12) x x x x x First I will show my opinion (U1.C28) Thank you now I’d like you to talk together about (U2.R2) Ahh, myself I pick up trash in the street and ahh put put them into the basket (U2.C4) For you which? (U2.C16) – Softskill (U2.C15-17) Note: (x) indicates that this function in the observation checklist (O’Sullivan, Weir, & Saville, 2002) was identified as occuring in the transcripts of oral performances 478 Appendix G: Sample rating scales for assessing speaking skills G.1a Example of a holistic (global) rating scale Interview assessment scale (Carroll, 1980) Band Expert speaker Speaks with authority on a variety of topics Can initiate, expand and develop a theme Very good non-native speaker Maintains effectively his own part of a discussion Initiates, maintains and elaborates as necessary Reveals humour where needed and responds to attitudinal tones Good speaker Presents case clearly and logically and can develop the dialogue coherently and constructively Rather less flexible and fluent than Band performer but can respond to main changes of tone or topic Some hesitation and repetition due to a measure of language restriction but interacts effectively Competent speaker Is able to maintain theme of dialogue, to follow topic switches and to use and appreciate main attitude markers Stumbles and hesitates at times but is reasonably fluent otherwise Some errors and inappropriate language but these will not impede exchange of views Shows some independence in discussion with ability to initiate Modest speaker Although gist of dialogue is relevant and can be basically understood, there are noticeable deficiencies in mastery of language patterns and style Needs to ask for repetition or clarification and similarly to be asked for them Lacks flexibility and initiative The interviewer often has to speak rather deliberately Copes but not with great style or interest Marginal speaker Can maintain dialogue but in a rather passive manner, rarely taking initiative or guiding the discussion Has difficulty in following English at normal speed; lacks fluency and probably accuracy in speaking The dialogue is therefore neither easy nor flowing Nevertheless, gives the impression that he is in touch with the gist of the dialogue even if not wholly master of it Marked L1 accent Extremely limited speaker Dialogue is a drawn-out affair punctuated with hesitations and misunderstandings Only catches part of normal speech and unable to produce continuous and accurate discourse Basic merit is just hanging on to discussion gist, without making major contribution to it Intermittent speaker No working facility; occasional, sporadic communication Non-speaker Not able to understand and/or speak 479 G.1b Example of an analytic rating scale The Foreign Services Institute (FSI) analytic rating scale for language proficiency interview testing (Keitges, 1982) Accent Pronunciation frequently unintelligible Frequent gross errors and a very heavy accent make understanding difficult, require frequent repetition “Foreign accent” requires concentrated listening and mispronunciations lead to occasional misunderstanding and apparent errors in grammar or vocabulary Marked “foreign accent” and occasional mispronunciations that not interfere with understanding No conspicuous mispronunciations but would not be taken for a native speaker Native pronunciation, with no trace of “foreign accent.” Grammar Grammar almost entirely inaccurate except in stock phases Constant errors showing control of very few major patterns and frequently preventing communication Frequent errors showing some major patterns uncontrolled and causing occasional irritation and misunderstanding Occasional errors showing imperfect control of some patterns but no weakness that causes misunderstanding Few errors, with no patterns of failure No more than two errors during the interview Vocabulary Vocabulary inadequate for even the simplest conversation Vocabulary limited to basic personal and survival areas (time, food, transportation, family, etc.) Choice of words sometimes inaccurate, limitations of vocabulary prevent discussion of some common professional and social topics Professional vocabulary adequate to discuss special interests; general vocabulary permits discussion of any nontechnical subject with some circumlocutions Professional vocabulary broad and precise; general vocabulary adequate to cope with complex practical problems and varied social situations Vocabulary apparently as accurate and extensive as that of an educated native speaker Fluency Speech is so halting and fragmentary that conversation is virtually impossible Speech is very slow and uneven except for short or routine sentences Speech is frequently hesitant and jerky; sentences may be left uncompleted Speech is occasionally hesitant, with some unevenness caused by rephrasing and groping for words Speech is effortless and smooth, but perceptibly non-native in speed and evenness Speech on all profeSSional and general topics as effortless and smooth as a native speaker’s Comprehension Understands too little for the simplest type of conversation Understands only slow, very simple speech on common social and touristic topics; requires constant repetition and rephrasing Understands careful, somewhat simplified speech directed to him/her, with considerable repetition and rephrasing Understands quite well normal educated speech directed to him/her, but requires occasional repetition or rephrasing Understands everything in normal educated conversation except for very colloquial or lowfrequency items or exceptionally rapid or slurred speech Understands everything in both formal and colloquial speech to be expected of an educated native speaker 480 G.2 Common Reference Levels proposed by the CEFR G.2a Common Reference Levels: qualitative aspects of spoken language use (Council of Europe, 2001, pp 28–29) C2 RANGE Shows great flexibility reformulating ideas in differing linguistic forms to convey finer shades of meaning precisely, to give emphasis, to differentiate and to eliminate ambiguity Also has a good command of idiomatic expressions and colloquialisms Has a good command of a broad range of language, allowing him/her to select a formulation to express him/herself clearly in an appropriate style on a wide range of general, academic, professional or leisure topics without having to restrict what he/she wants to say Has a sufficient range of language to be able to give clear descriptions, express viewpoints on most general topics, without much conspicuous searching for words, using some complex sentence forms to do so ACCURACY Maintains consistent grammatical control of complex language, even while attention is otherwise engaged (e.g., in forward planning, in monitoring others’ reactions) FLUENCY Can express him/herself spontaneously at length with a natural colloquial flow, avoiding or backtracking around any difficulty so smoothly that the interlocutor is hardly aware of it INTERACTION Can interact with ease and skill, picking up and using nonverbal and intonational cues apparently effortlessly Can interweave his/her contribution into the joint discourse with fully natural turntaking, referencing, allusion making, etc COHERENCE Can create coherent and cohesive discourse making full and appropriate use of a variety of organisational patterns and a wide range of connectors and other cohesive devices Consistently maintains a high degree of grammatical accuracy; errors are rare, difficult to spot and generally corrected when they do occur Can express him/herself fluently and spontaneously, almost effortlessly Only a conceptually difficult subject can hinder a natural, smooth flow of language Can select a suitable phrase from a readily available range of discourse functions to preface his remarks in order to get or to keep the floor and to relate his/her own contributions skilfully to those of other speakers Can produce clear, smoothly flowing, well-structured speech, showing controlled use of organisational patterns, connectors and cohesive devices Shows a relatively high degree of grammatical control Does not make errors which cause misunderstanding and can correct most of his/her mistakes Can produce stretches of language with a fairly even tempo; although he/she can be hesitant as he or she searches for patterns and expressions, there are few noticeably long pauses Can use a limited number of cohesive devices to link his/her utterances into clear, coherent discourse, though there may be some “jumpiness” in a long contribution B1 Has enough language to get by, with sufficient vocabulary to express him/herself with some hesitation and circum-locutions on topics such as family, hobbies and interests, work, travel, and current events Uses reasonably accurately a repertoire of frequently used “routines” and patterns associated with more predictable situations Can keep going comprehensibly, even though pausing for grammatical and lexical planning and repair is very evident, especially in longer stretches of free production A2 Uses basic sentence patterns with memorised phrases, groups of a few words and formulae in order to communicate limited information in simple everyday situations Uses some simple structures correctly, but still systematically makes basic mistakes Can make him/herself understood in very short utterances, even though pauses, false starts and reformulation are very evident Can initiate discourse, take his/her turn when appropriate and end conversation when he /she needs to, though he/she may not always do this elegantly Can help the discussion along on familiar ground confirming comprehension, inviting others in, etc Can initiate, maintain and close simple face-to-face conversation on topics that are familiar or of personal interest Can repeat back part of what someone has said to confirm mutual understanding Can answer questions and respond to simple statements Can indicate when he/she is following but is rarely able to understand enough to keep conversation going of his/her own accord C1 B2 481 Can link a series of shorter, discrete simple elements into a connected, linear sequence of points Can link groups of words with simple connectors like “and, “but” and “because” A1 Has a very basic repertoire of words and simple phrases related to personal details and particular concrete situations Shows only limited control of a few simple grammatical structures and sentence patterns in a memorised repertoire Can manage very short, isolated, mainly pre-packaged utterances, with much pausing to search for expressions, to articulate less familiar words, and to repair communication Can ask and answer questions about personal details Can interact in a simple way but communication is totally dependent on repetition, rephrasing and repair Can link words or groups of words with very basic linear connectors like “and” or “then” G.2b Common Reference Levels: global scale for speaking skills (Council of Europe, 2001, p 24) Category Proficient User Independent User Level Descriptors C2 Can express him/herself spontaneously, very fluently and precisely, differentiating finer shades of meaning even in more complex situations C1 Can express him/herself fluently and spontaneously without much obvious searching for expressions Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices May rarely make errors B2 Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options Does not make errors which cause misunderstanding Basic User B1 Can produce simple connected text on topics which are familiar, or of personal interest Can describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans Pauses for grammatical and lexical planning A2 Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need, although pauses and false starts are evident Uses short utterances and simple structures correctly, but may systematically make errors that create misunderstanding A1 Can use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help Pauses a lot for grammatical and lexical planning 482 G.2c Common Reference Levels: self-assessment grid for speaking skills (Council of Europe, 2001, pp 26-27) Category Level Descriptors Spoken Production Proficient User Independent User Basic User Spoken Interaction C2 I can present a clear, smoothly flowing description or argument in a style appropriate to the context and with an effective logical structure which helps the recipient to notice and remember significant points I can take part effortlessly in any conversation or discussion and have a good familiarity with idiomatic expressions and colloquialisms I can express myself fluently and convey finer shades of meaning precisely If I have a problem I can backtrack and restructure around the difficulty so smoothly that other people are hardly aware of it C1 I can present clear, detailed descriptions of complex subjects integrating sub- themes, developing particular points and rounding off with an appropriate conclusion I can express myself fluently and spontaneously without much obvious searching for expressions I can use language flexibly and effectively for social and professional purposes I can formulate ideas and opinions with precision and relate my contribution skilfully to those of other speakers B2 I can present clear, detailed descriptions on a wide range of subjects related to my field of interest I can explain a viewpoint on a topical issue giving the advantages and disadvantages of various options I can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible I can take an active part in discussion in familiar contexts, accounting for and sustaining my views B1 I can connect phrases in a simple way in order to describe expe-rien-ces and events, my dreams, hopes and ambitions I can briefly give reasons and explanations for opinions and plans I can narrate a story or relate the plot of a book or film and describe my reactions I can deal with most situations likely to arise whilst travelling in an area where the language is spoken I can enter unprepared into conversation on topics that are familiar, of personal interest or pertinent to everyday life (e.g family, hobbies, work, travel and current events) A2 I can use a series of phrases and sentences to describe in simple terms my family and other people, living conditions, my educational background and my present or most recent job I can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar topics and activities I can handle very short social exchanges, even though I can’t usually understand enough to keep the conversation going myself A1 I can use simple phrases and sentences to describe where I live and people I know I can interact in a simple way provided the other person is prepared to repeat or rephrase things at a slower rate of speech and help me formulate what I’m trying to say I can ask and answer simple questions in areas of immediate need or on very familiar topics 483 G.3 The CEFR-based English Competence Framework adopted in Vietnam (Edumax, 2008; IELTS, 1018b) Independent User Proficient User Language Competence THE FOREIGN LANGUAGE PROFICIENCY FRAMEWORK APPLIED IN VIETNAM BASED ON THE 2008 – 2020 “TEACHING AND LEARNING FOREIGN LANGUAGES IN THE NATIONAL EDUCATION SYSTEM” SCHEME CEFR Levels (equivalent level names) Qualification required by the MoET (Vietnam) C2: Mastery (Proficiency) Level Language competence framework: Global descriptions English for young learners (YALE) General English General English Business English Acedemic English Proficiency (CPE) IELTS C1: Effective Operational Proficiency (Advanced) Level 5: English major university graduate Advanced (CAE) B2: Vantage (Upperintermediate) Level 4: English major college graduate First (FCE) B1: Threshold (Intermediate) Level 3: Non-English major university graduate, vocational high school and upper secondary school graduate Level 2: Vocational training graduate and lower secondary school graduate A2: Waystage (Elementary) Basic User Equivalent international assessment programmes A1: Breakthrough (Beginner) Business Higher (BEC) 8.5 Business Vantage (BEC) 6.5 7.5 Level 1: Primary school graduate First for Schools (FCE for Schools) Young Learners Flyers (YLE Flyers) 5.5 Preliminary for Schools (PET for schools) Preliminary (PET) Key for Schools (KET for Schools) Key (KET) Business Preliminary (BEC) 4.5 Can understand with ease virtually everything heard or read Can summarise information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation Can express him/herself spontaneously, very fluently and precisely, differentiating finer shades of meaning even in more complex situations Can understand a wide range of demanding, longer texts, and recognise implicit meaning Can express him/herself fluently and spontaneously without much obvious searching for expressions Can use language flexibly and effectively for social, academic and professional purposes Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialisation Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc Can deal with most situations likely to arise whilst travelling in an area where the language is spoken Can produce simple connected text on topics which are familiar or of personal interest Can describe experiences and events, dreams, hopes and ambitions and briefly give reasons and explanations for opinions and plans Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g very basic personal and family information, shopping, local geography, employment) Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help Young Learners Movers (YLE Movers) 484 Thành thạo Năng lực sử dụng ngôn ngữ KHUNG NĂNG LỰC NGOẠI NGỮ ÁP DỤNG TẠI VIỆT NAM THEO ĐỀ ÁN “DẠY VÀ HỌC NGOẠI NGỮ TRONG HỆ THỐNG GIÁO DỤC QUỐC DÂN 2008 – 2020” Khung theo chuẩn Châu Âu (CEFR) Trình độ theo yêu cầu Bộ Giáo dục & Đào tạo (Việt Nam) C2: Thành thạo Bậc 6: Chương trình khảo sát quốc tế tương đương Khung lực ngoại ngữ: Mô tả tổng quát Tiếng Anh dành cho Thiếu nhi (YALE) Tiếng Anh Tổng quát Tiếng Anh Tổng quát Tiếng Anh Thương mại Tiếng Anh Học thuật Proficiency (CPE) IELTS C1: Cao cấp Bậc 5: Tốt nghiệp Đại học chuyên ngữ Advanced (CAE) 8.5 Business Higher (BEC) 7.5 Độc lập B2: Trung cao cấp Bậc 4: Tốt nghiệp Cao đẳng chuyên ngữ First for Schools (FCE for Schools) First (FCE) Business Vantage (BEC) B1: Trung cấp Bậc 3: Tốt nghiệp Cao đẳng Đại học không chuyên ngữ, Trung cấp chuyên nghiệp Trung học phổ thông Bậc 2: Tốt nghiệp trường nghề Trung học sở Preliminary for Schools (PET for schools) Preliminary (PET) Business Preliminary (BEC) Key for Schools (KET for Schools) Key (KET) Cơ A2: Sơ cấp A1: Căn Bậc 1: Tốt nghiệp tiểu học Young Learners Flyers (YLE Flyers) 6.5 5.5 Young Learners Movers (YLE Movers) 485 4.5 Có thể hiểu cách dễ dàng hầu hết văn nói viết Có thể tóm tắt nguồn thơng tin nói viết, xếp lại thơng tin trình bày lại cách logic Có thể diễn đạt tức thì, trơi chảy xác, phân biệt ý nghĩa tinh tế khác tình phức tạp Có thể hiểu nhận biết hàm ý văn dài với phạm vi rộng Có thể diễn đạt trơi chảy, tức thì, khơng gặp khó khăn việc tìm từ ngữ diễn đạt Có thể sử dụng ngôn ngữ linh hoạt hiệu phục vụ mục đích xã hội, học thuật chun mơn Có thể viết rõ ràng, chặt chẽ, chi tiết chủ đề phức tạp, thể khả tổ chức văn bản, sử dụng tốt từ ngữ nối câu cơng cụ liên kết Có thể hiểu ý văn phức tạp chủ đề cụ thể trừu tượng, kể trao đổi kỹ thuật thuộc lĩnh vực chuyên môn thân Có thể giao tiếp mức độ trơi chảy, tự nhiên với người ngữ Có thể viết văn rõ ràng, chi tiết với nhiều chủ đề khác giải thích quan điểm vấn đề, nêu ưu điểm, nhược điểm phương án lựa chọn khác Có thể hiểu ý đoạn văn hay phát biểu chuẩn mực, rõ ràng chủ đề quen thuộc công việc, trường học, giải trí, v.v Có thể xử lý hầu hết tình xảy đến khu vực có sử dụng ngơn ngữ Có thể viết đoạn văn đơn giản liên quan đến chủ đề quen thuộc cá nhân quan tâm Có thể mơ tả kinh nghiệm, kiện, giấc mơ, hy vọng, hồi bão trình bày ngắn gọn lý do, giải thích ý kiến kế hoạch Có thể hiểu câu cấu trúc sử dụng thường xuyên liên quan đến nhu cầu giao tiếp (như thông tin gia đình, thân, mua hàng, hỏi đường, việc làm) Có thể trao đổi thơng tin chủ đề đơn giản, quen thuộc ngày Có thể mơ tả đơn giản thân, môi trường xung quanh vấn đề thuộc nhu cầu thiết yếu Có thể hiểu, sử dụng cấu trúc quen thuộc thường nhật; từ ngữ đáp ứng nhu cầu giao tiếp cụ thể Có thể tự giới thiệu thân người khác; trả lời thơng tin thân nơi sinh sống, người thân/bạn bè v.v… Có thể giao tiếp đơn giản người đối thoại nói chậm, rõ ràng sẵn sàng hợp tác giúp đỡ Appendix H: Flowchart of procedures for data collection and data analysis DATA ANALYSIS DATA COLLECTION Before the test Preparatory steps: - Designing and validating research instruments - Initial contacts with prospective institutions - Recording devices QUAN and QUAL data collection: Test room observations QUAL data collection: Audiorecordings of speaking performance Connecting QUAN and QUAL data Adjusting interview protocols QUAN and QUAL data collection: Dec 2015 – Jan 2016 Interpretation of the merged results Context validity Interviews with individual test raters and focus groups of test takers RQ1a Test administration QUAL data collection: RQ1c Test tasks Questionnaire surveys for raters and test takers Protocol for judgements on test contents Integration of QUAN and QUAL data QUAL data collection: RQ1b Test contents Documents: test contents Synthesising qualitative data QUAL data collection: Documents: course outlines, test tasks and rating scales Mar 2015 – Nov 2015 After the test During the test QUAN and QUAL data collection: Judgements on the relevence of test contents QUAN data collection: Test scores Jan 2016 – Mar 2016 Flowchart of procedures for data collection and data analysis using a convergent mixed method design for the study on assessing EFL speaking skills (Adapted from Creswell & Clark, 2011, p 79; and Tsushima, 2015, p 112) 486 Scoring validity RQ2 Rating consistency Consequential validity RQ3 Washback effects Apr 2016 – Aug 2018 Appendix I: Transcription notation symbols (Adapted and modified from Atkinson and Heritage, 1984) unfilled pauses or gaps periods of silence, timed in seconds Micro-pauses (less than second) are symbolised (.); longer pauses or gaps appear as a time within parentheses, e.g (.5) represents a 5-second pause colon (:) a lengthened sound or syllable; more colons prolong the stretch dash (–) a cut-off, usually a glottal stop equal sign (=) a latched utterance, no interval between utterances percent signs (% %) quiet talk between brackets ([ ]) overlapping talk, where utterances start and/or end simultaneously parentheses ( ) transcription doubt, uncertainty; words within parentheses are uncertain double parentheses (( )) words within double parentheses describe non-vocal action, details of scene, e.g coughs, telephone rings arrow ( ->) a feature of interest to the analyst 10 inward arrows (> ) the talk slows down 11 ellipsis ( .) turns or part of a turn has been omitted 12 underlining or CAPS a word or SOund is emphasised 13 italics Vietnamese words 14 hah, huh, heh laughter, depending on the sounds produced 15 uhm, er, mm-hmm hesitation or filler words, depending on the sounds produced 16 tch a tongue click 17 punctuations markers of intonation rather than clausal structure; a period (.) is falling intonation, a question mark (?) is rising intonation, a comma (,) is continuing intonation, an exclamation mark (!) is animated intonation 487 Appendix J: List of tertiary institutions This list includes universities in HCMC (Vietnam) offereing training programmes for EFL majors (Updated 01/2017) Type of organisation Name of the Universities No in English in Vietnamese Public Private Banking University Đại học (ĐH) Ngân Hàng ü HCMC University of Social Sciences and Humanities ĐH Khoa Học Xã hội Nhân Văn TPHCM ü Van Lang University ĐH Văn Lang ü Hoa Sen University ĐH Hoa Sen ü The Saigon International University ĐH Quốc Tế Sài Gòn ü HCMC Open University ĐH Mở TPHCM Van Hien University ĐH Văn Hiến ü Nguyen Tat Thanh University ĐH Nguyễn Tất Thành ü Sai Gon University ĐH Sài Gòn ü 10 University of Pedagogy ĐH Sư Phạm ü 11 Gia Dinh Information Technology University ĐH Công Nghệ Thông Tin Gia Định 12 Ton Duc Thang University ĐH Tôn Đức Thắng 13 Foreign Languages - Information Technology ĐH Ngoại Ngữ - Tin Học ü 14 Hong Bang University International ĐH Quốc tế Hồng Bàng ü 15 University of Technology ĐH Công Nghệ ü 16 Industry University ĐH Công Nghiệp ü 17 Hung Vuong University ĐH Hùng Vương 18 University of Agriculture and Forestry ĐH Nông Lâm ü 19 University of Technology and Education ĐH Sư Phạm Kỹ thuật ü Source: http://kenhtuyensinh.vn/truong-dai-hoc-tai-tp-ho-chi-minh 488 ü ü ü ü Appendix K: Original quotes in Vietnamese K.1 Quotes from the Vietnamese press and literature Page in thesis 33 CHAPTER ONE: INTRODUCTION … hoạt động dạy học tổ chức thông qua môi trường giao tiếp đa dạng, phong phú với hoạt động tương tác (trò chơi, hát, kể chuyện, câu đố, vẽ tranh ), hình thức hoạt động cá nhân, theo cặp nhóm (Huy Lan & Lan Anh, 2010) Source: https://nld.com.vn/giao-duc-khoa-hoc/chuong-trinh-ngoai-ngu-10-nam-khoi-dong-lungtung 2010081711321136.htm 37 Sinh viên sau trường đáp ứng yêu cầu kỹ tiếng Anh người sử dụng khoảng 49%, có tới 18,9% sinh viên khơng đáp ứng 31,8% sinh viên cần đào tạo thêm Điều có nghĩa, nửa số sinh viên sau trường không đáp ứng đủ yêu cầu kỹ tiếng Anh (V Le, 2016) Source: https://baotintuc.vn/giao-duc/qua-nua-sinh-vien-tot-nghiep-kem-tieng-anh20160506225914927.htm 38 Việc đổi thi Ngoại ngữ vòng luẩn quẩn Trước năm 2006, học sinh làm thi đại học môn tiếng Anh gồm 20% trắc nghiệm 80% tự luận Từ năm 2006 trở đi, môn tiếng Anh thi 100% trắc nghiệm Sau đó, dạng trắc nghiệm bị phê phán không đánh giá thực lực học sinh, xa rời kỹ thực hành đề lại chuyển 80% trắc nghiệm 20% tự luận kỳ thi THPT quốc gia 2015 Thay đổi năm, kỳ thi 2017 lại dự định chuyển môn tiếng Anh 100% trắc nghiệm (Thanh Tam & Phuong Hoa, 2006) Source: https://vnexpress.net/tin-tuc/giao-duc/thi-trac-nghiem-100-co-the-la-buoc-lui-cua-montieng-anh-3465561.html 41 Mặc dù lệ phí thi VNU-EPT thấp so với chứng quốc tế khác sinh viên chịu đăng ký dự thi Đa số em cho rằng, doanh nghiệp không sử dụng dạng chứng Tôi cho rằng, để áp dụng hiệu VNU-EPT, cần tạo uy tín chứng xã hội, đặc biệt nhà tuyển dụng (Phuong Chinh, 2017) Source: http://www.sggp.org.vn/85-sinh-vien-chua-dat-chuan-trinh-do-tieng-anh-456676.html 41 Vấn đề cốt lõi việc nâng cao hiệu giảng dạy ngoại ngữ giới tích hợp ba thành tố quan trọng trình dạy học, giảng dạy, học tập, kiểm tra đánh giá Riêng Việt Nam, kiểm tra đánh giá khâu yếu cần có quan tâm nhiều (Vu, 2007) Source: http://vietbao.vn/Giao-duc/Hoc-tieng-Anh-10-nam-trong-truong-khong-su-dung-duocKiem-tra-danh-gia-dang-la-khau-yeu-nhat/40224569/202/ 489 K.2 Quotes from Vietnamese interviews with research participants Page in thesis Chapter Four: TEST TAKER CHARACTERISTICS AND TEST ADMINISTRATION 125 Em nghĩ em bị ảnh hưởng nhiều yếu tố Yếu tố ảnh hưởng tới em nhiều điều kiện sức khỏe Tại em tới kì thi bị căng thẳng nên bị đau bụng Cho nên từ tới kì thi lớn bị ảnh hưởng nhiều (U1G1.S2) 149 Em khơng thích hình thức Tại nói với máy tính có lẽ bớt căng thẳng hơn, mà em thấy nói với máy khơng cảm thấy thích thú kiểu khơng có lắng nghe Khi nói phải tương tác Mình tương tác với người thật dễ nói nói trước hình máy tính em nghĩ khơng có tự nhiên (U1G3.S2) 149 em nghĩ máy tính vừa có mặt lợi vừa có mặt hại Mặt lời phần nói chấm cách cơng bằng, nói máy ghi âm lại hết Mặt hại làm cho căng thẳng, nói chuyện trực tiếp với người giám khảo em nghĩ thoải mái hơn, cịn nói máy tính kiểu robot (U1G2.S4) 152 Tồn lẫn quẫn sách Nó khơng có lạ hết á, ví dụ câu mà Warmup, Lead-in Consolidation sách xài lại hết (U2.T3) 153 Em nghĩ (giáo viên cho nhận xét) tốt cho sinh viên Nhưng mà thực tế khó làm anh Có thể với điều kiện giáo viên dạy lớp họ theo sát sinh viên họ biết trình độ sinh viên nào, họ feedback Giống có lúc em dạy có lớp em làm chuyện (việc cho nhận xét), người học trị thích feedback cảm ơn (U1.T3) Page in thesis Chapter Five: CONTENT RELEVANCE OF SPEAKING TEST QUESTIONS 184 Dạ em lớp học kỹ nói giống hỗ trợ thơi, phần cịn lại thân Có câu hỏi khơng có sách vượt tầm tụi em Cần phải trao dồi kiến thức thêm, ngồi trả lời câu hỏi (U2G2.S3) 186 Bài kiểm tra giúp giáo viên nhận sinh viên mạnh yếu điểm Qua việc em nói biết em đạt mục tiêu môn học đến mức độ để (giáo viên) có phương pháp phù hợp giúp sinh viên cải thiện kỹ nói Nhiều em cần bổ sung kiến thức liên quan tới nội dung nói khả phát âm (tiếng Anh) em có Theo em nghĩ, việc học ngoại ngữ khơng nói ngơn ngữ mà cịn cần phải biết nói Cho nên kiến thức phần quan trọng để nói tốt (U3.T2) Page in thesis Chapter Six: SPEAKING TEST TASKS 236 Sinh viên bắt buột phải biết cách thuyết phục bạn đồng ý với quan điểm mình, phải biết cách disagree với bạn mình, biết cách debate cách tranh luận với Sinh viên cần ý (cái) phrases phải dùng formal language mà anh disagree mà anh phản biện lại ý bạn em phải đạt đến mục tiêu Trong phần mơ tả hình em phải nói (cái) main points mà em nhìn vào vấn đề Ví dụ anh nhìn vào hình anh mơ tả hình theo viewpoint anh Một hình mơ tả khác anh có quan điểm anh mà anh phải nhìn theme, chủ đề tranh (U1.T1) 237 Đa số thí sinh làm không tốt, không tốt ba phần: thứ nhứt nội dung ngôn ngữ (language content), thứ hai kiến thức ngôn ngữ (language knowledge), thứ ba chức ngơn ngữ (language functions) Bởi q trình học em khơng có lưu tâm đến, khơng có get involed vơ hoạt động mà lớp em khơng quen với content đó, khơng có familiar với nội dung Khi thi tồn nội dung nằm sách hết, em khơng trình bày tốt Thứ hai kiến thức ngơn ngữ, ví dụ văn 490 phạm từ vựng, em khơng quen với nội dung nên em khơng có nghĩ từ mà liên quan, văn phạm lấp bắp cố trình bày thơi, thật cấu trúc khơng tăng mức tối đa mong muốn Cịn phát âm mang tính chất cố hữu thơi, số em phát âm tốt, số em khơng trọng luyện tập cách phát âm Cịn chức ngơn ngữ nội dung khơng nắm được, kiến thức ngơn ngữ khơng tốt việc turn-taking em sử dụng ngôn ngữ lại khơng tốt (U2.T1) 239 Kỳ thi hội để thể học mà từ vựng thi nói liên quan đến từ vựng học được, chủ yếu dựa vào kinh nghiệm cá nhân biết thơi Mình nhớ nói đó, biết nói (U2G2.S4) 239 Những lúc thi xong em hay hối hận lúc lại nghĩ nói Có khoảng thời gian ngắn quá, khơng kịp tư hết được, nói hết khả (U1G1.S1) Page in thesis Chapter Seven: RATER CHARACTERISTICS AND RATING ORAL SKILLS 269 Em thường hay kiếm điểm cộng không kiếm điểm yếu để trừ Học viên khơng q giỏi để kiếm điểm trừ Thường sinh viên mức độ kiếm điểm cộng khơng kiếm điểm trừ (U3.T1) 270 Theo em nên hai giám khảo đánh giá Mỗi người có cảm nhận riêng nên nhìn riêng thí sinh nên em nghĩ nên hai giám khảo đánh giá đa dạng hơn, đa chiều nhiều mặt hơn, rút nhiều kinh nghiệm xác (U1G1.S1) 271 Em khối có nhiều giám khảo đánh giá Khi lỡ mà giám khảo chấm điểm thấp có giám khảo khác người ta giải thích “khơng thể thấp tơi thấy khía cạnh em tốt mà” có người nói nên phân tích kĩ với ý kiến (U3G2.S3) 271 Em thấy giám khảo thơi tốt nhứt Tại người ta nghe nói người ta ghi lại nói Người ta cảm thấy yếu chỗ nào, tốt chỗ người ta ghi lại Chứ khơng thiết có hai giám khảo Càng nhiều đơng áp lực thầy, em khơng thích (U2G1.S1) 271 Theo em giám khảo mà giám khảo phải có phong thái giám khảo đừng vơ phịng thi người đứng bên đây, người đứng bên kia, người bên ngó em khơng thích Khi giám khảo vơ bàn ngồi nói chuyện người bình thường thí sinh cảm thấy thoải mái hơn, thi tốt (U3G2.S1) Page in thesis Chapter Eight: IMPACT OF ORAL TESTING ON EFL TEACHING AND LEARNING 305 Buổi hơm trước phịng thi em sinh viên em cảm thấy khơng khả quan lần trước em gác thi… Em rút kinh nghiệm nhiều lắm.Thứ bạn không tự tin, muốn bạn tự tin phải làm gì, vô lớp bắt cho thảo luận nhiều, dành thời gian cho đứng lên nói cầm micro nói chung trước lớp phải thể nhiều tốt, trước đám đông bạn bước tự tin Đa số bạn thi mà bị vấp váp có nhiều bạn nói kiến thức tốt, ngơn ngữ tốt mà lúc diễn đạt không tốt bị lấp vấp, bị ngưng, bị thời gian nói chung silent time nhiều bạn chưa diễn đạt ý nhiều mà bị run nên trước em nghĩ lớp phải tập cho bạn nói nhiều, thảo luận nhiều, thực hành nhiều để bạn tự tin (U2.T2) 306 Em nghĩ có Tại qua đợt em học từ người thi, học từ người gíao viên đứng lớp Em dạy cho sinh viên trình tự buổi thi chẳng hạn, câu hỏi nào, dùng mẫu câu Rồi với dạng đề đó, câu hỏi có cách trả lời để ghi điểm (U1.T3) 491 309 Cái sinh viên Việt Nam làm khơng mind map giống take turn Giống em nói thích ăn kem, em nói lại thích uống Pepsi Rồi hai người qua lại đến hết chán có nghĩa vậy Ví dụ bạn thích uống Pepsi, bạn thích ăn loại kem mà khơng thích ăn kem vui tạo điều Cái thiếu kỹ sinh viên mà tới lớn bị Khi anh nói chuyện với Tây anh thấy có nghĩa Tây họ nói chuyện họ nói rõ quan điểm họ cịn ngại cịn nhiều hỏng dám nói khơng đồng ý quan điểm Chắc văn hóa anh Văn hóa không muốn disagree cách frankly (U2.T3) 492 ... between assessing speaking EFL skills in Vietnam and that in the world Bridging the gap is crucial to enhance the quality of EFL teaching, learning, and assessment in the Vietnamese educational... teaching EFL speaking skills 304 8.3.4 Implementing a new method of assessing speaking skills 314 8.4 Summary 316 CHAPTER NINE: SUMMARY AND CONCLUSIONS 317 9.1 Introduction... perform in a specified sociolinguistic context (Spolsky, 1978; Morrow, 1979) The psycholinguistic-sociolinguistic trend L2 learning involves mastering its skills (listening, speaking, reading and