The paper not only describes what such a holistic approach to English language evaluation involves but also investigates how it is being implemented in reality in an English language program in Hanoi University.
Ti u ban 1: Đào t o chuyên ng ĐÁNH GIÁ CHƯƠNG TRÌNH GIẢNG DẠY TIẾNG ANH THEO CÁCH TIẾP CẬN TOÀN DIỆN, HƯỚNG TỚI CẢI THIỆN CHẤT LƯỢNG DẠY VÀ HỌC: NGHIÊN CỨU ĐIỂN MẪU TẠI TRƯỜNG ĐẠI HỌC HÀ NỘI Ngô Tuy t Mai Trường Đại học Hà Nội Tóm t t: Việc đánh giá chương trình giảng dạy tiếng Anh ngày thu hút quan tâm nhà quản lý chương trình, nhà lập kế hoạch, nhà thiết kế chương trình nhà sách Việc ñánh giá nhằm hai mục đích chính, (i) giải trình (ii) cải thiện phát triển chương trình Trong việc ñánh giá theo ñịnh hướng giải trình thường xem xét ñến ảnh hưởng chương trình giảng dạy tiếng Anh sau chương trình kết thúc thường tổ chức nhằm phục vụ mục đích người có quyền định, việc đánh giá theo Giáo dục Quốc tế, Trường Đại học Hà Nội Abstract: Evaluation of English language programs has become of an increasing interest to program managers, program planners, educators and policy makers Two major different purposes for English language program evaluation are (i) program accountability and (ii) program development While accountability-oriented evaluation examines the effects of an English language program at significant end points of an educational cycle and is usually conducted ñịnh hướng nâng cao chất lượng lại nhằm mục đích for the benefit of an external audience or decision cải thiện chất lượng chương trình trình maker, development –oriented evaluation aims at thực improving the quality of a program as it is being Nhằm ñạt ñược hai mục đích quan trọng cần phải có cách tiếp cận khác ñánh giá chương trình giảng dạy tiếng Anh Bài viết nhấn mạnh tầm quan trọng việc cần phải tiến hành song song đánh giá chương trình hướng tới đạt hai mục đích – giải trình cải thiện phát triển chất lượng chương trình Bài viết trình bày nghiên cứu điển mẫu Trường Đại học Hà Nội, nơi ñã thử nghiệm thành cơng việc đánh giá chương trình giảng dạy theo cách tiếp cận toàn diện Bằng chứng từ nghiên cứu trường hợp cho thấy ñể ñáp ứng ñược hai mục đích – giải trình cải thiện chất lượng chương trình, tiếp cận tồn diện việc đánh giá chương trình dạy tiếng Anh cách tiếp cận phù hợp, thơng qua việc tập trung vào đánh giá nhiều khía cạnh khác nhau, từ việc thiết kế chương trình, nội dung chương trình, quy trình dạy học lớp học, sinh viên, giáo viên, ñào tạo giáo viên, mơi trường học tập, việc đưa định quan giáo dục Bài viết khơng miêu tả chi tiết cách tiếp cận toàn diện đánh giá chương trình giảng dạy tiếng Anh mà khai thác cách thức thực phương pháp tiếp cận thực tế Trung tâm 156 implemented Toward achieving such two major different purposes, different approaches to evaluation of an English language program are needed This paper takes both purposes of program development and program accountability into consideration and presents a case study conducted at Hanoi University where a holistic approach to evaluation has been successfully taken of an English language program towards improving its academic quality The evidence from the case study well illustrates that to fit for the dual purposes of program accountability and program development, a holistic approach to English language evaluation is recommendable by focusing on many different aspects, ranging from program design, program content, classroom processes, the students, the teachers, teacher training, the learning environment, decision making to the institution itself The paper not only describes what such a holistic approach to English language evaluation involves but also investigates how it is being implemented in reality in an English language program in Hanoi University Key words: evaluation, program development, English language program, academic quality Chi n l c ngo i ng xu th h i nh p Tháng 11/2014 A HOLISTIC APPROACH TO EVALUATION OF AN ENGLISH LANGUAGE PROGRAM TOWARDS IMPROVING ITS ACADEMIC QUALITY: A CASE STUDY AT HANOI UNIVERSITY Introduction Evaluation of an English language program really matters It really matters as it is concerned with answering a wide range of questions such as whether the program in place is responding to learners’ expected needs, whether it is achieving its goals, or whether students are learning sufficiently from it, whether further teacher training is required for teachers working in the program, or more generally, whether those affected by the programs (e.g., teachers, administrators, students, employers, external agencies) are satisfied with the program According to Richards (2005, p 286), evaluation of an English language program “focuses on collecting information about different aspects of a language program in order to understand how the program works, and how successfully it works.” Based on such important information collected, concerned stakeholders will make different kinds of important decisions about the program This explains why evaluation of English language programs has become of an increasing interest to such key stakeholders as program managers, program planners, educators and policy makers As far as the Vietnamese public universities’ context (where many English language programs are run) is concerned, evaluation has not practically been paid due attention to Many efforts have been made instead into planning and implementing English language programs without reflective and purposeful analysis of the practices that are involved in planning, teaching and implementing the programs In order to help Vietnamese program managers, planners, educators and policy makers to collect adequate information, analyze reflectively practices involved in planning and teaching/implementing a language course, and to make wise decisions about the English language programs, this paper (i) reviews relevant the literature on the topic of curriculum evaluation, (ii) explores different purposes for English language program evaluation, and (iii) presents a case study conducted at Hanoi University where a holistic approach to evaluation has been successfully taken of an English language program towards improving its academic quality Approaches to Curriculum Evaluation 2.1 The Nature of Curriculum Evaluation There are many different aspects that evaluation may focus on Such aspects are many, ranging from the quality of program planning and organization, the syllabus and program content, classroom processes, teaching materials, the teachers, teacher training, the students, monitoring of student progress, learner motivation, the institution, learning environment to staff development and decision making (Sanders, 1992; Weir & Roberts, 1994) It is also important to note that all such key factors represent an overall and interlinked system of elements (i.e., needs, goals, teachers, learners, syllabuses, materials and teaching) Such elements are those determining the successful design and implementation of language programs (Richards, 2005) In other words, the scope of evaluation should not be limited Rather it should be comprehensive For a language program to be successfully designed and implemented, evaluation should focus on all such key elements However, in reality, evaluators who are program managers, program planners, educators and policy makers, tend to focus on one single aspect at a time, or in some cases, focus on few aspects at a time As a consequence, judgments about one aspects or some aspects have been made without due consideration to other equally important aspects and decisions have been thus made without adequate information and evidence The author of this paper argues that an effective approach to evaluation should focus on all the key aspects well described by Sanders (1992) and Weir and Roberts (1994) because all those aspects from program planning to 157 Ti u ban 1: Đào t o chuyên ng implementation are all worth being evaluated and any lack of judgments over any of those aspects may consequentially result in inadequately informed decisions about the program 2.2 The Audience of Evaluation Language teaching programs have many different levels of involvement and thus create different kinds of audiences for evaluation According to Elley (1989), to conduct an evaluation, it is important to identify who the different audiences are and what kind of information they are most interested in In other words, different audiences of evaluation might be interested in different questions For example, students want to evaluate whether they learnt something from the course and/or how well their performance compared to other, or whether they need another course Whereas teachers teaching the materials might be primarily concerned that the books provide sufficient material for all the classes on the given timetable Teachers might also be interested in knowing whether students were satisfied with the course and how effective the course organization was Program administrators might be interested in knowing whether the timeframe of the course was appropriate or whether the program has achieved the set objectives and the learning outcomes At the macro level, officers in the Ministry might be keen on knowing whether how the money provided for the project is well spent and whether all components of the invested language teaching program or project are available and well implemented according to the schedule and the set objective The sponsors of the course were keen on such information as whether the cost of the course was justified, or if the course delivered what was promised, or if the course was well managed 2.3 Purposes of Evaluation: Program Accountability vs Program Development Purposes of evaluation vary according to the intention of program managers, planners, educator and policy makers Weir and Roberts (1994) review the literature and make a distinction between two major purposes for language program evaluation: (i) program accountability 158 and (ii) program development Accountability, in their words, “refers to the extent to which those involved in a program are answerable for the quality of their work” (Weir & Roberts, 1994, p 288) Accountability-oriented evaluation thus examines the effects of an English language program at significant end points of an educational cycle and is usually conducted for the benefit of an external audience or decision maker Whereas program development refers to the quality of the program being evaluated itself and development –oriented evaluation aims at improving the quality of a program as it is being implemented Development-oriented evaluation has a development focus and it may involve staff who are directly involved in the program, for example, language teachers and it has a teacher – development focus 2.4 Approaches to Evaluation In order to achieve different purposes, different approaches to evaluation are suggested Using a comprehensive typology of approaches to evaluation, Richards (2005) referred to three approaches, namely, formative, illuminative, and summative evaluation Each of these three approaches will be discussed in details below, starting with formative evaluation 2.4.1 Formative Evaluation Approach Formative evaluation is carried out on an ongoing basis, as part of the process of program development with the aim of finding out what is successful and what is not and what challenges or issues need to be addressed This type of evaluation is generally known as formative evaluation in the evaluation literature Its focus is on the continuous development and improvement of the program Formative evaluation typically attempts to answer a wide range of questions in relation to, among others, whether enough time has been allocated to particular objectives, whether teachers’ used methodologies are appropriate, and whether students benefit from the teaching program Rich information is often collected during formative evaluation, ranging from students’ achievement outcomes, to teachers’ methodologies, programs’ teaching materials and Chi n l c ngo i ng xu th h i nh p the adequacy of the pacing of the material Such information is often used to not only identify issues and problems and to address the identified ones and to improve the implementation of the program for the sake of a better quality program Formative evaluation, during the implementation of any new or old programs, can be carried out creatively in different forms or in a combination of various forms or evaluation activities, ranging from workshops, review meetings (with such stakeholders as teachers and students), to teachers’ reports, peer observations and student needs analysis surveys Such evaluation activities or a combination of all those activities can reveal a holistic picture of what is working well and what is not, and what need to be addressed in the program before specific actions are taken to improve it 2.4.2 Illuminative Evaluation Approach Besides formative evaluation, another common type of evaluation, generally known as illuminative evaluation is often carried out Illuminative evaluation is often described as evaluation that “seeks to find out how different aspects of the program work or are being implemented” (Richards, 2005, p 289) Unlike formative evaluation which covers a wide range of aspects of a program, illuminative evaluation often focuses on one single aspect of a program Rather than providing a holistic or comprehensive picture of a program, illuminative evaluation seeks to illuminate or provide a deeper understanding of the processes of teaching and learning that occur in the program without necessarily seeking to change the course in any way as a result Different illuminative evaluations focus on different aspects of the program, depending on what single aspects that evaluators and program managers are keen on illuminating Within a certain illuminative evaluation, certain questions might be asked to find out how a certain aspect of the program work or is being delivered or implemented If error-correction strategies are of evaluators’ interests, such questions as what type of and how error-correction strategies are used by teachers might be asked In case teacherstudent interaction patterns are of their interest, Tháng 11/2014 the question as to what type of teacher-student interaction patterns typically occur in classes To conduct illuminative evaluation, various evaluation activities can be of use by evaluators who are language teachers, for example, the questionnaire surveys among stakeholders including teachers and students or classroom observations with purposeful investigation, the use of recorded videos can be of use The most popular type of illuminative evaluation is classroom action research Block (1998) highlights the importance of using classroom action research as a type of illuminative evaluation in understanding learners’ interpretations of the language courses they attend and how learners make sense of their lessons Block recommends that teachers should interview learners on a regular basis to get their perspectives regarding what is going on in the course Through classroom action research, teachers can collect evaluation information about learners and their performance over a certain period of time, normally over an academic term of fifteen weeks or so, using classroom observation, learner journals, interviews and test/exam results In some situations, such an illuminative evaluation activity of action research might provide some surprising findings In other situations, evaluators or action researchers might not learn anything particularly surprising from their evaluation or investigation, such information collected from action research can help confirm and make explicit some things which they knew intuitively Teachers can learn a useful strategy or technique to use in order to more effectively facilitate their students’ learning As a result, the illuminative evaluation can provide teachers/evaluators with answers to such questions and how the teacher (in spotlight) went about doing a certain thing (e.g., group work, reading strategies, helping students understand the teacher’s intentions), and which way of doing it works best for the teacher 2.4.3 Summative Evaluation Approach Summative evaluation is a type of evaluation seeking to determine the effectiveness, its efficiency and its acceptability of a program and to make decisions about the worth or value of 159 Ti u ban 1: Đào t o chuyên ng different aspects of the curriculum According to Richards (2005), most teachers and program administrators are familiar with this approach to evaluation which takes place after the completion of a program Shaw and Dowsett (1986, p 66) suggest that three audiences are identifiable for all summative evaluation of language course, namely, other teachers in the program (the main audience) for course design and planning purposes, managers of the institution or program for the purpose of determining course offerings and placement, and the curriculum support or development unit for the purpose of monitoring the curriculum This approach generally seeks answers to a wide variety of questions such as to how effective the course was, whether it achieved its aims, or how well the course was received by students and teachers, or how appropriate the teaching methods were Answers to all such questions help reveal the effectiveness of a course or a program In order to decide whether a course is effective or not, criteria for effectiveness and different measures of a course’s effectiveness are identified Measures of a course’s effectiveness include mastery of objectives, performance on tests, measures of acceptability, retention rate or reenrollment rate, and efficiency of the course (Weir, 1995) and each measure can be used for different purposes (Richards, 2005) For example, to measure the extent to which the students have mastered a certain objective at the end of the course, each objective set in the course is examined and criteria for students’ successful achievement of each objective are chosen However, the mastery of objectives is not sufficient and does not always provide a full picture of the effectiveness of a course The reality often shows that objectives can still be achieved despite defects or shortcomings in the course Or perhaps mastery of an objective was achieved simply because students spent a lot of extra time in private study to compensate for the poor teaching performance or insufficient materials provided during the course Therefore other more formal ways of assessing mastery of objectives such as formal tests (e.g., unit tests given at the end of each unit of teaching materials, class tests 160 or quizzes devised by teachers and administered at various stages throughout the course) are used to measure students’ achievement Besides measures of mastery of objectives and performance on tests, measures of acceptability can be used through assessments of teachers and students Research shows that a course might lead to satisfactory achievement of its objectives and good levels of performance on exit tests yet still be negatively evaluated by teachers or students In fact, reasons for a course being considered acceptable or unacceptable might relate to such factors as timetabling, class size, choice of materials, or teachers’ teaching styles Another measure of the success of the course is concerned with the efficiency of the course or how straightforward the course was to develop and implement According to Richards (2005), the efficiency of the course can be determined through a reflection of the number of problems that occurred during the course, the time spent on planning and course development, the need for specialized materials and teacher training, and the amount of time needed for consultation and meetings To sum up, Richard (2005) suggests three different approaches of evaluation, each of which can achieve its own purposes While formative evaluation is conducted on the on-going basis as part of the process of program development to find out what is working well, and what is not, illuminative evaluation provides insights into the processes of teaching and learning occurring in the program Summative evaluation is conducted at the end of the program to determine its effectiveness, efficiency and its acceptability As each of these three approaches to evaluation, by nature, has its own purposes, own audiences and characteristics, it is important to choose the appropriate approach to evaluation to fit its purposes and audiences If the purpose of evaluation is for program accountability examining the effects of a program at significant end point, summative evaluation should be the appropriate choice Whereas if a program is evaluated for the purpose of program development Chi n l c ngo i ng xu th h i nh p aiming at improving the quality of a program as it is implemented, illuminative evaluation and/or formative evaluation are more appropriate approaches All the three approaches to evaluation – namely, formative evaluation, illuminative evaluation and summative evaluation seem to take narrow view of evaluation While summative evaluation has its central interest in demonstrating the “product value” of a program or its components (Weir & Roberts, 1994), illuminative and formative evaluation have associated commitments to a deeper professional understanding of processes of teaching and learning in the program This paper argues that program evaluation should take into careful consideration both the product and process of implementation of the program It should not be either the product or the process that is solely examined In light of this view, a broader view of evaluation is proposed and a holistic approach to evaluation seems more appropriate Also to serve the dual purposes of both program accountability and program development, a holistic approach to evaluation might be the most appropriate The so-called holistic approach is known as a type of evaluation which is conducted to examine the effects of all key aspects of a program both during and after its implementation Unlike the three other approaches proposed by Richards (2005), the holistic approach offers a more broad view of evaluation that, according to Weir and Roberts (1994, p 42) is characterized by (1) the need for both insider and outsider commitment and involvement to ensure adequate evaluation; (2) dual interest in both improvement and the product value of a program or its components; (3) an associated commitment to a deeper professional understanding of the processes of educational change as well as the results of that change; (4) a systematic documentation for evaluation purposes both during implementation and at the beginning and end of a program or project’s life; and (4) a willingness to embrace both qualitative and quantitative methodology appropriate for the purpose of evaluation and the context under review Tháng 11/2014 A Case Study in International Education Center, Hanoi University: A Holistic Approach to Evaluation 3.1 Context The International Education Center (IEC) is the Center affiliated to Hanoi University It was established in 2007 and its main mission is to offer Hanoi University’s joint training programs (including both Bachelor programs and Master programs) in cooperation with international universities Since its establishment, hundreds of students have successfully completed their studies and graduated with bright career future However, the number of students enrolled into the programs offered by the Center has been dramatically reduced, partly due to the economic crisis, partly due to the fierce competition in the education market among many reputable universities and partly due to its seemingly uncompetitive performance of the center Given this context, there was an urgent need for an appropriate approach to evaluation in the Center so that all aspects of training programs offered in the Center would be under scrutiny both at the beginning, during, and at the end of its programs’ life for the purpose of both program accountability and program development in IEC A holistic approach to evaluation is thus proposed in order to improve IEC’s program credentials, its performance, its competitiveness in the market, and ultimately to improve its reputation and its attraction of more and more students choosing to study in IEC’s Bachelor’s and Master’s programs 3.2 Implementation of the Holistic Approach to Program Evaluation in IEC, Hanoi University To fit its context and its evaluation purposes, the holistic approach to program evaluation was implemented in IEC, Hanoi University, starting in early April 2014 right after the researcher was appointed as the Director of the Center For the implementation of this holistic approach, both qualitative and quantitative methodology was employed, targeting at various audiences for evaluation, involving various participants in the evaluation process In language program 161 Ti u ban 1: Đào t o chuyên ng evaluation, according to Richards (2005, p 297), “both quantitative and qualitative approaches to collecting information are needed, because they serve different purposes and can be used to complement each other.” For quantitative evaluation, information was collected from a large number of people on specific topics and can generally be analyzed statistically so that certain patterns and tendencies emerge Questionnaire surveys were sent to IEC’s alumni, current students official and non-official teachers, and administrative staff for better understanding of the current context that IEC is situated in Externally, quick surveys were made in other contexts outside IEC, both those inside Hanoi University and in other similar Centers or Institutes in other competitive universities in Hanoi, including FPT University, the National Economics University, Foreign Trade University, Vietnam National University, and Vietnam Technology University Such quantitative information as course statistics, the number of students enrolled in each International Education Center/Institute, the number of international joint programs offered, their charged tuition fees have provided insights into the patterns of student admission and their competitiveness Though quantitative data are regarded as “rigorous” or conforming to scientific principles of data collection, the limitations of quantitative information are also recognized and there is the need to complement such information with qualitative information As soon as the researcher became the Director of IEC, with full support of both internal academic and non-academic staff, the researcher made detailed schedules for classroom observations, academic and nonacademic staff interviews, policy document analysis, course documents, course reviews, teacher satisfaction, observation of IEC’s student performance in classroom tasks Externally, field trips were made for IEC’s Board of Directors and key staff members (including the library staff) to visit five International Education Centers/Institutes in other reputable universities in Hanoi, namely those in FPT University, the National Economics University, Foreign Trade 162 University, Vietnam National University, and Vietnam Technology University During such field visits, interviews with Directors and Managers of those Centers were conducted to get deeper understanding of what is going on in their Centers and how they address their challenges and fierce market competition Together with quantitative information, such qualitative information can reveal the holistic picture of not only what is going on inside but also outside IEC, highlighting IEC’s strengths and weaknesses as well as its opportunities and challenges in relation to academic, administrative, financial and leadership matters In light of such internal and external situational/environmental analyses and evaluation, new strategies associated with new action plans for further improvement of IEC’s performance and its competitiveness are proposed to Hanoi University’s top leaders for their consideration and final decisions As for the audiences for evaluation, this holistic approach to evaluation identified its key audiences, including students, teachers, curriculum developers, administrators, and the curriculum support or development units For different audiences, the holistic evaluation is interested in different kinds of information for a wide variety of purposes, ranging from course design and planning purposes, to the purpose of determining course offerings and placement, or curriculum monitoring or quality control Two main types of participants were involved in the holistic evaluation conducted in IEC – insiders and outsiders Insiders are teachers, students, and anyone else (e.g., parents) closely involved in the development and implementation of training programs in IEC The involvement of such key insiders in the process of designing and carrying out evaluation is an important factor, because as a consequence, as Richards (2005, p 296) puts it, “they will have a greater degree of commitment to acting on its results.” Outsiders who are not directly involved in the program are consultants, inspectors, and administrators whose insights can supplement the insiders’ perceptions of what happened in IEC wit independent observation and more objective views of various aspects of training programs offered in IEC Chi n l c ngo i ng xu th h i nh p 3.3 Lessons learnt from the Case Study From this case study in IEC where the holistic approach to evaluation was implemented at the beginning, during and at the end of each semester or each training program, a lot of practical lessons have been learnt Due to the scope of this paper, only two significant lessons were highlighted Firstly comprehensive reports highlighting the findings of the holistic evaluation were made for various audiences including university leaders, IEC’s Board of Directors, their teachers and curriculum developers Evaluation’s findings in the format of reports provide an overview, a summary of all the information collected, the strength and weaknesses of IEC itself and its training programs, and most importantly a systematic documentation of recommendations for modifications in different aspects of the program as well as more University-based support for some aspects of the program Suggestions for teachers and students were also made in order to promptly adjust their methodology of teaching and learning All such recommendations and suggestions are basic foundation for specific actions to be taken by all the stakeholders, including those top leaders of Hanoi University and those directly implementing training programs offered in IEC The second significant lesson learnt is that the holistic approach to evaluation, as the case study in IEC suggests, is an appropriate approach that fits for the purpose of promoting review, reflection, and revision of performance and the curriculum based on careful compilation of information For an effective implementation of the holistic approach to evaluation on which key decisions can be made, the case study in IEC places emphasis on the importance of the adequate design and the carefully considered process of evaluation To this, careful consideration should be taken into the scope (the range of information collected that include all the significant aspects of the program being evaluated), the audience (the collected information adequately serve the needs of all the intended audiences), the reliability, objectivity, representativeness, timeliness and ethical considerations (whether the evaluation followed Tháng 11/2014 the accepted ethical standards) as well As the holistic approach to evaluation implemented in IEC meets those acceptable standards of adequacy, most of the information obtained can be made full use of for the dual purposes of both program accountability and program development Conclusion In conclusion, this paper highlights the need for an appropriate approach to evaluation that fits the dual purposes of program accountability and program development The evidence from the case study in the International Education Center, Hanoi University well illustrates that to fit for such dual purposes, a holistic approach to English language evaluation is recommendable by focusing on many different aspects, ranging from program design, program content, classroom processes, the students, the teachers, teacher training, the learning environment, decision making to the institution itself Using the empirical research method of case study, the paper not only describes what such a holistic approach to English language evaluation involves, its quantitative and qualitative methodology of evaluation, audiences and participants in the evaluation process but also investigates how it is being implemented in reality in a case study in Hanoi University, highlighting the two main practical lessons learnt from the case study This is an empirical research and thus has its own limitations Further research supported by sound theories is needed to provide further indepth understandings and insights into adequacy and appropriateness of approaches to evaluation REFERENCES Block, D 1998 Tale of a language learner Language Teaching Research 2(2): 148-176 Elley, W 1989 Tailoring the evaluation to fit the context In R K Johnson (ed.), The Second Language Curriculum New York: Cambridge University Press 270- 285 Richards, J C 2005 Curriculum Development in Language Teaching Cambridge University Press CUP Sanders, J R 1992 Evaluating School Programs: An Evaluator’s Guide Newbury Park, CA: Corwin Press Weir, C and J Roberts 1994 Evaluation in ELT Oxford: Blackwell 163 ... A HOLISTIC APPROACH TO EVALUATION OF AN ENGLISH LANGUAGE PROGRAM TOWARDS IMPROVING ITS ACADEMIC QUALITY: A CASE STUDY AT HANOI UNIVERSITY Introduction Evaluation of an English language program. .. approach to evaluation has been successfully taken of an English language program towards improving its academic quality Approaches to Curriculum Evaluation 2.1 The Nature of Curriculum Evaluation. .. holistic approach to English language evaluation involves, its quantitative and qualitative methodology of evaluation, audiences and participants in the evaluation process but also investigates how