EVALUATION OF UNICEF-SUPPORTED MOET’S INITIATIVE OF MOTHER TONGUE BASED BILINGUAL EDUCATION IN VIET NAM 2006 - 2014

Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT EVALUATION OF UNICEF-SUPPORTED MOET’S INITIATIVE OF MOTHER TONGUE BASED BILINGUAL EDUCATION IN VIET NAM 2006 - 2014 - Final Evaluation Report December 2015 i Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT TABLE OF CONTENTS ABBREVIATIONS AND ACRONYMS v ACKNOWLEDGEMENTS 1.0 INTRODUCTION 2.0 EVALUATION, PURPOSE, OBJECTIVE AND SCOPE 2.1 Purpose 2.2 Evaluation Objectives 10 2.3 Scope 10 3.0 EVALUATION METHODOLOGY 11 3.1 Data Collection Methods and Sources 12 3.2 Data Analysis 13 3.3 Data Gaps and Limitations 3.3.1 Data quality assurance and sound analysis 3.3.2 Data gaps and limitations 13 14 14 3.4 Ethical Standards 16 3.5 The Theory of Change 17 4.0 FINDINGS 20 4.1 Relevance 4.1.1 Defining relevance 4.1.2 Summary regarding Relevance 4.1.3 Relevance to the Vietnamese - context 4.1.4 Relevance to international treaties and commitments 4.1.5 Relevance to the Vietnamese legal and policy frameworks 4.1.6 Role in improving educational policy regarding the ethnic minority people 4.1.7 Relevance to UNICEF´s renewed focus on equity 20 20 20 21 22 25 28 29 4.2 Effectiveness 30 4.2.1 Defining effectiveness 30 4.2.2 Summary on effectiveness 30 4.2.3 Effectiveness in creating conditions to improving the learning outcomes of participating children 31 4.2.4 Effectiveness in creating conditions to improving the capacity of teachers and education managers 35 4.2.5 Effectiveness in considering contextual factors in MTBBE design and implementation 38 4.3 Impact 41 ii Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT 4.3.1 Defining impact 41 4.3.2 Summary on impacts 41 4.3.3 What impacts did the MTBBE approach have on the learning outcomes of participant students? 42 4.3.4 Impact on teachers, school management, board members and all level education managers 50 4.3.5 Impact on parents and the community 52 4.4 Efficiency 4.4.1 Defining efficiency 4.4.2 Summary on efficiency 4.4.3 Were activities cost-efficiency? 4.4.4 Were objectives achieved on time? 4.4.5 Was the programme or project implemented in the most efficient way compared to alternatives? 54 54 54 55 61 4.5 Sustainability 4.5.1 Defining sustainability 4.5.2 Summary on sustainability 4.5.3 Continuation of the programmes´ results in the AR MTBBE sites 4.5.4 Extent of capacity for scaling up in sites similar to the pilots 4.5.5 Elements in place/not in place for sustainability 64 64 64 67 71 73 5.0 LESSONS LEARNED 77 Identification of lessons learned 77 5.2 What could or should be done differently in future replications and/or scaling up? 78 6.0 CONCLUSIONS 79 7.0 RECOMMENDATIONS 83 REFERENCES 88 ANNEXES 92 Annex – Terms of Reference 93 63 Annex - Project Overview 109 Annex – Evaluation Design Matrix 119 Annex – List of People Consulted 139 Annex - Additional tables and graphs regarding the effects of the Action Research MTBBE on students´ learning outcomes 145 Annex - Selected testimonies on different issues 148 Annex – AR MTBBE Detailed Financial Information 158 Annex – Systematised list of benefits of the Action Research MTBBE 160 Annex – Additional information for the SWOT 162 Annex 10 – Summary of findings, conclusions and recommendations 169 Annex 11 – Questionnaires 183 iii Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT LIST OF TABLES Table Number of key informant interviews and focus groups, planned and actual 13 Table Brief general information of the target provinces 22 Table SWOT analysis of implied factors on the sustainability of MTBBE scaling up 73 LIST OF FIGURES Figure Percentage of MTBBE first cohort students according to grade and general scores 45 Figure Percent of students by language dimensions and scores in MOTHER TONGUE School year 20122013, 1st and 2nd cohorts 45 Figure Percent of students by language dimensions and scores in Vietnamese School year 2012-2013, 1st and 2nd cohorts 46 Figure Percent of fourth graders (First cohort) according to provinces and MTBBE / Non-MTBBE students, by scores in Vietnamese 48 Figure Per cent of fourth graders (First cohort) by provinces and MTBBE / Non-MTBBE students, by scores in Mathematics 48 Figure MTBBE Costs by component 58 Figure MTBBE Costs 2008-2014 60 Figure Schematic interpretation of the main intervening factors in the development of the AR MTBBE in the three participating provinces 70 Figure Path of factors involved in the replica and scaling up of the MTBBE in areas with characteristics similar to existing project sites 76 iv Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT ABBREVIATIONS AND ACRONYMS AR MTBBE Action Research on Mother Tongue-Based Bilingual Education BE Bilingual Education BOET Bureau of Education and Training Department (MOET´s district level) CEMA Committee for Ethnic Minorities DOET Provincial Education and Training Department (MOET´s provincial level) EAPRO UNICEF´s East Asia and Pacific Regional Office ECED Early Childhood Education Department of MOET EMED Ethnic Minority Education Department of MOET EM Ethnic Minority EML Ethnic Minority Language (s) NIESAC National Institute of Educational Strategy and Curriculum (under MOET) MLE Multilingual Education MoU Memorandum of Understanding between the Ministry of Education and Training and United Nations Children Fund Viet Nam of 15 August 2007, which defines the common understanding and operating modalities between the parties MOET Ministry of Education and Training MT Mother Tongue MTBBE Mother Tongue-Based Bilingual Education NIL National Institute of Linguistics v Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT PED Primary Education Department of MOET PID Project Initiation Document RCEME Research Centre for Ethic Minority Education, VIES SIL Summer Institute of Linguistics SWOT Strengths, Weaknesses, Opportunities and Threats Analysis ToR Terms of Reference UN United Nations UNESCO United Nations for Educational, Scientific and Cultural Organization UNEG United Nations Evaluation Group UNICEF United Nations Children's Fund VIES Viet Nam Institute for Educational Sciences1 This is the new name of the National Institute for Education Strategies and Curriculum since 2009 vi Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT ACKNOWLEDGEMENTS We thank to all relevant Government agencies, local and international organizations, experts, specialist, managers, who actively engaged in the preparation, field work and analysis of the evaluation We express great appreciation to Viet Nam Ministry of Education and Training, Departments of Education and Training in Lao Cai, Gia Lai and Tra Vinh, Ethnic Council of National Assembly/, UNICEF Viet Nam and UNICEF Regional Office for East Asia and the Pacific region for valuable guidance, inputs, technical supports, expertise and financial support for this evaluation In particular, our sincere gratitude goes to MOET Vice Minister Nguyễn Vinh Hiển, Trần Thị Thắm- Deputy Director of Primary Education Department, Prof Nguyễn Văn Kha- Former General Director of VIES and his team, Hà Đức Đà, RCEME director and his team, other entities within the MOET, managers of DOETs, BOETs; Madam Trieu Thi Nai, Vice-Chairman of the Ethnic Council of the National Assembly, and Nguyen Manh Quynh, Vice Director of the Ethnic Minority Department of the National Assembly Office; Mr.Youssouf Abdel-Jelil, UNICEF Viet Nam Representative, Jesper Moller-Deputy Representative Thanks to the close technical and management support by Joyce Patricia Bheeka- Education Chief, Vu Manh Hong-Chief, Monitoring and Evaluation and Dinh Phuong Thao and Nguyen Thi Thanh Nga, Inclusive Education Specialists, Nguyen Minh Nhat, Education Officer The Evaluation Reference Group was a valuable source of help with their comments on the Inception Report and the drafts of the Evaluation Report In the same vein we recognize the UNICEF Regional Office for East Asia and the Pacific whose valuable feedback improved this Evaluation Report Teachers, parents, community representatives, interpreters of the local languages deserve a special acknowledgement for their warm reception and readiness to answer our questions Without them we could not have done this work School children became the best part of our mission because their spontaneity, contagious joy and dreams tell us how much the MTBBE is transforming their lives as well as what to expect for the future of Viet Nam Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT EXECUTIVE SUMMARY Viet Nam has 54 officially recognised ethnic groups, many of which have their own distinct language and live in remote and economically disadvantaged areas2 By 2009 the total ethnic minority (EM) population was about 12.5 million, or 14 percent of the total population3 The official language of instruction in schools is Vietnamese, which is used to teach all children, regardless of their level of knowledge of this language This has created a “language barrier” for many EM children who have limited, or in some cases no, understanding of and/or proficiency in Vietnamese In this context, two key issues facing education in Viet Nam are: 1) how to cope with the needs of many different language groups, and 2) how to fulfil the right to education for all children in order that they learn and develop to their full potential The Action Research4 on Mother Tongue-Based Bilingual Education (AR MTBBE) has been an important initiative of UNICEF´s Education Programme during two Country Programmes of Cooperation 2006-2011 and 2012-2016 The AR has been implemented in the provinces of Lào Cai, Gia Lai and Trà Vinh, with the participation of students from three EM groups, i.e Mong, J´rai and Khmer The programme covered six years from pre-primary (one year) to primary education (five years) levels This is a cooperation project between the Ministry of Education and Training (MOET) and UNICEF since school year 2008 with two cohorts of EM children By the end of school year 2013-2014 (May 2014), the first cohort completed a six-year learning programme, followed by the second cohort by the end of the school year 2014-2015 in May 2015 The main objectives of the AR MTBBE are as follows: (i) To implement a valid and feasible design of bilingual education (BE) in ethnic minority languages (EML) and Vietnamese in pre-primary and primary schools; and (ii) To contribute to the development of policies and practices, including legal frameworks, that will promote the use and development of EML as a means to improve access, quality and equity of education and other social services The aim of this final evaluation, which covers the period 2007-2014, is to make an objective assessment of the MTBBE approach implemented through the AR MTBBE to date and document evidence and lessons learned for replication and scaling-up in the coming years This evaluation will serve MOET and UNICEF in fulfilling their mandate to “Successfully implement the AR MTBBE to inform the subsequent development of an appropriate and sustainable policy for BE in Viet Nam.” Other important users of the evaluation report are relevant policy-makers and decision-makers in MOET, CEMA, The World Bank, 2009, page 51 2009 Housing and Population Census of Viet Nam Action research is defined as observation, reflection, and planning activities which allow for improvement, correction, documentation and evaluation of project activities, in this case, what is happening in the selected schools Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT National Assembly, subnational authorities, education managers, teachers, UN agencies and various development partners The methodology for this evaluation was developed in accordance with the quality criteria set out in the context of the Organization for Economic Co-operation and Development (OECD) Development Assistance Committee’s (DAC) Quality Standards for Development Evaluation5, in line with the United Nations Evaluation Group’s (UNEG) Standards6 and Norms7, and with special reference to the provisions of the Global Evaluation Reports Oversight System8 (GEROS) The methodology was grounded in the Terms of Reference (ToR) and the approved Inception Report, where the latter included a detailed Evaluation Design Matrix In accordance with UNEG Standard 3.10, paragraph 21, the approved Inception Report constitutes the extent of the mutually agreed upon work programme The Evaluation follows a mixed methods approach, which created opportunities for the stakeholders’ participation in providing insights about pertinent issues This methodology used multiple lines of evidence that yielded robust information for the analysis The evaluation started in October 2014 and ended in August 2015 Fieldwork for data collection took place during the second half of November 2014, including a validation workshop Main Findings Relevance The MTBBE approach is relevant and responsive to the socioeconomic, legal and policy contexts of Viet Nam and UNICEF´s focus on equity MTBBE translates the Vietnamese legal and policy frameworks into a more inclusive and equitable education system, as expressed in the 2013 Constitution, revised 2009 Education Law, Education Strategic Development Plan (2011-2020), among other relevant laws and policy frameworks From the cultural point of view, the relevance of the programme is closely linked to its potential to contribute to the preservation and development of ethnic minority languages (EML) and cultural practices and beliefs Despite the attested enabling implementing environment, a review of the Vietnamese key legal and policy instruments suggests the need for revisiting and clearly strengthening the existing policies and laws towards MTB-MLE http://www.oecd.org/development/evaluation/qualitystandards.pdf http://www.uneval.org/document/detail/22 http://www.uneval.org/document/detail/21 http://www.unicef.org/evaluation/index_60830.html Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL EVALUATION REPORT Effectiveness The programme has been effective in developing and implementing a context-appropriate MTBBE model that has created conditions to improve the learning outcomes of participating EM children and the professional capacity of teachers and education managers By offering the programme from pre-school to the end of primary education and adopting a flexible approach, the curriculum design allowed the development of both the students’ mother tongues and Vietnamese right from the first day of schooling as well as offered optimal conditions for the transfer of knowledge from one language to the other Nevertheless, there were reports of misapplication of flexible teaching methodologies, as well as some hesitation about the teaching of Vietnamese as a second language, in particular in earlier stages of the programme Impact The MTBBE has produced factual and perceived impacts on its intended beneficiaries According to different internal/official assessments and evaluations, children’s learning outcomes have been at a high level in all schools of the three provinces, as compared with national standards In addition, children displayed confidence, self-esteem and good communication skills Teachers, school management, board members and all levels of education managers generally supported the programme and recognized its effect on improving their competences to successfully apply the MTBBE design and contributed to develop policies and practices promoting the use of EMLs Parents showed awareness and support to the MTBBE and recognized its benefits in the life of their children Efficiency Evidence provided by MOET and UNICEF indicated that monetary and time resources were used in an economically acceptable manner Stakeholders expressed awareness of the cost vs results relationship, and most think that the unit costs of the programme seemed high but compensated by its returns (opportunity costs), particularly its immediate impact on learning outcomes and teacher training, as well as its contribution to capacity building for policy making Sustainability The AR MTBBE has created favourable basic conditions for future replication and scaling up of the bilingual programme In addition the socio-political climate is also supportive of the scaling up of the MTBBE programme to new schools and new ethnic minority groups in Viet Nam Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Findings Impact Finding #3 The MTBBE has produced factual and perceived impacts on its intended beneficiaries According to different internal/official internal and evaluations, children learning outcomes have been acceptable in all schools of the three provinces, as compared to national standards In addition, children display confidence, self-esteem and good communication skills According to some indicators, girls have benefitted relatively most from the approach Teachers, school managements, board members and all levels of education managers generally supported the programme and recognize its effect in improving the their competences to successfully apply the MTBBE design and contribute to develop policies and practices promoting the use of EMLs Parents showed awareness and support to the MTBBE and recognized its benefits in the life of their children Conclusions Recommendations 8) The overwhelming positive academic results obtained by ethnic minority children, with special reference to girls, and its consistency with national standards confirm the pedagogical value of an education programme based on learners’ mother tongues The familiarity with the language of instruction makes EM children feel at ease in the classroom and creates the conditions for their active participation, which are important preconditions for learning The positive results achieved in Vietnamese also show that, instead of hindering, MTBBE facilitates the learning of the second language, the language regarded as the gateway to social integration and mobility in Viet Nam The general conclusion is that MTBBE has the potential to contribute to mind learning outcome gaps between Kinh and EM children and also between boys and girls, thus paving the way towards inclusion and equity in education and in the Vietnamese society at large 10) Conduct (longitudinal) studies of former MTBBE students’ performance at secondary school in order to assess the educational impact of MTBBE beyond primary education level The results of those studies will provide further contextspecific evidence about the feasibility and validity of the MTBBE for ethnic minority students in Viet Nam Responsibility: MOET (RCEME/VIES), in coordination with UNICEF in the search of resources and advocacy 175 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Findings Conclusions 9) As discussed, while recognizing the sociocultural value of using EMLs in formal education, parents and other community members were initially unsure about the pedagogical value of MTBBE, in particular in creating opportunities for effective learning of Vietnamese The fact that their attitudes and perceptions changed over the course of the programme, especially when they perceived substantial academic results exhibited by participating students, suggests that the achievement of positive educational and sociocultural outcomes may prove to be the best way to advocate for MTBBE In other words, while drawing on international experiences, advocacy initiatives should involve key actors (students, teachers, parents, education officials, etc.) and use the results from the pilot phase as the main arguments in favour of the MTBBE Recommendations 11) Develop a Behaviour Change Communication Strategy (campaign) to raise awareness of parents and communities on the purpose and benefits of the MTBBE and ensure their support and participation as well as to mobilize collective action in support of the programme Responsibility: MOET, in collaboration with UNICEF Findings Conclusions Recommendations Efficiency Finding #4 10) As international experiences have 12) Scale up the MTBBE programme, Evidence provided by MOET and UNICEF shown and also recognized by key involving more schools and EM students, 176 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Findings indicates that monetary and time resources were used in an economically acceptable manner Stakeholders expressed awareness of the cost vs results relationship, and most think that the unit costs of the programme are high but compensated by its returns (opportunity costs), particularly its immediate impacts on learning outcomes and teacher training, as well as its contribution to capacity building for policy making Conclusions participants in this evaluation, the expenditure of the pilot programme was higher than the average per pupil expenditure in primary education in Viet Nam because of its pilot nature The reliance on regular in-service training, the innovative effort placed on material development and the small size of the student population served are the main factors which explain the relatively high costs incurred in the pilot phase However, the positive results of obtained, including the improvement of learning outcomes of ethnic minority students and change of the mind-set of education managers regarding the role and value of EMLs in education indicate that the MTBBE is a worthwhile investment In fact, as attested internationally, in a post-pilot phase, the cost of the programme is likely to be lower and similar to that of the mainstream programme in Vietnamese 11) The availability of basic and good quality teaching and learning materials is a necessary precondition for a successful implementation of any educational programme The evidence gathered suggests that this precondition was, in general, fulfilled in the AR MTBBE Recommendations thus maximizing the academic and sociocultural benefits of the approach and reducing its costs by capitalizing on economies of scale Responsibility: MOET (DOETs, RCEME, VIES) 13) Consolidate all reference materials (i.e teaching guides, grammar books, dictionaries, additional reading and reference books, and visual aids) in the list of basic materials to be produced and supplied to MTBBE schools 177 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Findings Findings Sustainability Finding #5 The AR MTBBE has created basic favourable conditions for future replication and scaling up of the bilingual programme In addition the socio-political climate is also supportive of the scaling up of the Conclusions However, in addition to very basic materials, it is also necessary to furniture bilingual schools with complementary material to aid teaching and learning, in particular in EMLs As attested internationally, when students, teachers and parents understand that education programmes in low status languages are poorly equipped in terms of teaching and learning aids (and also in terms of quality of the teaching staff) when compared to programmes in the mainstream language, as it is the case of Vietnamese, they may opt out those programmes This is to be avoided, in particular considering the equity agenda underlying the provision of education in EMLs Recommendations Responsibility: MOET (DOETs, RCEME, VIES) 14) Mobilize independent writers and publishers to produce low-cost and/or nocost reference materials in EML for MTBBE schools; this may include award offers and promotion of contests and competitions culminating with the publication of the best manuscripts, including on-line publication Responsibility: MOET (DOETs, RCEME, VIES) Conclusions Recommendations 12) As international experience has shown, in addition to technical preparedness, political will is one of the key factors for a successful and sustainable implementation of MTBBE initiatives Although, in general, these conditions are in place in Viet Nam, there 15) Develop a strategy for a gradual expansion and scaling up of the MTBBE programme This should have two goals: (a) the continuity and dissemination of MTBBE in the three pilot provinces and (b) the design and implementation of new 178 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Findings MTBBE programme to new schools and new ethnic minority groups in Viet Nam However, for a sustainable scaling up of the MTBBE programme there will be a need to reform the education law to allow EML could be in the role of language of learning, together with supportive policies and permanent dialogue between central and provincial governments Conclusions are some other factors which may threaten the sustainability of the programme if not adequately addressed These include mainly claims of lack of financial resources to implement the programme in some provinces and the lack of a clear policy on MTBBE in Viet Nam In fact, the need of financial support for teacher training and material development is generally perceived by stakeholders, especially at the district and provincial levels, as the main challenge for a sustainable replication and scaling up of the programme Other less voiced constraining factors include the difficulty in managing language diversity in some provinces and the existence of alternative approaches to education of ethnic minority children, especially the teaching of Vietnamese as a second language, usually regarded as less expensive and less demanding Recommendations MTBBE interventions in provinces with characteristics similar to those of the provinces involved in the pilot stage or in other provinces fitting MOET’s criteria Responsibility: MOET, in coordination with UNICEF and relevant DOETs 16) Continue to provide financial support for teacher training and development of teaching and learning materials to the experimental provinces for three to five years while they will continue to find other donors to sustain the programme Responsibility: MOET, UNICEF and other donors of the education sector in Viet Nam 17) Provide support to teachers for them to effectively deliver a multi-language policy by ensuring pre- and in-service training which lead to the improvement of their own proficiency in the language of instruction and in teaching skills in ethnic minority languages, including early literacy development in these languages Responsibility: MOET 179 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Findings Conclusions Recommendations 18) Establish and maintain a dialogue with relevant provincial authorities in order to clarify responsibilities and find solutions for the financial needs of local governments to sustain current MTBBE efforts and results and set the bases for the expansion and scaling up of the programme in those provinces Responsibility: MOET and UNICEF 19) Bring language of instruction to the forefront of dialogue on education sector funding with the government partners, partially in relation to both quality and equity concerns, ensuring the coverage of MTBBE is highlighted as international indicator of education quality Responsibility: UNICEF development partners and other 20) Mobilize resources to support the introduction of MTBBE and offer technical assistant to develop flexible competency targets, literacy materials and an 180 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Findings Conclusions Recommendations assessment framework languages Responsibility: UNICEF development partners Findings General contribution to gender and equity In general terms MTBBE has also contributed to the objectives of UNICEF Viet Nam on gender and equity MTBBE is per se an equity-oriented policy instrument as soon as it attempts to fill ethnic and gender gaps in education by providing an effective opportunity to ethnic minority children to overcome the language barrier The efforts in terms of policy made under the MTBBE approach are also addressed to attain equity educational goals in favour of ethnic minority populations Conclusions Recommendations 13) The MTBBE approach has also proved to be relevant to UNICEF’s renewed focus on equity, particularly considering its contribution to promote inclusive policies on education and other social affairs By helping ethnic minority children to overcome the language barrier in school, MTBBE can be regarded as one of the platforms than can be used to improve the academic performance of these children and set the ground for official use of EML in other formal setting, including in the health and legal sectors As a part of the equity efforts, the MTBBE made special efforts to have good participation of girl students The programme achieved well this objective since girls came to represent a small See recommendation No in and multiple other MTBBE is an important tool for addressing gender and equity issues in Viet Nam (more girls in schools, more girls finishing primary school, better participation of girls and so on) UNICEF should use the results of this programme to show that by using EM languages in other arenas (e.g child protection, health and legal sectors) more women and EM people in general could attend/participate/benefit more from those services, which is a way of social integration and equity 181 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Findings Conclusions Recommendations majority of the total enrolment The evidence indicates that girls’ academic performance was as good as that of boys 182 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Annex 11 – Questionnaires 9.1 Key Informant Interview Questionnaires Four versions of the key informant questionnaire were required for: KIICM Key Informant Interviews with MOET and other central level managers KIIL Key Informant Interviews with DOET and other provincial and local authorities KIISP Key Informant Interviews with school principals KIIU Key Informant Interviews with UNICEF management Relevance To what extent is the MTBBE relevant and responsive at the national level? (1.1.2.1) KIICM KIIL KIISP KIICM To what extent is the MTBBE relevant and responsive at the sub-national level? (1.1.2.2) KIIL To what extent is the MTBBE relevant to national policies? (1.3.4) To what extent is the MTBBE relevant to Institutions in the national education system? (1.3.5) To what extent is the MTBBE relevant to MET’s Educational Development Strategy 2011-2020? (1.3.6) What role has MTBBE played in terms of improving educational policy with respect to the ethnic minorities of the country? (1.4) To what extent is the MTBBE approach relevant to UNICEF’s renewed focus on equity? (1.5) To what extent would you support or not the continuation and expansion of the MTBBE? (1.6) KIICM KIICM KIICM KIICM KIIL KIIU KIICM KIIL KIISP To what extent you know and understand the MTBBE? (1.1.1) Effectiveness To what extent is the MTBBE an effective approach to master the Mother Tongue? (2.1.1.1) To what extent is the MTBBE an effective approach to master the Vietnamese? (2.1.1.2) To what extent is the MTBBE an effective approach to learn Mathematics? (2.1.2) To what extent is the MTBBE an effective approach to improve children´s confidence? (2.1.4) To what extent is the MTBBE an effective approach to improve children´s communication skills? (2.1.5) To what extent is the MTBBE an effective approach to develop or improve other academic and non-academic skills? Please specify (2.1.3) (2.1.6) In what ways has the MTBBE approach been useful to improve your capacity as an education manager? (2.2.2) 183 KIICM KIISP KIIL KIICM KIISP KIIL KIICM KIISP KIIL KIICM KIISP KIIL KIICM KIISP KIIL KIICM KIISP KIIL KIICM KIISP KIIL Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Which were the main factors or criteria which you [Central/local/school authority] considered when designing the MTBBE? (2.3.1) (2.3.2) Which were the main factors or criteria which you considered when implementing the MTBBE? (2.3.3) To what extent have the MTBBE materials in the target languages [for teaching, learning and advocacy] been appropriate for their respective purposes? (2.3.4) To what extent has the pre-service teacher training met the quantitative and qualitative requirements of the MTBBE program? Is there a deficit in this regard? Are there relevant statistics? (2.3.5) To what extent has in-service teacher training met the quantitative and qualitative requirements of the MTBBE program? Is there a deficit in this regard? Are there relevant statistics? (2.3.6) To what extent have children´s learning outcomes been assessed? At what levels? What assessment types have been applied? (2.3.7) Regarding the statistical evidence you know, to what extent is the MTBBE program achieving children´s learning outcomes? (i.e good command of mathematics, mother tongue and Vietnamese languages: listening comprehension, speaking, vocabulary) (2.4A.1)? To what extent is this statistic evidence good enough to conclude that the MTBBE program is really an appropriate approach to fill the learning gap that affects ethnic minority students? (2.4A.1.1) KIICM KIISP KIIU KIICM KIISP KIIU KIICM KIIL KIIU KIICM KIIL KIIU KIISP KIICM KIIL KIIU KIISP KIICM KIIL KIIU KIISP KIICM KIIL KIISP KIIU KIICM KIIL KIISP KIIU Comparing MTBBE to non-MTBBE students, what differences you perceive in terms of: - attitudes toward learning? (2.4A.2) KIISP - self-confidence? (2.4A.3) KIISP - methods of learning? (2.4A.4) KIISP - collaboration with school activities? (2.4A.5) KIISP - communication and collaboration with teachers (2.4A.6) KIISP Comparing the MTBBE to non-MTBBE teachers, what differences you perceive in terms of: - professional capacity (2.4B.2) KIISP - approaches and attitude towards learning? (2.4B.5) KIISP - self-confidence? (2.4B.4) KIISP - in-class teaching and organizing skills to facilitate learning? (4B.1) KIISP 184 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT - collaboration with school affairs? (2.4B.6) KIISP - approaches towards other teachers? (2.4B.7) KIISP - commitment toward community´s needs? (2.4B.10??) KIISP To what extent has the MTBBE approach had some positive or negative effect: - on school management? (2.4B.8) KIISP - on board members? (2.4B.9) KIISP - on girl student and gender issues at school levels? (2.4A.7) KIISP - on girl student and gender issues at community levels? (2.4A.8) KIISP - on parents’ and community participation in school activities? (2.4C.1) parents (2.4C.5) community KIISP - on parents’ participation in learning and extra-curricular activities? (2.4C.2) KIISP - on parents’ relationships with schools and community? (2.4C.3) KIISP - on parents’ acceptance and views about MTBBE? (2.4C.4) KIISP - on managers’ [all levels] support for and participation in implementation of Action Research Activities? (2.4D.1) KIISP - on managers’ acceptance of and views about the MTBBE approach and its continuity and scale-up? (2.4D.2) KIISP Efficiency What has been the monetary cost of teacher training in general and per head? (3.1.1) Does some provincial difference exist in this cost? (3.1.2) To what extent is this cost higher or lower than the training of non-MTBBE teachers? (3.1.3) What has been the monetary cost of MTBBE materials [teaching, learning, and advocacy]? (3.1.4) To what extent has this cost been over or under the expected estimates? (3.1.5) To what extent have trained teachers been appropriately allocated? [or according to needs] (3.1.6) To what extent have MTBBE materials [teaching, learning, advocacy] been appropriately distributed? (3.1.7) To what extent has this received sufficient support from higher authorities [province, district, school] to meet the demand in terms of trained and un-trained teachers? (3.1.8) To what extent has this received sufficient support from higher authorities [province, district, school] to meet the demand in terms of MTBBE materials [learning, teaching and advocacy]? (3.1.9) Sustainability To what extent is the MTBBE program prepared for continuation once the external support from UNICEF is 185 KIICM KIIL KIIU KIICM KIIL KIIU KIICM KIIL KIIU KIICM KIIL KIIU KIICM KIIL KIIU KIICM KIIL KIIU KIICM KIIL KIIU KIIL KIISP KIIL KIISP KIICM Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT withdrawn? Which are the program´s achievements that are better positioned for this change? Which are the achievements more at risk due to the possible change? (4.1) To what extent can the MTBBE approach be replicated and scaled up in areas with characteristics similar to existing project sites [e.g presence of students from one single ethnic group, Mong or J´rai or Khmer in the class?] (4.2) To what extent is leadership in place for sustainability? (4.3.1) To what extent is ownership in place for sustainability? (4.3.2) To what extent are financial resources in place for sustainability? (4.3.3) To what extent are teaching and learning materials in place for sustainability? (4.3.4) To what extent are qualified teachers in place for sustainability? (4.3.5) To what extent are local/DOET resources in place for sustainability? (4.3.6) What could or should be done differently in future replications and/or scaling up? (4.4) KIIL KIISP KIIU KIICM KIIL KIISP KIIU KIICM KIIL KIISP KIIU KIICM KIIL KIISP KIIU KIICM KIIL KIISP KIIU KIICM KIIL KIISP KIIU KIICM KIIL KIISP KIIU KIICM KIIL KIISP KIIU KIICM KIIL KIISP KIIU 9.2 Focus Group Discussion Themes Categories FGC FGL FGP FGS FGSN FGT Community, Civil Society, Academics DOET, other provincial, schools, local level managers, Board members Parents of MTBBE students MTBBE students Non-MTBBE students Teachers Relevance FGC FGL FGP What you know about the MTBBE Action Research? (1.1.1) 186 Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT FGS FGT FGSN FGL To what extent is the MTBBE relevant and responsive at the provincial/local level? (1.1.2.1) (1.1.2.2) How relevant/important is the MTBBE Program for you as parents/community leaders? (1.1.2.2) FGP FGC Effectiveness Is the MTBBE an effective approach to achieve learning outcomes? Do you like, or not, like the experience of your first school learning in your mother tongue? Why? (2.1.1.1) FGS Do you like, or not, learning Vietnamese at school? Why? / FGSN: Would like to be enrolled in the MTBBE program? Why (2.1.1.2) Do you like Mathematics? Is Mathematics easy or difficult to learn? Why? Are you really learning Mathematics at school? (2.1.2) Comparing non-MTBBE students/MTBBE students, how secure you feel about your progress in learning Mathematics? (2.1.4) How easy or difficult has it been for you to learn at school in the Vietnamese language? (2.1.5) Do you like, or not, to speak Vietnamese? Why? What problems, if any, you have to talk with Vietnamese speakers? (2.1.5) To what extent is the MTBBE an effective approach to learn? How much progress you observe in children´s learning? (Mother tongue, Vietnamese, Mathematics) (2.1.1.1) (2.1.1.2) (2.1.2) Do you observe any other areas in which children are showing progress? Please explain (2.1.3) (2.1.6) Do you perceive some learning improvement in other areas? Which areas? Please explain (2.1.3) (2.1.6) To what extent has the MTBBE program helped teachers to improve their teaching capacity? (2.2.1) To what extent has the MTBBE program helped education managers to improve their administrative capacities? (2.2.2) Which were the main factors or criteria which the local/school authority considered when approaching and designing the MTBBE? (2.3.1) (2.3.2) Which were the main factors or criteria which local/school authorities considered when implementing the MTBBE? (2.3.3) What positive or negative impacts did the MTBBE approach have? Comparing MTBBE to non-MTBBE students, what differences you perceive in terms of school achievements [grades]? What supporting evidence you have on this matter? (2.4A.1) To what extent is the MTBBE approach improving, or not, students´ attitudes toward learning? In what way is this effect better, the same, or worse than the one in the case of the non-MTBBE program? (2.4A.2) To what extent are the issues you learn at school useful for your life? Up to what grade or level [primary, secondary, etc.] would you like to attend school? Why? (2.4A.2) To what extent is the MTBBE approach improving, or not, students´ self-confidence? In what way is this effect better, the same, or worse than the one in the case of non-MTBBE program? (2.4A.3) How secure/self-confident you feel in terms of how much and how well are you learning Vietnamese at school? Please explain (2.4A.3) How secure/self-confident you feel in terms of how much and how well are you learning Mathematics and other matters at school? Please explain (2.4A.3) To what extent is the MTBBE approach improving, or not, students´ capacity for self-learning? In what way is this effect better, the same, or worse than the one in the case of non-MTBBE program? (2.4A.4) To what extent have you received some instructions or training on self-learning techniques from your teachers? 187 FGS FGSN FGS FGSN FGS FGSN FGSN FGS FGSN FGP FGL FGT FGP FGL FGT FGS FGT FGL FGL FGT FGL FGT FGL FGT FGL FGT FGC FGL FGP FGT FGS FGSN FGC FGL FGP FGT FGS FGSN FGS FGSN FGC FGL FGP FGT FGS Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT Please mention some examples (2.4A.4) To what extent is the MTBBE approach improving, or not, students´ attitude to collaborate with school activities? In what way is this effect better, the same or worse than the one in the case of non-MTBBE program? (2.4A.5) To what extent you like to collaborate with school activities? Please provide some examples (2.4A.5) To what extent is the MTBBE approach improving, or not, students´ communication and collaboration with teachers? In what way is this effect better, the same, or worse than the one in the case of non-MTBBE program? (2.4A.6) To what extent are you able to communicate with your teachers? (2.4A.6) To what extent is the MTBBE approach improving, or not, the treatment to both girl students and gender issues at school level? (2.4A.7) How frequently your teachers organize talks about girls´ rights to education and respect from others? Please explain what you talk about (2.4A.7) To what extent is the MTBBE approach improving, or not, the treatment to both girl students and gender issues at community level? (2.4A.8) To what extent is the MTBBE approach improving, or not, teachers’ in-class teaching and organizing skills in facilitating MTBBE learning? How is this effect better, the same, or worse that in the case of non-MTBBE teachers? To what extent is the MTBBE approach improving, or not, MTBBE teachers’ professional capacity? (2.4B.2) What positive or negative impacts did the MTBBE approach have on teachers’ attitude? (2.4B.3) What positive or negative impacts did the MTBBE approach have on teachers’ self-confidence? (2.4B.4) What positive or negative impacts did the MTBBE approach have on teachers’ approaches towards learning? (2.4B.5) What positive or negative impacts did the MTBBE approach have on teachers’ approaches towards schools? (2.4B.6) What positive or negative impacts did the MTBBE approach have on teachers’ approaches towards other teachers? (2.4B.7) To what extent is the MTBBE approach improving, or not, MTBBE school management? (2.4D.8) To what extent is the MTBBE approach improving, or not, MTBBE board members’ administrative capacities? (2.4D.9) To what extent is the MTBBE approach improving, or not, parents’ participation in school activities? (2.4C.1) To what extent is the MTBBE approach improving, or not, parents’ participation in learning and extra-curricular activities? (2.4C.2) 188 FGSN FGC FGL FGP FGT FGS FGSN FGC FGL FGP FGT FGS FGSN FGC FGL FGP FGT FGS FGSN FGC FGL FGP FGT FGC FGL FGP FGT FGC FGL FGT FGC FGL FGT FGC FGL FGT FGC FGL FGT FGC FGL FGT FGC FGL FGT FGC FGL FGT FGC FGL FGP FGT FGC FGL Evaluation of UNICEF-Supported MOET’s Initiative of Mother-Tongue Based Bilingual Education in Viet Nam, 2006 – 2014 FINAL REPORT To what extent is the MTBBE approach improving, or not, parents’ relationships with schools and community? (2.4C.3) To what extent is the MTBBE approach improving, or not, parents’ acceptance and views about MTBBE? (2.4C.4) To what extent is the MTBBE approach improving, or not, community’ participation in school activities? (2.4C.5) To what extent is the MTBBE approach improving, or not, community’s participation in learning and extracurricular activities? (2.4C.6) To what extent is the MTBBE approach improving, or not, community’s relationships with schools and parents? (2.4C.7) To what extent is the MTBBE approach improving, or not, community’s acceptance and views about MTBBE? (2.4C.8) To what extent is the MTBBE approach improving, or not, managers’ [all levels] support for and participation in implementation of Action Research Activities? (2.4D.1) To what extent is the MTBBE approach improving, or not, managers’ acceptance of and views about the MTBBE approach and its continuity and scale-up? (2.4D.2) Efficiency To what extent has this district/school received enough MTBBE materials [teaching, learning, and advocacy]? (3.9) To what extent has this district/school received sufficient support from higher authorities to meet the demand in terms of [trained and un-trained] teachers? (3.8) To what extent has this district, school received sufficient support from higher authorities to meet the demand in terms of MTBBE materials [learning, teaching and advocacy]? (3.9) Sustainability What could or should be done differently in future replications and/or scaling up? (4.4) 189 FGP FGT FGC FGL FGP FGT FGC FGL FGP FGT FGC FGL FGP FGT FGC FGL FGP FGT FGC FGL FGP FGT FGC FGL FGP FGT FGC FGL FGT FGC FGL FGT FGL FGT FGL FGT FGL FGT FGC FGL FGP FGT