LV Thạc sỹ_Improve information system supporting monitoring and evaluation of greater mekong subregion sustainable tourism development project in Viet Nam

1 ACKNOWLEDGEMENT The Final Thesis on “Improving Information System supporting Monitoring and Evaluation (M&E) of Greater Mekong Subregion Sustainable Tourism Development Project in Viet Nam” has received valuable assistance and cooperation from many people, whom I am sincerely grateful for The first person that I would like to express my gratitude is Dr … I also would like to thank the respondents from the Project Implementation Unit (PIUs) at the provincial, district and communal levels and other stakeholders: ADB’s assistants, the GoV and related agencies’ officers for their extensive knowledge and experience shared as well as their concerns on issues of M&E Information System TABLE OF CONTENTS ABBREVIATION LIST OF FIGURES EXECUTIVE SUMMARY .8 INTRODUCTION 10 RATIONALE 10 RESEARCH OBJECTIVES: 11 RESEARCH QUESTIONS: 12 RESEARCH METHODOLOGY .12 4.1 Research phases 12 4.2 Methods of data collection 13 RESEARCH SCOPE 15 THESIS STRUCTURE 15 CHAPTER 1: THEORETICAL BACKGROUND ON INFORMATION SYSTEM (IS) AND INFORMATION SYSTEM DEVELOPMENT METHODS 16 1.1 Theoretical background on Information System .16 1.1.1 Definition of Information System 16 1.1.2 Information system perspectives 16 1.1.3 Functions of IS in organization 18 1.1.4 Impacts of information systems on organizations 19 1.2 Information system development methods 20 1.3 Criteria to evaluate an Information System 23 CHAPTER 2: MONITORING AND EVALUATION (M&E) AND INFORMATION SYSTEM (IS) SUPPORTING M&E IN ODA PROJECTS 27 2.1 Overview of Monitoring & Evaluation 27 2.1.1 Definitions of M&E .27 2.1.2 Monitoring & Evaluation classification 30 2.1.3 Role of M&E in project implementation 31 2.1.4 General regulations on ODA project M&E 32 2.1.5 M&E steps 32 2.2 The supportive relationship between IS and M&E 33 2.3 Principles to build an information system supporting Monitoring & Evaluation 35 2.4 Experience in applications of Information system supporting M&E in some ODA projects in Viet Nam 35 2.4.1 Experience in application of Information system supporting M&E in Road Network Improvement Project (RNIP) funded by the World Bank 36 2.4.2 Experience in application of Information system supporting M&E in Viet Nam Land Administration Project (VLAP) funded by the World Bank 38 CHAPTER 3: CURRENT SITUATION OF INFORMATION SYSTEM SUPPORTING MONITORING &EVALUATION IN STDPVN 41 3.1 STDPVN introduction 41 3.1.1 The overview of project 41 3.1.2 Project Organizational Chart 42 3.2 Monitoring & Evaluation in the Project 43 3.2.1 M&E system 43 3.2.2 Reporting arrangement 45 3.3 Description of STDPVN’s IS supporting M&E 45 3.3.1 Overview of STPDVN’s IS supporting M&E 45 3.3.2 Description of STDPVN’s IS supporting M&E 47 3.4 Assessment on the current STDPVN’s IS supporting M&E 55 3.4.1 Level of use 55 3.4.2 Users’ satisfaction 56 3.4.3 Attitudes of users toward the current information system 64 3.4.4 The extent to which current IS meet specified objectives .65 3.4.5 Financial payoff in application of the information system .66 CHAPTER 4: RECOMMENDATIOINS FOR IMPROVING INFORMATION SYSTEM SUPPORTING MONITORING AND EVALUATION IN STDPVN 68 4.1 Gaps between key requirements and current situation of IS 68 4.2 Recommendations for improving IS supporting M&E 70 4.3 Implementation plan to improve STDPVN’s IS supporting M&E: 80 4.4 Budget for improving IS supporting M&E 83 CONCLUSION 85 BIOGRAPHIC REFERENCES 86 APPENDICES Appendix 1: Survey questionnaires Appendix 2: List of respondents Appendix 3: Sample key in-depth Interviews guide Appendix 4: List of interviewees Appendix 5: RNIP’s AMT formats Appendix Summary of VLAP – M&E - IS solution Appendix 7: Key elements of RFP to the IS supporting M&E in STDPVN Appendix 8: STDPVN’s IS output – Weighted progress ABBREVIATION ADB Asian Development Bank CTGs Community Tourism Groups CPCs Communal People Committee GoV The Government of Viet Nam LAN Local Area Network MCTO Mekong Tourism Coordination Office NPSC National Project Steering Committee ODA Official Development Assistance PC Personal Computer PCU Project Coordination Unit PIU Project Implementation Unit PPC Provincial People’ Committee PPSC Provincial Project Steering Committee SPSC Sub regional Project steering Committee STDPVN Greater Mekong Subregion Sustainable Tourism Development Project in Viet Nam MCST Ministry of Culture, Sports and Tourism IS Information System IT Information Technology M&E Monitoring & Evaluation MPI Ministry of Planning and Investment MOF Ministry of Finance MBA Master of Business Administration VLAP Viet Nam Land Administration Project WB World Bank LIST OF TABLES Table Positive and negative impacts of information systems 20 Table Comparison of Systems-Development Methods 23 Table Differences between Monitoring and Evaluation 28 Table M&E classification 31 Table STDPVN’s reporting arrangement 45 Table Frequency of use of main information sharing methods 55 Table Level of using various tools and technologies 56 Table Perception of respondents on IS contents’ quality 57 Table Respondents’ assessment on training course .62 Table 10 STDPVN’s IS SWOT analysis 66 Table 11 Gaps between key requirements and current IS situation 68 Table 12 Summary of recommendations for improving STDPVN’s IS 80 Table 13 Action plan to improve STDPVN’s IS 81 Table 14 Cost estimate for building and installing STDPVN_M&E_IS .84 LIST OF FIGURES Figure Research phases 12 Figure The MTO model - Information system perspective 17 Figure The IPO model - Functions of IS in organization 19 Figure “Operation area” of Monitoring and Evaluation 29 Figure M&E data process .30 Figure The model of supportive relationship between IS and M&E 34 Figure Project Organizational Chart .42 Figure Summary of STDPVN’ IS functions supporting PPMS 46 Figure Input sheets of STDPVN’ IS 48 Figure 10 Frequency of use of each storage method 52 Figure 11 IS flow from PIUs to PCU 54 Figure 12 Users’ satisfaction levels to IS phases and factors 57 Figure 13 Chart of information updating level of the current IS 60 Figure 14 Survey on the organized training courses 61 Figure 15 Respondents’ ideas for IS improvement 65 Figure 16 Support level of IS to M&E 65 EXECUTIVE SUMMARY Greater Mekong Subregion Sustainable Tourism Development in Viet Nam (STDPVN) is financed by Asian Development Bank (ADB) with total amount of US $11.792.000 million toward sustainable development and contribution to the country’s eco-socio development, hunger elimination, poverty reduction and tourism infrastructure development It has five components carried out in five provinces (Cao Bang, Bac Kan, Quang Binh, Quang Tri, Hue) The project’s duration is years (2009-2013) and it is in its second year Under ADB and the Government (GoV)’s regulations, the project has set up a Project Performance Monitoring System (PPMS) including Information System (IS) building However, the IS supporting PPMS has not effectively functioned causing delay in project implementation The main objective of this research is to assess the current situation of IS supporting M&E activities in STDPVN Specifically, to determine the problems and recommend solutions for improving it in reference with the experience in successful application of IS in another ODA projects To conduct the research, the author has combined the methods of desk research, observation, in-depth interview to 12 people who are almost leaders, managers, and M&E staffs in STDPVN and other projects which have applied the IS supporting M&E as well as survey 30 STPDN’s officers The followings are main findings: - Information produced by the current IS are inconsistent, inaccurate and unexhausted; - The current IS has too many sheets with complicated formats and too many items in each sheet leading to difficulties in extracting needed information; - Untimely information aggregation and slowly information updating/disseminating; - Unskillful M&E staff and unclear cut responsibilities and functions of people involved; - The project has lacked of appropriate technology: No full internal network, not standardized software used for M&E, limited storage volume and no good antivirus system The reasons for above weaknesses are - Low managers’ awareness on the role of IS in supporting M&E at all levels causing the loose coordination among units/departments in IS operation and IS has not been set up since the project inception; - Lack of appropriate investment in IT leading to irrelevant and unsustainable IS applied; - Not good M&E staffing, lack of relevant training courses, lack of mechanism to motivate M&E staff, almost M&E staff work in pluralism The followings are the author’s recommendation to improve IS supporting M&E in STDPVN: Management - Increase managers’ awareness on the importance and issues related to the IS supporting M&E; - Promote the use of IS as a management tool Technology - Refresh and strengthen hardware to serve the current IS’ s improvements; - Setting up the internal network (LAN/WAN) to all PCU, PIUs; - Selecting relevant method to develop the IS - Application Software Package to improve the technology infrastructure: large database system; Frequent and automatic storage and backup; Using firewall or some proper anti-virus program for secure Organization - Good M&E staffing at all levels: Professional M&E staff at all levels for separate tasks of M&E; Organizing relevant training courses; Creating suitable policies to motivate M&E staff; - Building a well-organized system with clear functions and responsibilities of people involved 10 INTRODUCTION RATIONALE Official Development Assistance (ODA) is one of official funds from developed countries to developing countries with aim at supporting the economic development and social welfare for these countries Over the last 15 years, Viet Nam has significantly mobilized all resources and taking advantages of this fund for socioeconomic development, poverty reduction and hunger mitigation However, due to limited management capacity, bureaucratic mechanism, and corruption, projects are often stuck with challenges, and not really effectively operated Delays in ODA project progress are the main challenge for Viet Nam to reach the target timely According to the World Bank (WB), general project progress is often slower than expected plan about years Disbursement rate of Viet Nam is also considered the lowest in the region Outstanding reason for limited management capacity in project is the lack of good operation of monitoring and evaluation (M&E) In addition to that, one of the most important factors affecting effectiveness of M&E is the support of Information System (IS) In project management, M&E plays a determined role to success of project It is highly required to consistently and effectively those activities under the Donor and the Government’s requirements, however awareness of IS in project management and the interrelation between IS and M&E is inadequate causing ineffectiveness of M&E, and making delays in project management This is also the case of Greater Mekong Subregion Sustainable Tourism Development Project (STDPVN) STDPVN is financed by Asian Development Bank (ADB) with total amount of US $12 million toward sustainable development and contribution to eco-socio development; especially focusing on hunger elimination and poverty reduction, tourism infrastructure development…It has components carried out in provinces (CIO Bang, Bac Kan, Quang Binh, Quang Tri, Hue) The project lasts in a period of years (2009-2013) and is now being in the early of second year A lot of stakeholders involved in project implementation such as National Steering 101 - Their suggestions for applying the IS to STDPVN What are the requirements for the IS supporting M&E in ODA project in term of IS organizational, management and technical aspects? Please briefly indicate your IS building process? How the IS has been functioned in term of input-processingoutput with focusing on the IS formats, content and timeliness? As your experience, what are your suggestions for applying this IS in STDPVN? M&E and IT staff Key ideas to explore: - Their feedback on IS functions: input-processing-output/storage with forcusing on formats, content and timeliness - Advantages and disadvantages in IS performance - User’s satisfaction to the IS - Ideas to improve the current IS Currently, what kinds of IS tools or softwares are used to support M&E in the project? Please list What is the most effevtive one used? In term of the IS formats, are they simple, friendly to be used? Please indicate as detailed as possibe with demonstration in the IS In term of the contents in the IS, is the collected information relevant, exhaustive and accurate? If not please give the reasons and explain In term of timeliness, could you please give your ideas about the smoothness of information flow? (Very quickly, quickly, normally, slowly, extremely slowly) Please express and explain the reasons Does the current IS automatically process the information of the entire project? Is the reports produced timely and the information is available for all levels? What are the main devices for information storage? (Probe about tools like website, big volume USB, IS software, computer, others) According to you, what is the current informationn updating level? 102 (Constantly updated, periodically updated, updated if requested, slowly updated, no updated) With the working reality in your unit, what are the methods/tools used for information sharing? (Workshops, seminars, meetings, IS application, website, email, phone, fax…) What is the most effectively used to serve information sharing? Please explain 10 Have you fully trained on M&E and IS performance? If you have, please show your evaluation on that training courses in term of the contents, the agenda, the relevance and indicate the training courses as many as possible 11 Please express your satisfication level to the IS perfomance? (Including data collection, processing, storing and sharing, reporting…) 12 Could you please give your ideas about the advantages in IS performance (qualified staff, good equipments, relevant trainings, high interests of managers, simple formats ) 13 Could you please give your ideas about the disadvantages in IS performance (lack of qualified M&E staff, lack of trainings, lack of good IS, complicated procedures…) 14 As your experience, please provide your suggestion to improve the current IS? Closing key components: - Additional comments - Next steps - Thank you Is there any more would you like to add? I’ll be analyzing the information you and others gave me That is such a very usefull information source for us Thank you for your time 103 Appendix 4: List of interviewees No Full Name Unit Position Dr Tran Van Ngoi PCU Director Truong Tuan Phuong PCU Deputy Director Nguyen Huy Quang Tri PIU Director Luong Anh Bien Bac Kan PIU Deputy Dang Minh Tue Quang Binh PIU M&E staff Don Townsend Consulting Firm M&E specialist Nguyen Van Binh Consulting Firm M&E specialist Nguyen Van Mau VLAP Coordinator 10 Tran Van Linh RNIP Deputy Director 11 Ta thi Hong Thuy PCU IT 12 Le Cuong Quang Tri PIU M&E staff 104 Appendix 5: RNIP’s AMT formats Source: RNIP’s AMT, 2008 105 Appendix Summary of VLAP – M&E - IS solution Analysis of VLAP_M&E_IS solution seen from IPO model VLAP_M&E _IS is considered as a Web-based software tools and functions of input-processing-output and aggregating project information These data and information sources will help managers to monitor and evaluate project implementation and thence they make decisions on management and necessary adjustments to reach the scheduled objectives Basically, data and information sources in IS including: (1) Annual project plans (Work plan, budget plan, training plan, procurement plan, M&E plan ect); (2) Activities implementation progress as scheduled (3) Data on outcomes/outputs M&E (4) Data/information on financial management of each PMU and the entire project via AMT reports (5) Report system to ADB and the GoV This data/information is expected to be updated by PMUs quaterly, monthly or annually The system will process, integrate and store the information in the forms of report tables Management level and users can access the system via internet to use this report source Inputs An M&E staff of VLAP commented that VLAP_M&E_IS has good formats in term of Graphic design (nice colors, relevant font, good aligning and design ECT) The formats also look simple and easy to use Source: VLAP_M&E_IS 106 Sample of VLAP_M&E_IS formats The IS not only has good formats but also the indicators are relevantly selected based on M&E plan This is one of important advantage which help the lower levels in IS use Processing Each provincial PMU has seperate account to update information The IS automatically processes the information of the entire project Therefore, the information is available for sharing and decision making Outputs The IS also ensure to produce the reports with separate information of each provincial PMU and the central PMU Source: VLAP_M&E_IS Demonstrations on IS facilities Basing on the information automatic aggregation, the reports are also automatically produced This helps to ensure the currency and frequency of data Beside the report system, the IS output are also able to provide other materials in various formats (word, pdf, ejp ect) serving various concerns of stakeholders Analysis of VLA M&E_IS seen from MTO model Management The IS has been used as a management tool because it is such a useful tool with outstanding functional features: user’s decentralization; access dairy; information of all levels by time 107 Source: VLAP_M&E_IS Demonstrations on the use of IS as a management tool Technology Internal network This system includes servers connected (LAN/WAN) in a proper system with the decentralization of system resources The internal network will consist of the host computer system supplying network service such as DNS (Domain Name Service), DHCP, confirmation and user admin Application system Application system includes applications operated with the model of client/server Generally, the applications may be the softwares run on single PC or on the network The current trend expressed that a Portal-gate built – a Web interface environment which is unified to provide the ability of easy access, data security and user’s applications The Portal-Gate is able to collect, integrate and supply information according to the particular individual demand, assisting units to find out the senior and private communication method Internet system This is a separate part from the internal network which help to connect the information with global network High risks on network security will be appear as the internal network is connected to the Internet, thus normally the network must have a Firewall to manage the Internet entry/exit Database admin system Database admin system is a host computer storing all data in the forms of records, using queries to search, add, delete or edit data tables The database is stored in the 108 server’s hard disk as normal files, except the capacity of information searching and displaying The overall information architecture is described as the following figure: The system consists of one/more web-server system, Database – Server system and a storage system This system is connected with internet environment via a Firewall in order to ensure the system safety IS IS IN IN CPMU CPMU INTERNET INTERNET ENVIROMNET ENVIROMNET DomainPortal LDAP Mail host Host Host Host com com com com PC Compute rs Applicatio Databa Database n host com se host com LAN IN PPMUs LAN in Genernal Land Administraion (iii) Basic softwares: LAN in MONRE Source: VLAP_M&E_IS.Introduction book NOTES: Security connection Internet connection The overall information architecture Basic softwares Currently, there are a lot of Operation System used such as UNIX, WINDOWS; however, MS Windows 2003 Server is preferred to the host computer and Windows XP Operation System to the server Network security and connection safeguard Normally, the following methods will help to secure the network: 109 - Use Virtual Private Networks –VPNs to set up the network security as going through the public environment - usually the Internet and expanding the network connection to the remote grass-roots (remote offices, mobile users, telecommuters extranet partners) - Using equipments to encode the transferring line as going through the external environment – usually Leased line X.25 Frame Relay Bboth two methods use encoding technology to prevent the information from violation on the large scale network IS Data safety/ Storage and backup Data/information safety is the most important prior in any IS The database on the host computer is frequently and automatically stored The system of storage and backup is implemented in several equipments and several places such as magnetic tape, hard disk…in various host computers to ensure the data safety in any bad cases All databases can be recovered as it was before breakdown Anti-Virus system - Being able to detect, prevent, protet and destroy virus on the entire network system - Capable of automatically scan and destroy virus, setting up the timetable for automatic scan to all computers in the system at the same point of time - Ability of integrating into email system to destroy virus - Capacity of files, database and Operation System scan - Ability of integrating into Proxy system which allows to detect and prevent virus as gateway virus - Ability of remote set up and assistance - Being able to comprehensively upgrade all computers at the same time ensures the highest safety and synchronousness, and avoiding an unclosed circle of virus destroying causing a problem that the system is still infected 110 IS development process A To IS software design: 1) System survey and analysis 2) System design and analysis 3) IS software program 4) IS software test 5) Making guidelines on installation, admin and system use B IS implementation on a large scale: 1) Training users and system admin 2) IS software installation 3) Set up and update baseline data/ initial data 4) Testing C System liquidation 1) Documents, materials 2) CDs for recording program, materials 3) Liquidation D Guarantee, maintenance and technical assistance 1) System guarantee (48 months since liquidation) 2) System maintenance (via specific contracts) 3) Technical assistance (via telephone, Email during project implementation time) Organization In this part of content, the project M&E staff said that they focused on the training to M&E staff and clearly cutting the responsibilities and duties of users and stakeholders with following details: - Training users and system admin - User and stakeholder classification:  Leaders: 111 - Leader of MONRE, NPSC: Exploit information at general level and ask the lower levels for providing detailed information of all activities aspects at anytime - Leader of PPCs, PPSCs and related departments: Exploit information at general level and ask the lower levels for providing detailed information under project scope - Leaders of CPMU: Exploit information at general level and ask the departments for providing detailed information under CPMU scope - Leaders of PPMU: Exploit information at general level and ask the departments for providing detailed information under PPMU scope  Chief of CPMU and PPMU’s departments: - Direct the collection, creation and updating of target data, setting up and submitting reports under assigned responsibility, including: Aggregation group, Financial group, Technical group  Exploiting information CPMU and PPMU’s IS staffs: Collecting the target data and making reports under assigned tasks  - Information collection and processing - Set up, update the target data - Set up, update reports - Exploiting information System admin: Take functions of system admin 112 Appendix 7: Key elements of RFP to the IS supporting M&E in STDPVN Supplier’s requirements: (i) Set up an IS at the CPU (ii) The supplier defines the necessary equipments with technical qualifications to the IS (host computer configuration, server, printers, LAN/WAN…) to server the IS performance (iii) Training IS operation to CPU and PIUs; (iv) Providing user’s guidance and technical assistance to PCU and PIUs within the first 12 months of IS implementation, including: installation, data entry, producing reports, repairing and maintaining the IS (v) Providing guarantee service within the first year of IS using and IS maintenance during project cycle IS’ technical requirements: 2.1 General introduction: STDPVN_M&E _IS is to manage almost information stipulated in M&E system toward satisfying three main M&E purposes: Implementation, Compliance and Impact M&E 2.2 System requirements: The IS will be based on advance programming language with normal requirements on computer system used in office 2.3 Operation System Requirements: The IS will run on Windows XP and Vista well 2.4 Database requirements: Database admin system must be able to manage big volumn database, automatic backup 2.5 IS interface: The IS has friendly interface in English and Vietnamese 2.6 IS modules: + Module of user admin:  Allowing to define and decentralize users at levels: system admin, information updating and printing  User decentralization by components: Central, provincial  Allowing to add, edit, delete users: Deactivate the deleted account 113  Storing main tasks (log) of users As log file become too large, the old tasks will automatically deleted Accounts for data entry associate with the administrative units That mean each decentralized unit has it own account for data entry but all accounts are in the same system + Module of data entry/updating:  Making, updating project plans including overall plan and specific plan of each province (consist of outcome indicators to each activities)  Making, updating and processing project implementation plan progress ( specific to each activity, each component), including outputs achieved compared with the indicators (specific to each PIU and aggregating at the PCU);  Updating the implementation results of M&E plan Almost input requirements for IS are reflected in STDPVN PPMS + Module of management information system:  Project material administration system  Display reports under the requirements reflected in STDPVN PPMS as follows: - Report to ADB :  Project progress report on the quarter basic (to each PIU and aggregated in PCU)  Annual progress report  Midterm report  Completion report  Audit reports  Resettlement report  Environmental report  Other reports: Statistics reports on delay activities in each PMU 114 - Reports under Decision No 803//2007/QĐ-BKH dated 30/07/2007 by the Ministry of Investment and Planning, including: Quaterly reports:  Form 2: Quarter progress report  Form Contracts details  Form ODA fund progress report  Form Impress account report  Form Counterpart progress report  Form 10 Bidding process  Form 11 Bidding results  Form 19 Logical frame  Form 20 Main activities outcomes Annual reports: Annex Excel format Completion report: Annex Excel format + Module of Database connection and aggregation:  The Database is concentratively managed at the central level (PCU)  PIUs take manipulation with the software via online method 115 Appendix 8: STDPVN’s IS output – Weighted progress Source: STDPVN_M&E_IS ... Information System IT Information Technology M&E Monitoring & Evaluation MPI Ministry of Planning and Investment MOF Ministry of Finance MBA Master of Business Administration VLAP Viet Nam Land... an information system supporting Monitoring & Evaluation 35 2.4 Experience in applications of Information system supporting M&E in some ODA projects in Viet Nam 35 2.4.1 Experience in. .. objectives .65 3.4.5 Financial payoff in application of the information system .66 CHAPTER 4: RECOMMENDATIOINS FOR IMPROVING INFORMATION SYSTEM SUPPORTING MONITORING AND EVALUATION IN STDPVN 68 4.1