Monitoring and evaluation system and project success

MONITORING AND EVALUATION SYSTEM AND PROJECT SUCCESS A CASE STUDY OF UNFPA RWANDA Lucie MASUDI MBA/0320/12 A Research Project Submitted in Partial Fulfillment for the Award of a Degree in Master of Business Administration (Project Management Option) of Mount Kenya University APRIL, 2015 DECLARATION This research study is my original work and has been presented to any other institution No part of this research should be reproduced without the author’s consent or that of Mt Kenya University Student name: MASUDI YOHARI LUCIE Sign……………………………………Date…………………… Declaration by the supervisor(s) This research has been submitted with my approval as the Mt Kenya University Supervisor Name: Dr Alfred OTARA Sign……………………………… Date…………………… i DEDICATION To my Lord Jesus Christ, my husband, children, parents, brothers, sisters and friends for their moral and financial support during the tiring moment of the study ii ACKNOWLEDGEMENTS First of all, I give glory to the almighty God who redeems my life from pit and crown me with steadfast, grace/mercy and make all things possible I am deeply indebted to my supervisor, Dr Alfred OTARA for his intellectual guidance and research skills that enable me to finish my research work Thanks a lot Special thanks go to Mount Kenya University, Kigali Campus and all their lectures for equipping me with the knowledge that enable me to succeed this thesis My sincere gratitude and appreciation go to UNFPA Rwanda which allowed me working with their staff during research Finally I owe my deepest sense of gratitude to my fellow classmates for their assistance and encouragement during the two years of studies My heartfelt thanks go to my mother, all my sisters and brothers, more especially my husband for his encouragement and assistance and my children for their patience during this endeavor Thanks a lot iii ABSTRACT The purpose of this research was to assess how the monitoring and evaluation lead to project success The studylooked also at how M&E system was being implemented and the research findings showed more light on how M&E relationship with project goals achievement, especially at UNFPA Rwanda The study used a descriptive survey design where quantitative and qualitative approaches to data collection were used based on a target sample population from UNFPA Rwanda The findings of this study were important to UNFPA Rwanda which integrated M&E system to lead to project success The total number of population was 15 and the sample size was 15 where we had 10 respondents from staff of UNFPA and for project staff The method used to find the sample size was census method The study was carried out from July 2013 to October 2014 The content validity was assured through expert judgment, this where the copies of questionnaire were given to staff and project staff of UNFPA to determine the validity of these questionnaires The researcher critically assessed the consistency of the responses on the questionnaire in order to make a judgment on their reliability We have three variables in the conceptual framework, the dependent, independent and moderating variables Primary and secondary data were used, primary data was collected with the use of questionnaire that was closed ended and open ended questions from the field and secondary data was collected from journals, books and others researchers’ work on related topics The data collected was analyzed using excel software and the researcher drawn conclusion basing on the facts from the data collected Recommendations from the findings on the M&E system and project success that the researcher need improvement was highlighted and finally suggestions were made for future researchers to carry on and improvement on this work iv TABLES OF CONTENTS DECLARATION i DEDICATION .ii ACKNOWLEDGEMENTS iii ABSTRACT iv TABLES OF CONTENTS v LIST OF TABLES ix LIST OF FIGURES x ACRONYMNS AND ABBREVIATIONS xi OPERATIONAL DEFINITION OF KEY TERMS xii CHAPTER ONE: INTRODUCTION 1.0 Introduction 1.1 Background of the Study 1.2 Statement of the Problem 1.3 Objectives of the Study 1.3.1 General objective 1.3.2 Specific Objectives 1.4 Research Questions 1.5 Significance of the Study 1.6 Limitation of the Study 1.7 Scope of the Study 1.7.1 Concept Scope v 1.7.2 Content Scope 1.7.3 Geographical Scope 1.7.4 Time Scope 1.8 Organization of the study CHAPTER TWO: LITERATURE REVIEW 2.0 Introduction 2.1 Theoretical Literature Review 2.1.1 Tools and procedures used by UNFPA 2.1.2 Process of M&E System and Project Success 10 2.1.2.1 Monitoring 10 2.1.2.2 Evaluation 11 2.1.2.3 Definition of Monitoring and Evaluation 12 2.1.2.7 Leadership 14 2.1.2.9 Principles and Guideline for monitoring 15 2.1.2.10 Guidelines for Project Evaluation 15 2.1.2.11 Monitoring and Evaluation Plan 16 2.1.2.12 Monitoring and Evaluation in Development Programs 18 2.1.2.13 M&E Systems for Complex Organizations 19 2.1.2.14 Project Success 20 2.1.3 Relation between M&E and project success 21 2.3 Critical Review and Research Gap 25 vi 2.4 Conceptual framework 27 2.5 Summary 28 CHAPTER THREE: RESEARCH DESIGN AND METHODOLOGY 30 3.0 Introduction 30 3.1 Design of the study 30 2.Target population 31 3.3 Sample Size and Sampling Procedures 31 3.4 Data Collection Procedures/ Instruments 32 3.4.1 Questionnaire 32 3.4.2 Validity 33 3.4.3 Reliability 33 3.5 Data Processing 34 3.6 Data Analysis 34 CHAPTER FOUR: DATA ANALYSIS, PRESENTATION AND INTERPRETATION35 4.0 Introduction 35 4.1 Presentation of Findings 35 4.1.2 Project Staff 36 4.2 Suggestions of project staff and UNFPA staff to improve the M&E system of UNFPA Rwanda 43 4.3 Summary of Data Analysis 44 CHAPTER FIVE: SUMMARY OF FINDINGS, CONCLUSION AND RECOMMENDATIONS 46 vii 5.1 Introduction 46 5.2 Summary of the findings 46 5.2.1 Analyze M&E tools and procedures used by UNFPA 46 5.2.2 Assess how the process of M&E system affects the Project Success at UNFPA Rwanda 47 5.2.3 Establish the relationship between Monitoring and Evaluation system and project success at UNFPA Rwanda 48 5.3 Conclusion 48 5.4 Recommendations 50 5.5 Suggestions for further study 51 REFERENCES 52 APPENDICES 56 viii LIST OF TABLES Table 3.1: Target Population 31 Table 3.2: Sample Size 32 Table 4.3: Experience of UNFPA staff 36 Table 4.4 Distribution of the respondents by their education level 37 Table 4.5: Experience of project staff 37 Table 4.6: Financial and Narrative report 38 Table 4.7: M&E Procedures and implementing of Partners 39 Table 4.8: Resources available to complete the project on time 40 Table 4.9: M&E System information relevant to objectives 40 Table 4.10: M&E System leads to project success 41 Table 4.11:Relationship between M&E system and Project success 42 ix In addition, some M&E system of UNFPA Rwanda did not show a positive, but rather a negative relation The M&E system leads to project success when by identifying information needs to guide the project strategy, ensure effective operations and meet external reporting requirements Then decide how to gather and analyze this information and document a plan for the M&E system The process of working out how to monitor and evaluate a project inevitably raises questions about the project strategy itself, which can help improve the initial design Setting up the M&E system with a participatory approach builds stakeholders understanding about the project and starts creating a learning environment Information comes from tracking which outputs, outcomes and impacts are being achieved and checking project operations (e.g., activity completion, financial management and resource use) After information gathering and management starts, problems can be solved problems and new ideas can be obtained for improving the initial M&E plan Once information has been collected it needs to be analyzed and discussed by project stakeholders In these reflections and discussions, you will probably notice information gaps These can trigger adjustments to the M&E plan to ensure the necessary information is being collected.The results of M&E need to be communicated to the people who need to use it M&E system will be said successful when it includes reporting to funding agencies but is much broader Project progress and problems must be shared with project participants to identify solutions together Reports to funding agencies need to balance successes and mistakes and, above all, be analytical and action-oriented Some of those who are to use the information may have been involved in collecting data and/or analyzing part of it However, plan how to inform those who were not involved Results from M&E both the communication processes and the information will improve the project strategy and 49 operations that will lead to project success.Senior management is responsible for seeing to this with the support of M&E staff Sometimes improvements can be immediate According to the Analysis done of findings, it was found that M&E system at UNFPA Rwanda remains a challenge where management is not able to regularly assess the linkages and effects between activities, output and outcomes and assess the level of achievements with reference to indicator targets Measuring project success is not a simple task, Project success is measured as the ability to complete the project according to desired specifications, and within the specified budget and the promised time schedule, while keeping the customer and stakeholders happy Effective M&E system allows the success of the project Resources availability help the completion of the project on time, mid and End year evaluation enhance also the project success 5.4 Recommendations According to the research done at UNFPA Rwanda, the researcher recommends the following: UNFPA Rwanda is recommended to dispose resources that will help the completion of project on time Mid and End year evaluation must be done by Project staff to evaluate the project and communicated to stakeholders UNFPA Rwanda is recommended to have good collaboration at different manager’s level and Review the M&E tools UNFPA Rwanda is recommended to keep on more training and enhancing the capacity for staff or refresh training to implementing partners 50 UNFPA Rwanda is recommended to Review targets and baseline on a regular basis 5.5 Suggestions for further study Future researchers should carry out further research on the following areas: Project cycle Management and project success Resources Management and technical project success 51 REFERENCES Abdullah, A A Rahman, H A Harun, Z Alashwal A M & Beksin, A M (2010),“Literature mapping: A bird’s eye view on classification of factors influencing project success”, African Journal of Business Management, 4(19), pp 4174-4182 Ashrafi R & Hartman, F (2002) Project Management in the Information System and Information Technologies Industries Project Management Journal, vol 33, pp 5-15 Belassi, W and Tukel, O I (1996) “A New Framework for Determining CriticalSuccess/Failure Factors in Projects.” International Journal of Project Management 14(3): 141-151 Belout, A (1998), “Effects of human resource management on project effectiveness and success: toward a new conceptual framework”, International Journal of Project Management, 16, pp 21–26 Caldwell, R.(2002), Project Design Handbook Atlanta: CARE International Catherine,E (2006) Monitoring and Evaluation (M&E) for Development in peaceprecarious situations Chan, I.P.C Scott, D & Lam, E.W.M (2002), “Framework of success criteria for design/build projects”, Journal of Management in Engineering, 18, 120–128 Chinnanon, S (2002) A manual on Monitoring and Evaluation forAlternative Development Projects Cooke-Davis, T (2002) The ‘real’ success factors on projects International Journal of Project Management, Vol 20, pp 185-190 Cooper, R (2008) Business Research Methods New york: Mc Graw-Hill Crawford, P and Bryce, P (2003) Project Monitoring and Evaluation: A Method for Enhancing the Efficiency and Effectiveness of Aid Project Implementation International Journal of Project Management , 363-373 52 Donald K Kombo and Delno (2006) Proposal and Thesis Writing an Introduction Nairobi Estrella, Marisol and Gaventa (1997) Who counts reality? Participatory Monitoring and Evaluation: A literature Review Fortune, J (2006) Framing of the Project Critical Success Factors by a Systems Model International Journal of Project Management , 53-65 Freeman, M & Beale, P 1992, Measuring project success, Project Management Journal, Vol 23, No 1, pp 8–17 Freeman, H.E., Rossi, P.H and Wright, S.R., (2003) Evaluating Social Projects in Developing Countries Paris Gemunden H Salomo S & Krieger A (2005), “The influence of project autonomy on project success”, International Journal of Project Management, 23 (2005), pp 366373 Guijt and Abbot (1998) A working paper on participatory Monitoring of the Environment: Internatinal Institute for Environment and Development (IIED) Hunter (2009) Monitoring and Evaluation Systems Publisher Namibia Institute for democracy 1st Hyväri, I (2006) “Success Of Projects In Different Organizational Conditions.” ProjectManagement Journal 37 (4): 31-41 International Fund For Agriculture Development (2001) Managing for Impact in Rural Development: A guide for project monitoring and Evaluation Jones (2011) A guide to Monitoring and Evaluating policy influence Kerzner, H (1989) Project Management a system approach to Planning, Scheduling and Controlling (third edition ed.) Van Nostrand Reinhold Inc Kinda, O (2012) The monitoring and Evaluation System of the millenium villages project: Close Look at the Mid- term Evaluation Report Journal of Sustainable Development 53 Kusek, J (2004) Ten Steps to a Results-Based Monitoring and Evaluation System Washington D.C Lester, A (2007) Project Management: Planning and Control Elsevier, New York Lim C and Mohamed M (1999), “Criteria of project success: an exploratory reexamination”, International Journal of Project Management, 17(4), pp 243-48 Mackay, K (2007) How to build monitoring and Evaluation system to support better government Washington, DC: Worl bank Meredith (1996) Educational research: An introduction London: Longman Publishers Mugenda O.M and Mugenda., A (2003) Research Methods: Quantitative and Qualitative approaches Nairobi, Kenyan:: Lectern Publishers Nguyen & Lan (2004) A study on project success factors in large construction projects in Vietnam Construction and Architectural Management, vol 11, 404 - 413 Orodho (2009) Regional Inequalities in EDucation, Population and Poverty Patterns in Kenya Population of Kenya Journal Nairobi Orodho, A (2009) Techniques of writing Research proposal and techniques Nairobi Patton, M (2001) Qualitative research & Evaluation methods 3rd ed Thousand Oaks: Sage Publications, Inc Pinto, J.K & Slevin, D.P 1988, “Project success: definitions and measurement techniques”, Project Management Journal, Vol 19 No 3, pp.67–73 Pollack (2007) The changing Paradigms of Project Management Internatioanl Journal of Project Management , 266-274 Project Management Body of Knowledge (2004) A guide to the project Management Body of Knowledge USA Project Management Institute Rose, K (2005) Project Quality Management: why, what and How Florida 54 Rossi, Peter H., Freeman, Howard A, Lipeey and Mark W (2004) Evaluation a System Approach Simister, N (2009) Developing Monitoring and Evaluation Systems for Complex Organizations: A methodology Monitoring and Evaluation Sivagnanasothy (2007) Institutionalization of Monitoring and Evaluation System in Sri Lanka Slevin, D.P and Pinto, J.K (1986) The Project Implementation Profile: a new tool for project managers Project Management Journal, 17 (4) 57 -70 Solomon, P and Young, R (2007) Performance Based-Earned Value, John Wiley and Sons Ltd Publishing House, San Francisco Starling (2010) Monitoring and Evaluating advocacy: Lessons from Oxfam GB's climate change campaign Development in Practice volume 20, Number Thomas, G & Fernandez, W 2008, "Success in IT projects: A matter of definition?",InternationalJournal of Project Management, Vo 26, pp 733–742 United Nations Development Programme (2013) Handbook on Monitoring and Evaluation for Result United Nations Development Programme (2001) Managing forResults: Monitoring and Evaluation in UNDP United Nations Population Fund (2010) Monitoring Performance for Results at UNFPA United Nations Population Fund (2012) Monitoring and Evaluation Plan Zwikael S.G (2002) Impact of the project Manager on project management planning process Project management Journal 55 APPENDICES 56 MONITORING AND EVALUATION SYSTEM AND PROJECT SUCCESS: A CASE STUDY OF UNFPA RWANDA INFORMED CONSENT I am giving consent to be part of the research study of Ms Lucie Masudi that will focus on emotional intelligence and leadership styles I shall be assured of privacy, anonymity and confidentiality and that I will be given the option to refuse participation and right to withdraw my participation anytime I have been informed that the research is voluntary and that the result will be given to me if I ask Initial & Signature: ………………………………………………………………… Date: ……………………………………………… APPENDIX 1: Questionnaire addressed to UNFPA Staff You are cordially invited to answer by ticking to the series of questions below: Question What is your level of Education? PhD Masters Bachelor Degree Degree & M&E Professional Professional Training Question Working Experience i Less than a year ii 1-3 years iii 4-7 years iv Over years Question 3: Are you satisfied with the M&E system of UNFPA Rwanda? Agree Strongly Agree Disagree Strongly Disagree Question 4: Is Financial and Narrative report done on time? i Financially Quarterly progress reported Yes ii Narrative Quarterly progress reported Yes No No Question 5: Is Project Evaluation at UNFPA Rwanda done i Mid-year review ii End year Evaluation iii Both Question 6: Does UNFPA Rwanda has a Monitoring and Evaluation procedures well documented and knownby its staff and implementing partners Agree Strongly Agree Disagree Strongly Disagree Question 7: Are M&E tools friendly user and facilitate timely reporting on project activities? Yes No Question 8: Are resources available to complete the project on time? Yes No Question 9: Does the M&E system collects information relevant to objectives of the projects Yes No Question 10: Implementing partners policies contribute positively to the effectiveness of M&E activities Agree Strongly Agree Disagree Strongly Disagree Question 11: If disagree or strongly disagree what can be done to improve this? ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ………………………………………………………………………………… Question 12: Is there any relationship between M&E system and project success? Agree Strongly Agree Disagree Strongly Disagree Question 13: Does the M&E system of UNFPA leads to Project Success? Yes No Question 14: What would you suggest to improve the M&E System of UNFPA Rwanda? ……………………………………………………………………………………………… Thank you for your participation APPENDIX 2:Questionnaire addressed to Project staff You are cordially invited to answer by ticking to the series of questions below: Question What is your level of Education? PhD Masters Bachelor Degree Degree & M&E Professional 10 Professional Training Question Working Experience v Less than a year vi 1-3 years vii 4-7 years viii Over years Question 3: Are yousatisfied with the M&E system of UNFPA Rwanda? Agree Strongly Agree Disagree Strongly Disagree Question 4: Is Financial and Narrative report done on time? i Financially Quarterly progress reported Yes ii Narrative Quarterly progress reported Yes Question 5: Is Project Evaluation at UNFPA Rwanda done i Mid-year review ii End year Evaluation iii Both No No Question 6: Does UNFPA Rwanda has a Monitoring and Evaluation procedures well documented and knownby its staff and implementing partners Agree Strongly Agree Disagree Strongly Disagree Question 7: Are M&E tools friendly user and facilitate timely reporting on project activities? Yes No Question 8: Are resources available to complete the project on time? Yes No Question 9: Does the M&E system collects information relevant to objectives of the projects Yes No Question 10: Implementing partners policies contribute positively to the effectiveness of M&E activities Agree Strongly Agree Disagree Strongly Disagree Question 11: If disagree or strongly disagree what can be done to improve this? ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ………………………………………………………………………………… Question 12: Is there any relationship between M&E system and project success? Agree Strongly Agree Disagree Strongly Disagree Question 13: Does the M&E system of UNFPA leads to Project Success? Yes No Question 14: What would you suggest to improve the M&E System of UNFPA Rwanda? ……………………………………………………………………………………………… Thank you for your participation