Risk management in software project scheduling using bayesian networks

MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Nguyen Ngoc Tuan RISK MANAGEMENT IN SOFTWARE PROJECT SCHEDULING USING BAYESIAN NETWORKS PhD DISSERTATION ON SOFTWARE ENGINEERING Hanoi – 2021 MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Nguyen Ngoc Tuan RISK MANAGEMENT IN SOFTWARE PROJECT SCHEDULING USING BAYESIAN NETWORKS Major: Software Engineering Code No.: 9480103 PhD DISSERTATION ON SOFTWARE ENGINEERING SUPERVISORS: Assoc Prof Dr Huynh Quyet Thang Dr Vu Thi Huong Giang Hanoi – 2021 DECLARATION: I certify that this thesis and the work presented in it are products of my own work, and that any ideas or quotations from other people work published or otherwise, are fully acknowledged in accordance with the standard referencing practices of the discipline This thesis has not been submitted for any degree or other purposes Hanoi, April 29, 2021 PhD STUDENT Nguyễn Ngọc Tuấn ON BEHALF OF SUPERVISORS Assoc Prof Dr Huỳnh Quyết Thắng Acknowledgements First of all, I would like to express my sincere gratitude to my first supervisor Assoc Prof Dr Huynh Quyet Thang for his invaluable guidance and support throughout my research Professor Thang has supported me all the way, all the time It is his patience that keeps me always committed to doing this research and reaching the end of PhD student period I am also very grateful to my second supervisor Dr Vu Thi Huong Giang whose bright hints and expertise has been always helpful to me My special thanks go to Ms Vo Thi Huong, Ms Bui Thi Quynh Nga, Mr Tran Trung Hieu, Mr Tran The Anh, Mr Tran Bao Ngoc and Mr Cao Manh Quyen, who were master and bachelor students at School of ICT, Hanoi University of Science and Technology and helped me with building the tools as well as testing our models I am also indebted to Dr Nguyen Thanh Nam (former CEO of FPT and former President of FSOFT), Mr Luu Quoc Tuan (Tinh Van Outsourcing Jsc.), Mr Ngo Quang Vinh (Evizi), Mr Nguyen Huy Binh (FIS) who provide helpful real software project data and valuable expertise judgments on the data Finally, my greatest appreciation is to my family, especially to my wife Tran Thi Bich Ngoc and to my son Nguyen Minh Huy Without their love, patience and sacrifice, this achievement would never be possible Summary Software project management is an art and science of planning and leading software projects In software industry, project managers mostly rely on their experience and skills to manage their projects and lack of scientific tools to support them Risk management is a crucial part of software project management that helps prevent software disasters In this research, risks are defined as uncertain events or conditions that, if they occur, they would have a bad impact on one or more software project outcomes (cost, time, quality) Identifying and dealing with risks or uncertainty in early phases of software development life cycle would lessen longterm cost and enhance the chance of the project success The most important part of risk management is risk analysis which assesses the risks and their impact to the outputs of the software project To overcome subjective assessment based on development team’s experience, the team needs a quantitative risk analysis method Software project scheduling is one part of software project planning Since in practice, most software projects are over-budget and behind schedule, software project scheduling needs to be taken into careful consideration We come up with the following questions:  How to schedule software projects better?  How to better manage risks in software projects?  How to quantitatively analyse risks? Some researchers say that Bayesian Networks can be used to quantify uncertain factors in (general) project scheduling and improve project risk assessment and analysis Our research is aimed at taking those advantages of Bayesian Networks into software project scheduling by addressing common software project features The research provides answers to the above questions with probabilistic approaches and tools to assess the impacts of risk factors on software project scheduling; proposing list of common risk factors and Bayesian Network model of these risk factors; and proposing advanced scheduling methods based on incorporating Bayesian Networks into popular scheduling techniques such as CPM, PERT or agile iteration scheduling etc Bayesian Networks help quantify the factors, and hence help better manage them as well as enhancing the predictability of things happen in the project This research first takes a literature review on (general) project planning issues, project scheduling techniques, project scheduling tools, uncertainty and risk characteristics in software projects, risk management processes, project risk analysis in order to apply state-of-the-art techniques into software projects (Chapter 1) After that, Bayesian Networks are applied in building and experimenting risk factors in software project scheduling BRI (Bayes Risk-Impact) algorithm is proposed to assess risk factors’ impact on software scheduling (Section 2.1) The first set of risk factors with risk factors are examined using a probabilistic ownbuilt tool CKDY to analyse risks in software project scheduling (Section 2.2) The research proposes an advanced algorithm for agile iteration scheduling using Bayesian Networks The advantages of this method are providing a schedule and the probability of finishing agile iteration on time (Section 3.1) In addition, the author goes further with a more refined list of 19 risk factors in software scheduling and uses them in software scheduling methods The research also incorporates Bayesian Network with CPM and PERT scheduling techniques in traditional software projects together with the Bayesian Networks of common risk factors (Section 3.2 and Section 3.3) The list of 19 risk factors in agile software development is also examined in agile iteration scheduling (Section 3.4) The experimental results show that our models are reliable and our approaches have practical implications, i.e we can take advantage of Bayesian Networks in modelling and quantifying risks/uncertainty in software projects How to read this report? The author highly recommends that you read this report from beginning to the end However, if at any point you want to look at specific important pieces of information, the following guide could be helpful:  To get the motivation, the overview of related work, the objectives, the scope, the hypothesis and methodology of this research, please go to the Introduction section  To get an overview of software project scheduling and risk management in software project scheduling, please go to Sections 1.1, 1.2 and 1.3  To get an overview of Bayesian Networks, please go to Section 1.4  To get details on main contributions and key findings of the research, please read Chapter and Chapter  To get information on common risk factors in software project scheduling, you can have a look at Section 2.3  The Chapter is about building tools and doing experiments on applying Bayesian Networks into risk management in software project planning (Section 2.1) and some key risk factors (Section 2.2)  The Chapter is about incorporating Bayesian Networks and common risk factors into software project scheduling techniques such as CPM (Section 3.2), PERT (Section 3.3), Agile software development scheduling (Section 3.4)  To get to know the conclusions, the limitations as well as the further research of the study in this PhD thesis, please read the Conclusion section Content Acknowledgements Summary………… How to read this report? List of symbols and abbreviations 10 List of tables 12 List of figures 13 Introduction 15 Motivation 15 Related work 18 Research scope 20 Research objectives 21 Scientific and realistic meaning 21 Research hypothesis and methodology 21 Expected results 22 Structure of the thesis 22 Chapter Overview of software project scheduling and risk management 24 1.1 Software project management and software project scheduling 24 1.1.1 Software project management 24 1.1.2 Software project scheduling 26 1.2 Software project scheduling methods and techniques 27 1.2.1 Overview 27 1.2.2 Traditional scheduling methods and techniques 27 1.2.3 Agile software project scheduling 32 1.3 Risk management in software project scheduling 34 1.3.1 Overview of project risk management 34 1.3.2 Project risk analysis 36 1.3.3 Unknown risks 37 1.3.4 Risk aspects in software project scheduling 37 1.4 Bayesian Networks 38 1.4.1 Bayesian approach vs classical approach 38 1.4.2 Probabilistic approach using Bayesian Networks 39 1.4.3 Bayesian Inference 41 1.4.4 Bayesian Networks and project risk management 42 1.5 Chapter remarks 44 Chapter Common risk factors and experiments on Bayesian Networks and software project scheduling 46 2.1 Application of Bayesian Networks into schedule risk management in software project 46 2.1.1 Common risk factors in software project management 47 2.1.2 Bayesian Networks of risk factors 48 2.1.3 Risk impact calculation 54 2.1.4 Bayesian Risk Impact algorithm 57 2.1.5 Tool and experiments 58 2.1.6 Conclusion and contribution 63 2.2 Experiments on common risk factors 64 2.2.1 Discovering the top ranked risk factors 64 2.2.2 Tool CKDY 68 2.2.3 Experiments and analysis 70 2.2.4 Conclusion and contribution 74 2.3 Proposed common risk factors in software project scheduling 75 2.3.1 The 19 common risk factors in traditional software project 75 2.3.2 The 19 common risk factors in agile software project 77 2.3.3 Conclusion and contribution 79 2.4 Chapter remarks 79 Chapter Incorporation of Bayesian Networks into software project scheduling techniques 81 3.1 Applying Bayesian Networks into specific software project development 81 3.1.1 Introduction 81 3.1.2 Optimized Agile iteration scheduling 82 3.1.3 Optimization model for Agile software iteration 83 3.1.4 Tool and experimental results 88 3.1.5 Conclusion and contribution 92 3.2 Incorporation of Bayesian Networks into CPM 92 3.2.1 The RBCPM Model 93 3.2.2 The RBCPM Method 96 3.2.3 Tool and experimental results 97 3.2.4 Conclusion and contribution 101 3.3 Incorporation of Bayesian Networks into PERT 102 3.3.1 Proposed model 102 3.3.2 Tool development and data collection 106 3.3.3 Experimental results and analysis 110 3.3.4 Conclusion and contribution 112 3.4 Incorporation of Bayesian Networks into Agile software development scheduling 112 3.4.1 Incorporation of risk model 113 3.4.2 Tool and experimental results 113 3.4.3 Conclusion and contribution 115 3.5 Chapter remarks 116 Conclusion 117 What has been done 117 [63] Patterson F.D and Neailey K (2002), “A Risk Register Database System to Aid the Management of Project Risk”, International Journal of Project Management, 20(5), pp 365-374 [64] Hillson D (1999), “Developing Effective Risk Responses”, Proceedings of the 30th Annual Project Management Institute Seminars and Symposium, Philadelphia USA [65] Al-Bahar J and Crandall K.C (1990), “Systematic Risk Management Approach for Construction Projects”, Journal of Construction Engineering and Management, 116(3), pp 533-546 [66] UK Ministry of Defence (1991), “Risk Management in Defence Procurement”, Ministry of Defence, Whitehall, London [67] del Caano A and de la Cruz M.P (2002), “Integrated Methodology for Project Risk Management”, Journal of Construction Engineering and Management, 128(6), pp 473-485 [68] Wideman R.M (1992), “Project and Program Risk Management”, Newtown Square, PA, USA, Project Management Institute [69] BSI (1999), “Guide to Project Management”, London, British Standard [70] Rosenberg L.H et al (1999), “Continuous Risk Management at NASA”, NASA, available online [71] Defense Systems Management College (2000), “Risk Management Guide for Dod Acquisition”, USA, Department of Defense [72] US Department of Transportation (2000), “Project Management in the Department of Transportation” [73] Baber R.B (2005), “Understanding Internally Generated Risks in Projects”, International Journal of Project Management, 23(8): 584-590 [74] Goldstein M (2006), “Subjective Bayesian analysis: Principle and practice”, Bayesian Analysis, 1(3), pp 403-420 [75] Joshua H., Martin N., Norman E F (2020), “Product risk assessment: a Bayesian network approach”, Proceedings of the 2020 ACM Southeast Conference, April 2020, pp 34–38 125 [76] McCabe B (1998), “Belief Networks in Construction Simulation”, Proceedings of the 30th Conference on Winter simulation, IEEE Computer Society Press [77] Nasir D., McCabe B et al (2003), "Evaluating Risk in Construction-Schedule Model (Eric-S): Construction Schedule Risk Model", Journal of Construction Engineering & Management, 129(5), pp 518-827 [78] Houston D (2000), “Survey on potential effects of major development risk factors”, Arizona State University Research Project [79] Cortellessa V et al (2005), “Model-Based Performance Risk Analysis”, IEEE Transactions on Software Engineering, 31(1): 3–20 [80] Islam S (2012), “Software Development Risk Management Model - A GoalDriven Approach”, Technical Report [81] Alberts C.J and Dorofee A.J (2010), “Risk management framework”, SEI Technical Report [82] NASA Policy Detective (2005), NPD 2820.1A NASA Software Policies [83] IEEE Computer Society (2001), “IEEE Standard for Software Life Cycle Processes - Risk Management” [84] Tore D and Torgeir D (2008), “Empirical studies of agile software development: A systematic review”, Information and Software Technology 50.9-10, pp 833–859 [85] Augustine S (2005), “Managing Agile Projects”, Upper Saddle River, NJ, USA: Prentice Hall PTR [86] Tsun Chow and Dac-Buu Cao (2008), “A survey study of critical success factors in agile software projects”, Journal of System and Software 81(6), pp 961– 971 [87] Schwaber K and Beedle M (2001), “Agile Software Development with Scrum” [88] Martin R.C (2002), “Agile Software Development, Principles, Patterns and Practices” [89] Miller A (2008), “Distributed Agile Development at Microsoft patterns and practices” 126 [90] Agile Alliance, “Manifesto for agile software development”, [Online] Retrieved 14 May 2017 Available at: http://agilemanifesto.org [91] Nguyen N.T and Huynh Q.T (2013), “Combining Maturity and Agility – Lessons Learnt From A Case Study”, Proceedings of the 4th International Symposium on ICT SoICT 2013, pp 267-274 [92] VersionOne, 7th Annual Survey (2013), “The State of Agile Development”, Full Data Report [93] Fox T L and Spence J W (1998), “Tools of the trade: a survey of project management tools”, Project Management Journal, 29, pp 20-28 [94] Pollack-Johnson B (1998), “Project management software usage patterns and suggested research directions for future development”, Project Management Journal, 29, pp 19-29 127 Index A agile iteration scheduling 3, 4, 81, 92 agile software development 4, 22, 27, 32, 34, 36, 91, 92, 117, 126, 127 R Bayesian Networks 3, 4, 5, 18, 22, 23, 38, 39, 42, 46, 48, 81, 92, 102, 112, 116, 117, 118, 119, 121, 123 BNs18, 19, 21, 22, 40, 42, 43, 44, 45, 46, 48, 54, 64, 66, 68, 73, 74, 78, 79, 80, 81, 82, 85, 86, 92, 93, 97, 98, 100, 101, 102, 105, 107, 112, 113, 115, 116 BRI 4, 10, 23, 46, 57, 63, 79, 117 risk analysis 3, 4, 16, 17, 30, 36, 37, 42, 43, 123 risk factors 3, 4, 5, 18, 19, 20, 22, 23, 31, 45, 46, 47, 48, 50, 51, 54, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 70, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 92, 93, 94, 96, 100, 101, 102, 103, 104, 105, 112, 116, 117, 118, 126 Risk management 3, 34, 119, 126 risks 3, 4, 15, 16, 17, 18, 19, 21, 22, 23, 34, 35, 36, 37, 38, 43, 44, 46, 54, 57, 58, 61, 62, 63, 64, 68, 69, 74, 75, 76, 77, 79, 81, 82, 85, 92, 93, 95, 97, 98, 101, 102, 103, 104, 105, 112, 115, 117, 118 C S CKDY 4, 19, 23, 46, 64, 68, 73, 74, 75, 79, 117 CPM 3, 4, 5, 10, 17, 18, 19, 21, 22, 23, 27, 28, 29, 30, 45, 64, 75, 92, 93, 96, 97, 98, 101, 102, 111, 112, 116, 117 Scrum 32, 34, 114, 124, 126 Software project management 3, 24, 25 Software project scheduling 3, 26, 27, 82 B T M makespan 89 P PERT 3, 4, 5, 10, 11, 17, 18, 20, 21, 22, 23, 27, 29, 30, 45, 64, 75, 101, 102, 103, 104, 106, 110, 112, 116, 117, 121 project management 3, 5, 16, 17, 18, 22, 24, 25, 27, 29, 31, 34, 37, 44, 46, 47, 48, 60, 61, 62, 66, 123, 127 The RBCPM Method 96 The RBCPM Model 93 the tool BAIS 88, 90, 113, 114, 116 The tool RBCPM 97 The tool RBPERT 106 U uncertainty 3, 4, 17, 18, 19, 21, 22, 27, 29, 30, 31, 32, 34, 35, 36, 37, 43, 44, 46, 64, 81, 82, 85, 92, 98, 101, 102, 112, 116, 118, 121 Q quantitative risk analysis 3, 17, 36 128 Appendix Sub Bayesian Networks of the 24 risk factors This appendix demonstrates in details the sub BNs associated 24 risk factors which was examined in Section 2.1.2 staff_experience_shortage +untrained_staff +staff_training +project_schedule Figure A sub BN for the risk factor “Staff experience shortage” +decision_make_delay reliance_on_a_few_person +productivity +low_moral Figure A sub BN for the risk factor “Reliance on few key person” 129 Figure A sub BN for the risk factor “Schedule pressure” Figure A sub BN for the risk factor “Low productivity” 130 lack_of_staff_commitment ++productivity +loss_of_staff +staff_experience_shortage Figure A sub BN for the risk factor “Lack of staff commitment” +defect_rate ++lack_of_client_input +lack_of_staff_commitment lack_of_client_support ++missed_requirement +creeping_user_requirements Figure A sub BN for the risk factor “Lack of client support” 131 ++decision_making_delay ++low_moral +rework lack_of_contact_person_ competence +++schedule_pressure +communication_overhead ++missed requirement +creeping_user_requirements Figure A sub BN for the risk factor “Lack of contact person competence” lack_of_quantitative_historical_data ++inaccuring_cost_estimating Figure A sub BN for the risk factor “Lack of quantitative historical data” 132 inaccurate_cost_estimating +staff_experience_ shortage ++schedule_pressure Figure A sub BN for the risk factor “Inaccurate cost estimating” ++large_and_complex_project large_and_complex_external_interface Figure 10 A sub BN for the risk factor “Large and complex external interface” +communication_overhead +++defect_rate large_and_complex_project Figure 11 A sub BN for the risk factor “Large and complex project” 133 ++project_size unnecessary_features Figure 12 A sub BN for the risk factor “Unnecessary features” ++rework ++project_size creeping_user_requirement Figure 13 A sub BN for the risk factor “Creeping user requirement” +defect_rate ++schedule_delay unreliable_subproject_delivery Figure 14 A sub BN for the risk factor “Unreliable subproject delivery” 134 Figure 15 A sub BN for the risk factor “Incapable project management” +staff_experience_ shortage +low_moral lack_of_senior_managem ent_commitment ++project_schedule +schedule_pressure Figure 16 A sub BN for the risk factor “Lack of senior management commitment” 135 lack_of_organization_maturity ++incurate_cost_estimating ++inadequate_process_method ++schedule_pressure Figure 17 A sub BN for the risk factor “Lack of organization maturity” +rework +schedule_pressure ++inadequate_process_ method +productivity immature_technology +defect_rate Figure 18 A sub BN for risk factor “Immature technology” 136 +rework +productivity ++defect_rate inadequate_configuration_ control +manual_efforts +project_schedule Figure 19 A sub BN for the risk factor “Inadequate configuration control” +defect_rate +productivity ++low_moral excessive_paper_work Figure 20 A sub BN for the risk factor “Excessive paperwork” 137 +schedule_pressure ++inaccurate_reporting ++inaccurate_cost_ estimating inaccurate_metrics Figure 21 A sub BN for the risk factor “Inaccurate metrics” +inaccurate_cost _estimating +schedule_pressure excessive_reliance_on_a_sing le_process_improvement +defect_rate Figure 22 A sub BN for risk factor “Excessive reliance on a single process improvement” 138 +communication_overhead ++productivity lack_of_experience_with_ project_environment ++staff_training Figure 23 A sub BN for the risk factor “Lack of experience with project environment” +defect_rate +communication_overhead lack_of_experience_with_project_software +productivity +staff_training Figure 24 A sub BN for the risk factor “Lack of experience with project software” 139 ...MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Nguyen Ngoc Tuan RISK MANAGEMENT IN SOFTWARE PROJECT SCHEDULING USING BAYESIAN NETWORKS Major: Software Engineering... suitable for capturing uncertainty in projects Yet, BNs are rarely applied in project risk management in general as well as in software project management and software project scheduling The author... initial attempts of applying BNs into risk management in software project scheduling as well as experiments on common risk factors and their impacts in software project scheduling 19 common risk