Critical Systems Validation

Critical Systems Validation

©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 1Critical Systems Validation©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 2Objectives To explain how system reliability can bemeasured and how reliability growth modelscan be used for reliability prediction To describe safety arguments and how theseare used To discuss the problems of safety assurance To introduce safety cases and how these areused in safety validation©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 3Topics covered Reliability validation Safety assurance Security assessment Safety and dependability cases ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 4Validation of critical systems The verification and validation costs for criticalsystems involves additional validation processesand analysis than for non-critical systems:• The costs and consequences of failure are high so it ischeaper to find and remove faults than to pay for systemfailure;• You may have to make a formal case to customers or to aregulator that the system meets its dependabilityrequirements. This dependability case may requirespecific V & V activities to be carried out.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 5Validation costs Because of the additional activities involved,the validation costs for critical systems areusually significantly higher than for non-critical systems. Normally, V & V costs take up more than50% of the total system development costs.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 6Reliability validation Reliability validation involves exercising the programto assess whether or not it has reached the requiredlevel of reliability. This cannot normally be included as part of a normaldefect testing process because data for defecttesting is (usually) atypical of actual usage data. Reliability measurement therefore requires aspecially designed data set that replicates thepattern of inputs to be processed by the system. ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 7The reliability measurement processComputeobservedreliabilityApply tests tosystemPrepare testdata setIdentifyoperationalprofiles©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 8Reliability validation activities Establish the operational profile for thesystem. Construct test data reflecting the operationalprofile. Test the system and observe the number offailures and the times of these failures. Compute the reliability after a statisticallysignificant number of failures have beenobserved.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 9Statistical testing Testing software for reliability rather than faultdetection. Measuring the number of errors allows the reliabilityof the software to be predicted. Note that, forstatistical reasons, more errors than are allowed forin the reliability specification must be induced. An acceptable level of reliability should bespecified and the software tested and amended untilthat level of reliability is reached. ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 10Reliability measurement problems Operational profile uncertainty• The operational profile may not be an accuratereflection of the real use of the system. High costs of test data generation• Costs can be very high if the test data for thesystem cannot be generated automatically. Statistical uncertainty• You need a statistically significant number offailures to compute the reliability but highlyreliable systems will rarely fail.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 11Operational profiles An operational profile is a set of test data whosefrequency matches the actual frequency of theseinputs from ‘normal’ usage of the system. A closematch with actual usage is necessary otherwise themeasured reliability will not be reflected in the actualusage of the system. It can be generated from real data collected from anexisting system or (more often) depends onassumptions made about the pattern of usage of asystem.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 12An operational profile .Number ofinputsInput classes ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 13Operational profile generation Should be generated automaticallywhenever possible. Automatic profile generation is difficult forinteractive systems. May be straightforward for ‘normal’ inputsbut it is difficult to predict ‘unlikely’ inputs andto create test data for them.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 14Reliability prediction A reliability growth model is a mathematical model ofthe system reliability change as it is tested and faultsare removed. It is used as a means of reliability prediction byextrapolating from current data• Simplifies test planning and customer negotiations.• You can predict when testing will be completed anddemonstrate to customers whether or not the reliabilitygrowth will ever be achieved. Prediction depends on the use of statistical testing tomeasure the reliability of a system version.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 15Equal-step reliability growthReliability(ROCOF)t1 t2 t3 t4 t5Time ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 16Observed reliability growth The equal-step growth model is simple but it doesnot normally reflect reality. Reliability does not necessarily increase with changeas the change can introduce new faults. The rate of reliability growth tends to slow down withtime as frequently occurring faults are discoveredand removed from the software. A random-growth model where reliability changesfluctuate may be a more accurate reflection of realchanges to reliability.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 17Random-step reliability growtht1 t2 t3 t4 t5TimeNote dif ferent r elia bilityimprovementsFault r epair ad ds ne w faultand decreases reliability(increases ROC OF)Reliability(ROCOF)©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 18Growth model selection Many different reliability growth models havebeen proposed. There is no universally applicable growthmodel. Reliability should be measured and observeddata should be fitted to several models. The best-fit model can then be used forreliability prediction. ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 19Reliability predictionReliabilityRequir edreliabilityFitted r eliabilitymodel curv eEstima tedtime of relia bilityachie vementTime= Measur ed r eliability©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 20Safety assurance Safety assurance and reliabilitymeasurement are quite different:• Within the limits of measurement error, youknow whether or not a required level ofreliability has been achieved;• However, quantitative measurement of safety isimpossible. Safety assurance is concerned withestablishing a confidence level in the system.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 21Safety confidence Confidence in the safety of a system canvary from very low to very high. Confidence is developed through:• Past experience with the company developingthe software;• The use of dependable processes and processactivities geared to safety;• Extensive V & V including both static anddynamic validation techniques. ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 22Safety reviews Review for correct intended function. Review for maintainable, understandablestructure. Review to verify algorithm and data structuredesign against specification. Review to check code consistency withalgorithm and data structure design. Review adequacy of system testing.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 23Review guidance Make software as simple as possible. Use simple techniques for software developmentavoiding error-prone constructs such as pointers andrecursion. Use information hiding to localise the effect of anydata corruption. Make appropriate use of fault-tolerant techniquesbut do not be seduced into thinking that fault-tolerantsoftware is necessarily safe.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 24Safety arguments Safety arguments are intended to show that thesystem cannot reach in unsafe state. These are weaker than correctness argumentswhich must show that the system code conforms toits specification. They are generally based on proof by contradiction• Assume that an unsafe state can be reached;• Show that this is contradicted by the program code. A graphical model of the safety argument may bedeveloped. ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 25Construction of a safety argument Establish the safe exit conditions for a component ora program. Starting from the END of the code, work backwardsuntil you have identified all paths that lead to the exitof the code. Assume that the exit condition is false. Show that, for each path leading to the exit that theassignments made in that path contradict theassumption of an unsafe exit from the component.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 26Insulin delivery codecurrentDose = computeInsulin () ;// Safety check - adjust currentDose if necessary// if statement 1if (previousDose == 0){if (currentDose > 16)currentDose = 16 ;}elseif (currentDose > (previousDose * 2) )currentDose = previousDose * 2 ;// if statement 2if ( currentDose < minimumDose )currentDose = 0 ;else if ( currentDose > maxDose )currentDose = maxDose ;administerInsulin (currentDose) ;©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 27Safety argument modelcurrentDose = 0currentDose=0if statement 2then branchexecutedcurrentDose =maxDosecurrentDose=maxDoseif sta tement 2else branchexecutedif sta tement 2not e xecutedcurrentDose >= mini mumDose andcurrentDose <= maxDoseorcurrentDose >maxDoseadministerInsulinContr adictionContr adiction Contr adictionPre-conditionfor unsafe sta teOverdoseadminister edassignassign ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 28Program paths Neither branch of if-statement 2 is executed• Can only happen if CurrentDose is >= minimumDose and<= maxDose. then branch of if-statement 2 is executed• currentDose = 0. else branch of if-statement 2 is executed• currentDose = maxDose. In all cases, the post conditions contradict theunsafe condition that the dose administered isgreater than maxDose.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 29Process assurance Process assurance involves defining a dependableprocess and ensuring that this process is followedduring the system development. As discussed in Chapter 20, the use of a safeprocess is a mechanism for reducing the chancesthat errors are introduced into a system.• Accidents are rare events so testing may not find allproblems;• Safety requirements are sometimes ‘shall not’requirements so cannot be demonstrated through testing.©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 30Safety related process activities Creation of a hazard logging and monitoringsystem. Appointment of project safety engineers. Extensive use of safety reviews. Creation of a safety certification system. Detailed configuration management (seeChapter 29). [...]... problems are more generic - many systems suffer from the same problems; Safety problems are mostly related to the application domain ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 36 Security validation  Experience-based validation • The system is reviewed and analysed against the types of attack that are known to the validation team.  Tool-based validation • Various security... observed data should be fitted to several models.  The best-fit model can then be used for reliability prediction. ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 1 Critical Systems Validation ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 2 Objectives  To explain how system reliability can be measured and how reliability growth models can be used... safety arguments and how these are used  To discuss the problems of safety assurance  To introduce safety cases and how these are used in safety validation ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 3 Topics covered  Reliability validation  Safety assurance  Security assessment  Safety and dependability cases ©Ian Sommerville 2004 Software Engineering, 7th edition.... use. ©Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 24 Slide 39 The system safety case  It is now normal practice for a formal safety case to be required for all safety -critical computer-based systems e.g. railway signalling, air traffic control, etc.  A safety case is: • A documented body of evidence that provides a convincing and valid argument that a system is adequately safe... 20.02.20 System: Insulin Pump System Safety Engineer: James Brown File: InsulinPump/Safety/HazardLog Log version: 1/3 Identified Hazard Insulin overdose delivered to patient Identified by Jane Williams Criticality class 1 Identified risk High Fault treeidentified YES Date 24.01.99 Location Hazard Log, Page 5 Fault treecreators Jane Williams and Bill Smith Fault treechecked YES Date 28.01.99 Checker James . edition. Chapter 24 Slide 4Validation of critical systems The verification and validation costs for criticalsystems involves additional validation processesand. edition. Chapter 24 Slide 5Validation costs Because of the additional activities involved,the validation costs for critical systems areusually significantly