Accident Precursor Analysis And Management ppt

Advances: Engineering Risk Analysis Page 1 of 40 Ch 16 060502 V04 16 The Engineering Risk Analysis Method and Some Applications M. Elisabeth Paté-Cornell ABSTRACT Engineering risk analysis methods, based on systems analysis and probability, are generally designed for cases in which sufficient failure statistics are unavailable. These methods can be applied not only to engineered systems that fail (e.g., new spacecraft or medical devices), but also to systems characterized by performance scenarios including malfunctions or threats. I describe some of the challenges in the use of risk analysis tools, mainly in problem formulation, when technical, human and organizational factors need to be integrated. This discussion is illustrated by four cases: ship grounding due to loss of propulsion, space shuttle loss caused by tile failure, patient risks in anesthesia, and the risks of terrorist attacks on the US. I show how the analytical challenges can be met by the choice of modeling tools and the search for relevant information, including not only statistics but also a deep understanding of how the system works and can fail, and how failures can be anticipated and prevented. This type of analysis requires both imagination and a logical, rational approach. It is key to pro-active risk management and effective ranking of risk reduction measures when statistical data are not directly available and resources are limited. Advances: Engineering Risk Analysis Page 2 of 40 Ch 16 060502 V04 CONTENTS Engineering Risk Analysis Method: Imagination and Rationality Pro-Active Risk Management Early Technology Assessment and Anticipation of “Perfect Storms” Remembering the Past While Looking Ahead A Brief Overview of the Method and Formulation Challenges The Challenge of Structuring the Model Dynamic Analysis Imagination and Rationality Incomplete Evidence Base Data The Tool Kit Extension of RA to Include Human and Management Factors: The SAM Model Example 1. Ship Grounding Risk: Influence Diagram and SAM Model Representation The Grounding of Oil Tankers or Other Cargo Ships Problem Formulation Based on a SAM-Type Influence Diagram The Overall Risk Analysis Model Example 2. A Two-Dimensional Risk Analysis Model: The Heat Shield of the Space Shuttle Orbiters Example 3. A Dynamic Analysis of Accident Sequences: Anesthesia Patient Risk Example 4. Probabilistic Analysis of Threats of Terrorist Attacks Conclusions Advances: Engineering Risk Analysis Page 3 of 40 Ch 16 060502 V04 Engineering Risk Analysis Method: Imagination and Rationality Risk analysis for well known, well documented and steady-state systems (or stable phenomena) can be performed by methods of statistical analysis of available data. These include, for example, maximum likelihood estimations, and analyses of variance and correlations. More generally, these methods require a projection in the future of risk estimates based on a sufficient sample, of preferably independent, identically distributed data, and other experiences from the past. However, when designing or operating a new type of engineered system, one can seldom rely on such a body of evidence, even though there may exist relevant data regarding parts of the system or the problem. The same is true in all new situations in which the risk can only be evaluated from a rigorous and systematic analysis of possible scenarios, and from dependencies among events in a scenario. For instance, assessing the risk of a terrorist attack on the US requires “imagination” as emphasized in the 9/11 Commission Report (NCTA, 2004). To do so, one has to rely first on a system’s analysis, and second, on Bayesian probability and statistics (e.g., Savage, 1954; Press, 1989). The engineering method of “Probabilistic Risk Analysis” (PRA or here, simply RA), which was designed in the nuclear power industry among other fields (USNRC, 1975; Henley and Kumamoto, 1992; Bedford and Cooke, 2001), was presented in the previous chapter 1 . In what follows, I describe some specific features and applications of the engineering risk analysis method with the objective of finding and fixing system weaknesses, whether technical or organizational 2 (Paté-Cornell, 2000, 2002a). The focus is mostly on the formulation phase of a risk analysis, which can present major challenges. I describe and illustrate four specific problems and possible solutions: the explicit inclusion of human and management factors in the assessment of technical failure risks using influence diagrams 3 , with as an Advances: Engineering Risk Analysis Page 4 of 40 Ch 16 060502 V04 example, the case of ship grounding due to loss of propulsion; the characterization of the dynamics of accident sequences illustrated by a model of analysis of patient risk in anesthesia; the treatment of spatially-distributed risks with a model of the risk of an accident caused by a failure of the tiles of the NASA space shuttle; and the challenge of structuring the modeling of a type of threat that is new –at least on the scale that has been recently observed– illustrated by the risks of different types of terrorist attacks on the US. Pro-Active Risk Management Early Technology Assessment And Anticipation Of “Perfect Storms” The risk analysis (RA) method used in engineering is based both on systems analysis and probability and allows computation of the risk of system failure under normal or abnormal operating circumstances 4 . More importantly, it permits addressing and computing the risk of “perfect storms”, i.e., rare conjunctions of events, some of which may not have happened yet even though some of their elements may have been observed in the past. These events can affect, for instance, the performance of a new space system faced with a combination of challenges (e.g., a long voyage, cosmic rays, planetary storms, etc.). The same method can be used to perform early technology assessment, which is especially critical in the development of systems such as medical devices, which are expensive to engineer and less likely than not to pass the statistical tests required by the USFDA before approval (Pietzsch et al., 2004). In that context, RA can thus be used to anticipate the effectiveness and the safety of a new medical device when the practitioners may not be accustomed to it, when there may be some design problems, and/or Advances: Engineering Risk Analysis Page 5 of 40 Ch 16 060502 V04 when the patients happen to be particularly vulnerable (e.g., premature babies). In a different setting, one can also use this type of analysis to assess the risks of combined factors on a firm’s bottom line, for example, a competitor’s move, a labor strike that affects its supply chain, and/or a dip in demand caused by a major political event. In that perspective, RA can be applied, for instance to the quantification of the risks of bankruptcy in the insurance industry when a company is faced with a decline in market returns, repeated catastrophes, and prosecution of its executives for professional misconduct (Paté-Cornell and Deleris, 2005). Also, as described further, the same RA method can be used to support the choice of counter-terrorism measures, given the limited information provided by the intelligence community, in the face of ever- changing situations (Paté-Cornell, 2002b). Remembering the Past While Looking Ahead Anticipating rare failures, as well as shedding light on mundane but unrecognized problems, can provide effective support for risk management. But there is a clear difference between probabilistic risk analysis and expected-utility decision analysis (e.g., Raiffa, 1968), in which the decision makers are known at the onset of the exercise (Paté-Cornell, 2006). The risk analysis question is often: what are the risks (as assessed by an analyst and a group of experts), and how can the results be formulated to best represent uncertainties and be useful to the eventual decision maker(s)? The key issue, in all cases, is to anticipate problems that may or may not have occurred before, and to recognize existing ones in order to devise pro-active risk management strategies. The engineering risk analysis method permits ranking risk management options and setting priorities in the use of resources. The quantification of each part of the problem by probability Advances: Engineering Risk Analysis Page 6 of 40 Ch 16 060502 V04 and consequence estimates allows their combination in a structured way, using both Bayes’ theorem (to compute the probability of various scenarios) and the total probability theorem (to compute the overall probability of total or partial failures). Effective risk management options can then be formulated. They include for instance, adding redundancies, but also, the observation of precursors, i.e., signals and near-misses, which permit anticipating future problems and implementing pro-active measures (Phimister et al., 2004). A Brief Overview of the Method and Formulation Challenges The Challenge of Structuring the Model The first step in a risk analysis is to structure the future possible events into classes of scenarios 5 as a set of mutually exclusive and collectively exhaustive elements, discrete or continuous. Each of these scenarios is a conjunction of events leading to a particular outcome. The choice of the model structure, level of detail, and depth of analysis is critical: as one adds more details to a scenario description (A and B and C etc.), its probability decreases. In the limit, the exact realization of a scenario in a continuous space would have a zero probability, making the exercise useless. Therefore, one needs first to formulate the model at a level of detail that is manageable, yet sufficient to identify and characterize the most important risk reduction options. This level of detail may vary from one subsystem to the next. Second, one needs to compute the probability of the outcomes that can result from each class of scenarios, adjusting the level of detail, as shown later, to reflect the value of the information of the corresponding variables as support for risk management decisions. Finally, one needs to quantify the outcomes of these scenarios and to aggregate the results, sometimes as a probability distribution for a single Advances: Engineering Risk Analysis Page 7 of 40 Ch 16 060502 V04 attribute (e.g., money), displayed as a single risk curve (e.g., the complementary cumulative distribution of annual amounts of potential damage); or as the joint distribution of several attributes of the outcome space 6 (e.g., human casualties and financial losses). To represent the fundamental uncertainties about the phenomenon of interest, one can display a family of risk curves, which represent a discretization of the distribution of the probability (or future frequency) of exceeding given levels of losses per time unit (Helton, 1994; Paté-Cornell, 1996, 1999b). One can thus represent accident scenarios in various ways. The first is simply accident sequences, starting with initiating events followed by a set of intermediate events leading to an outcome described either by a single measure (e.g., monetary) or by a multi-attribute vector. The distribution of these outcomes allows representation of the risk at various levels of failure severity. Another analytical structure is to identify “failure modes” or min-cut sets, i.e., the conjunctions (without notion of sequencing) of events that lead to system failure described as a Boolean variable (USNRC, 1975). These failure modes account for the structure of the system, e.g., the fact that the failure of a redundant subsystem requires failure of all its components. To model the risk using the accident sequence approach, note p(X) the probability of an event per time unit (or operation), p(X|Y) the conditional probability of X given Y, p(X,Y) the joint probability of X and Y, IE i the possible initiating events of accident sequences indexed in i, and F the (total 7 ) technical failure of the system. In its simplest form, one can represent the result of the PRA model as the probability p(F) of a system failure per time unit or operation as: p(F) = Σ i (p(IE i ) x p(F|IE i ) (1) Advances: Engineering Risk Analysis Page 8 of 40 Ch 16 060502 V04 where p(F|IE i ) can be computed as a function of the conditional probabilities of the (intermediate) events that follow IE i and lead to F. The accident sequences can be systematically represented, for instance through event trees and influence diagrams. Alternatively, one can start from the system’s failure modes. Noting M j these conjunctions of events (min-cut sets), one can write the probability of failure p(F) using the total probability theorem as: p(F) = Σ j p(M j ) – Σ j Σ k p(M j , M k ) + p (three failure modes at a time) – etc. (2) External events that can affect all failure modes (e.g., earthquakes) or the probabilities of specific events in an accident sequence can be introduced in the analysis at that stage. The method is to condition the terms of the equation(s) on the occurrence (or not) of the common cause of failure and its severity level. The choice of one form or another (sequences vs. failure modes) depends on the structure of the available information. In the ship grounding risk analysis model and the risk analysis of a shuttle accident presented later, the accident-sequence structure was chosen because it was the easiest way to think systematically through a collectively exhaustive and mutually exclusive set of failure scenarios. However, faced with a complex system, best described by its functions and by a functional diagram, focusing on the failure modes might be an easier choice. Dynamic Analysis The robustness of a system as well as the challenges to which it is subjected may change over time. A structure fails when the loads exceed its capacity. On the one hand, one may want to account for the long-term pattern of occurrences of the loads (e.g., earthquakes), as well as the short-term dynamics of the different ways in which such events can unfold, for example, the Advances: Engineering Risk Analysis Page 9 of 40 Ch 16 060502 V04 time-dependent characteristics of the pre-shocks, main shock and aftershocks of an earthquake that can hit a structure. On the other hand the system’s capacity may vary as well. It can deteriorate independently from the loads (e.g., by corrosion), or it can decrease because of the fatigue caused by repeated load cycles (e.g., the effect of the waves on a structure at sea). Accounting for variations of loads and capacities requires a knowledge base that may come from different domains, e.g., from geophysics to structural engineering in the case of seismic risks. Another form of dynamic analysis may be required to analyze the evolution of accident sequences in which the consequences depend on the time elapsed between the initiating event and the conclusion of an incident. This is the case of an analysis of risks of fires in oil refineries (Paté-Cornell,1985) as well as that of patient risks in anesthesia described further. In both cases, stochastic processes were used to describe the evolution of the system over time, which is needed when the timing of human intervention is essential to effective risk management. Imagination and Rationality This RA method has been developed in details in the past for specific cases such as electrical circuits, civil engineering systems, nuclear reactors, aircraft, and space systems. But in its principles, RA as shown further, has applications to many other problems for which one needs to “imagine” systematically, beyond a simple, arbitrary “what-if” exercise, the potential failures in absence of directly relevant experience. In these cases, the choice of evidence is critical because available information may be incomplete and imperfect, yet essential to support a rational decision that needs to be made, before the occurrence of an event such as a specified type of terrorist attack or before a medical device is used in a real setting. Advances: Engineering Risk Analysis Page 10 of 40 Ch 16 060502 V04 Imagination and rationality are thus two main bases of the PRA method. Risk analysis is meant to support risk management decisions, assuming a rational decision maker or a homogenous group of them 8 . Rationality is defined here by the von Neumann axioms of decision making (von Neumann and Morgenstern, 1947), and by the definition of probability that they imply 9 . This Bayesian definition differ from the classical frequentist approach in that it relies on a degree of belief based on a decision maker’s willingness to make bets and to choose among lotteries given all available evidence. Therefore, by definition, this kind of probability cannot be “validated” in the classical statistical sense, at least not until one has gathered a sufficient body of experimental data, and provided that the system has remained in a steady state. This is rarely the case in innovative engineering or policy making. Instead, one has to seek justification of the model through a careful presentation of assumptions, reasoning, data and conclusions. Incomplete Evidence Base The Bayesian RA method is thus at the root of evidence-based decisions 10 , but this does not necessarily imply that the evidence involves a complete set of classic statistical data. Again, this is true because one often has to make such decisions in the face of uncertainty (e.g., in medicine or in astronautics) before complete information can be obtained. Therefore, the method uses all the evidence that exists, imperfect as it may be when needed, as opposed to the “perfect” one that one would want to have to follow the classic statistics path. In effect, the inputs of the RA method, i.e., the best information available, may be subjective and imperfect, but it may be the best one has and the process by which the output is generated is a rigorous one. Since one often needs to use the concept of Bayesian probability based on a degree of belief, the first question is, of course: whose beliefs? At the onset of a risk analysis, the identity [...]... Decision Analysis, 1: 71-78 Kahneman, D., P Slovic, and A Tversky, Eds (1982) Judgment Under Uncertainty: Heuristics and Biases Cambridge University Press Keeney, R.L and H.Raiffa (1976) Decision Analysis with Multiple Objectives: Preferences and Value Trade-offs John Wiley and Sons, New York Murphy, D.M and M.E Paté-Cornell (1996) The SAM Framework: A Systems Analysis Approach to Modeling the Effects of Management. .. system failure indexed in h, Advances: Engineering Risk Analysis Page 14 of 40 Ch 16 060502 V04 (DAm) the probabilities of the decisions and actions of the different actors, and MNn the relevant management factors that affect peoples decisions and actions Management Factor #1 Management Factor #2 MANAGEMENT SYSTEM Level 3 Decision 1 DECISIONS AND ACTIONS Decision 2 Level 2 Initiating Event #1 Intermediate... organizational factors and risk management options Risk Analysis 17(4): 511–523 Paté-Cornell, M.E., D.M Murphy, L.M Lakats and D M Gaba (1996b) Patient risk in anesthesia: probabilistic risk analysis, management effects and improvements Annals of Operations Research 67(2): 211–233 Paté-Cornell, M E (1996) Uncertainties in risk analysis: six levels of treatment, Reliability Engineering and System Safety,... pro-active risk management as an alternative to merely responding to the last event, and to set priorities among risk management under common constraints of time and money The keys to its success Advances: Engineering Risk Analysis Page 34 of 40 Ch 16 060502 V04 remain imagination, i.e., the willingness and the ability to face events that have not occurred yet) and rationality, i.e., the discipline and the... for example, on economic analysis When expanded to the analysis of risk management decisions, the tool kit includes decision trees (and the corresponding version of influence diagrams) and utility functions, single or multi-attribute (Keeney and Raiffa, 1976) Simulation is often needed to propagate uncertainties through the model in order to link uncertainties in the input and those in the output To... lead to a breach in the hull and in the case of oil tankers, release of various quantities of oil in the sea, and possibly, sinking of the ship Advances: Engineering Risk Analysis Resource Constraints (time and budget) Maintenance Quality Page 17 of 40 Ch 16 060502 V04 Level 3 Personnel Management MANAGEMENT LEVEL Level 2 Skill level of the Captain and the crew HUMAN DECISIONS AND ACTIONS Level 1 Weather... inspection and repair when needed The decisions and actions of crews are treated here as random events and variables conditional on a particular management system In this example, the evidence base includes mostly statistics of the frequency of loss of propulsion for the kind of ship and propulsion system considered and on expert opinions The Overall Risk Analysis Model Based on the influence diagram... in Safety Assessments of Technological Systems Science, 250:1359-1364 Bedford T and R.M Cooke (2001) Probabilistic Risk Analysis: Foundations and Methods Cambridge University Press Bier V.M and Louis A Cox (2006) Probabilistic Risk Analysis for Engineered Systems, Chapter 16 in Advances in Decision Analysis, Edward, Miles and von Winterfeldt Eds., Cambridge University Press Budnitz R J., Apostolakis... Probabilistic Seismic Hazard Analysis Risk Analysis 18 (4): 463-469 Clemen R.T and R.L Winkler (2006) Aggregation of expert probability judgments Chapter 8 in Advances in Decision Analysis, Edward, Miles and von Winterfeldt Eds., Cambridge University Press Cooke R.M (1991) Experts in uncertainty: opinion and subjective probability in science Oxford University Press Davoudian K., J.-S Wu, and G Apostolakis Incorporating... assessment through the analysis of work processes Reliability Engineering and System Safety, 45: 85-105 Advances: Engineering Risk Analysis Page 35 of 40 Ch 16 060502 V04 Helton, J.C (1994) Treatment of uncertainty in performance assessments for complex systems Risk Analysis, 14: 483-511 Henley E., and H Kumamoto, (1992) Probabilistic Risk Assessment: Reliability Engineering, Design, and Analysis, IEEE Press: . understanding of how the system works and can fail, and how failures can be anticipated and prevented. This type of analysis requires both imagination and. information. In the ship grounding risk analysis model and the risk analysis of a shuttle accident presented later, the accident- sequence structure was chosen