Safer Surgery 354 base on surgical training and technical performance, the technical skill of the surgeon (i.e., surgeons’ motor skill) has been recognized as a factor that mediates the relationship between patient risk factors and patient outcomes (e.g., Aggarwal et al. 2004, Dankelman and DiLorenzo 2005, Fried and Feldman 2008). In the past ve years, it has been proposed that this relationship should be qualied further. A number of other skills and factors have been suggested as potential determinants of performance and outcomes, alongside motor skills. These include cognitive and behavioural skills and also other factors, such as the operating theatre environment and procedures. Because of its multi-factorial perspective on surgical performance, error and outcome, this approach has been termed the ‘systems approach’ (Calland et al. 2002, Healey and Vincent 2007, Vincent et al. 2004). The systems approach to surgical performance is novel, and thus the existing conceptual and empirical work that stems from it is still somewhat limited. Interpersonal and cognitive skills are required for many occupations; specic subsets of such skills have been identied in the relevant literature as being essential for surgeons (Healey et al. 2006a, Yule et al. 2006). On the one hand, there are skills that are related to the way the operating surgeon interacts with other members of the operating theatre team – including assistant surgeon(s) and members of the anaesthetic and nursing sub-teams. On the other hand, there are the skills that reect surgeons’ cognitive competence, such as decision-making. These skills are complementary to the manual dexterity of the surgeons and they have been termed, collectively, ‘non-technical skills’. The working hypothesis for non-technical skills is that they contribute to surgical patient safety. The focus of this chapter is on one of these non-technical skills, namely decision-making. Decision-making is becoming increasingly prominent in the surgical skills and training literature (e.g., Flin et al. 2007, Sevdalis and McCulloch 2006). Decision- making is a cognitive skill, involving a number of inter-related steps (which may occur in very rapid, often unconscious sequence): recognition that choice between different courses of action is available; perception of different courses of action; evaluation of each course of action in relation to its potential risks and benets; actual choice; monitoring of the patient’s progress in relation to the decision taken (with review and change of plan if appropriate). As in other high-risk environments, in surgery there are a number of different decision-making ‘modes’ (Flin et al. 2007, Sevdalis and McCulloch 2006). These modes of thinking span a continuum, ranging from implicit/cognitively untaxing modes, characterized by intuition and simply ‘knowing what to do’, (recognition- primed decisions; Klein 1998) to explicit/cognitively taxing (analytical decisions, in which the various options available are weighed up consciously and 1. 2. 3. 4. 5. Surgical Decision-Making 355 carefully). Typically, a surgeon applies different decision strategies to different problems, depending on a variety of factors including the type of problem, whether it is familiar or not, time pressure, the potential consequences, the surgeon’s level of expertise, and other factors (Flin et al. 2007, Sevdalis and McCulloch 2006). For instance, the decision whether to offer surgery to a patient presenting with symptoms suggestive of gallstone disease may be intuitive (if the patient is signicantly symptomatic from gallstones and t for an operation, making the decision clear cut), or nearer the analytical end of the continuum if the patient is elderly and frail and the balance of risks and benets is less clear cut. In contrast, the decision when a patient can eat and drink post-operatively appears to be a rule- based decision (with rules that differ somewhat between surgeons; Jacklin et al. 2008a). (ii) Surgical Decision-Making in Context Surgeons are faced with important choices at all stages of patient care and their decisions may be made in very different ways. At the pre-operative stage, deciding whether or not to operate can be a delicate balance of risks and benets, made at leisure and collaboratively with the patient in the context of elective surgery. In the emergency situation, the same decision is likely to be made under time pressure. Intra-operatively, a sequence of operative steps is carried out, but decision-making continues – for example, deciding whether the anatomy has been clearly identied and whether the operation can proceed as anticipated, or the steps require re- evaluating. For instance, if a strangulated hernia is operated on, a decision must be made as to whether the bowel that was contained within it remains viable (and can be returned to the abdomen safely) or is beyond salvage (and must be resected), with attendant risks of leak from the anastomosis and longer post-operative recovery. As in other high-risk environments, decision-making in surgery has some distinctive characteristics. Firstly, many surgical decisions carry high stakes. For the patient, the potential consequences of an operation (in addition to the desired ones) may include disability or death. Operating without causing some harm is impossible, so in every decision to operate there is a trade-off between the likely benet and the inevitable anatomical (as well as physiological and psychological) damage caused to the patient as a primary consequence of the surgery itself. Secondly, surgery is associated with complications and their consequences, which may be difcult to predict for any individual patient, but tend to have a relatively predictable incidence overall. In other words, surgical decisions are associated with uncertainty about their outcomes at the level of the individual patient. For example, elective open repair of abdominal aortic aneurysms has a mortality of up to 7 percent in some institutions (Hertzer 2006) A third special feature of surgical decisions is their potential irreversibility. Once a laparotomy has been performed or tissue has been excised, this cannot Safer Surgery 356 be undone. Reversibility of surgical decisions is relative, as some aspects of an operation may be reversible. For example, where a stoma has been created and the gut left in discontinuity, the continuity may subsequently be restored at a further operation. Finally, unlike many other medical decisions, intra-operative decision-making usually cannot be shared with the patient. Although operative details and likely ndings can be discussed with patients pre-operatively, signicant procedures take place under general anaesthetic or with spinal/local block often in combination with sedation. Patients are either unconscious or have impaired consciousness, with responsibility for operative aspects of decision-making resting with the primary or senior surgeon and their team. (iii) Methods for Studying Surgical Decision-Making There is a substantial established body of theory and experimental research into the psychology of decision-making and the related eld of problem solving, and the cognitive processes that they involve. For a comprehensive summary of the eld, see Koehler and Harvey’s handbook (2004). In addition, there is a related literature on the nature of expert performance. Research into these areas has been published for the most part in a number of dedicated journals, such as the Journal of Behavioral Decision Making, Organizational Behavior and Human Decision Processes, the Journal of Risk and Uncertainty, Group Decision and Negotiation and Medical Decision Making. Although the methods used by decision researchers are unfamiliar to many surgical researchers, many have been adapted for use in the healthcare setting. A number of theoretical approaches and empirical applications are described in Chapman and Sonnenberg’s Decision Making in Health Care edited volume (2003). The book incorporates various decision research approaches to medical decision-making (including decision modelling, health economics and cognitive approaches) and covers a number of applications of this line of work (including decision support and assessment of patients’ preferences). From a methodological viewpoint, Harries and Kostopoulou (2005) provide a thorough review of empirical approaches to the study of medical decision-making. Building on these reviews, in what follows we provide a brief overview of selected methodological approaches for empirical research into how surgeons actually make decisions in practice. (iii-i) Self-report Self-report, typically elicited in interviews, offers the opportunity for detailed exploration of particular decision-making issues. Interviewing surgeons about how they arrive(d) at a decision complements other methodological approaches by providing detailed information about the relevant cues to a decision, or identifying important contextual factors. Retrospective investigation of specic, Surgical Decision-Making 357 recent, real-life decisions using detailed interviews forms the basis of the critical incident approach. This is used extensively within a research paradigm known as Naturalistic Decision-Making (NDM; Klein et al. 1993, Lipshitz et al. 2001). NDM approaches focus on how experts actually make decisions in naturally occurring working environments (e.g., in the operating theatre). In surgery, interviews have been used successfully to capture surgeons’ subjective experience of stressors and impediments to their performance (Arora et al. 2009, Wetzel et al. 2006) and they can be useful additions to direct observation in the development of tools that capture expert knowledge in an accessible form and facilitate teaching/training of novice professionals, such as Hierarchical Task Analysis (e.g., Sarker et al. 2008). In the empirical section of this chapter, we describe an interview-based knowledge elicitation approach, in which interviews were used to capture expert knowledge on a given topic. (iii-ii) Structural Approaches and Decision Modelling Structural approaches are a group of methodologies that have in common their statistical approach to modelling relationships between inputs to a judgement or decision (i.e., decision cues) and their outputs. Judgement Analysis (Cooksey 1996a, 1996b) is one such structural approach. This methodology is based on presentation of a series of multiple-cue case proles. The impact of different factors may be assessed by varying the values of the selected cues in differing combinations. Using linear regression it is then possible to determine the inuence of different variables/factors on participants’ judgements. The analysis yields multiple regression-derived beta weights for individual risk factors, and model t. Data are typically analysed separately for each participant, but can also be aggregated to allow subgroup comparisons (e.g., novice vs. expert participants; Sevdalis and Jacklin 2008). Within surgery, this approach has been used to model surgeons’ prioritization decision-making for cardiac surgery (Kee et al. 1998) and general surgery (MacCormick and Parry 2006), and treatment of prostate cancer (Clarke et al. 2007) – and there are other clinical applications, including management of typical non-end-stage renal disease (Pster et al. 1999) and dental surgery (Koele and Hoogstraten 1999). One study described below used this approach to investigate how surgeons generate risk estimates on the basis of combinations of pre-operative risk factors. (iii-iii) Observational and Process Approaches Interpreted broadly, these approaches seek to characterize the decision-making process as it unfolds over time. Observation of real-life clinical work may focus on explicit information search, communication with colleagues and the patient during the decision-making process, the use of artefacts, or non-verbal interactions. In such approaches, researchers typically seek to capture clinicians’ thinking and/or behaviours reliably and robustly as these unfold over the time it takes to complete Safer Surgery 358 the task under investigation (e.g., arrive at a diagnosis or treatment decision, or carry out a procedure). Such thinking or behavioural ‘protocols’ (verbal or non-verbal) have been reported in the medical literature – with ‘think aloud’ protocols being perhaps the most well known (Ericsson and Simon 1984). In such protocols, study participants report the content of working memory during a decision-making task (Denig and Haaijer-Ruskamp 1994). Non-verbal approaches, such as eye-tracking of gaze patterns, can give information about information search patterns, and can also be used to test the veracity of subjective accounts (e.g., Leong et al. 2007). Alternatively, researchers may choose to develop an observation protocol (usually after some pilot observations and testing), which is then used to collect data in real time. Such observational tools focus more on observable behaviours and less on underlying cognitive processes (as the former are easier to observe directly than the latter). Studies of medical and surgical working environments have used this approach successfully (e.g., Coiera and Tombs 1998, Healey et al. 2006b). The third set of empirical studies that we report below uses a simulated clinical setting (simulated operating theatre) to provide a safe and standardized setting to assess surgical decision-making (among other skills) via real time observation of key behaviours between team members. (iv) A Multimodal Approach to Surgical Decision-making To date, our research group has investigated surgical decision-making using the following three approaches: knowledge elicitation from expert surgeons, used to model a care pathway from the decision-making perspective (Jacklin et al. 2008a); experimental approaches to the modelling of surgical risk estimation (Jacklin et al. 2008b, Sevdalis and Jacklin 2008); simulation-based assessment of decision-making, in which decision- making is assessed alongside other skills (Undre et al. 2007, Koutantji et al. 2008). In what follows, we present the methodology and key ndings of these studies (readers are referred to original publications for more detail). We discuss the ndings and their implications in the concluding section of the chapter. (iv-i) Knowledge elicitation One of the distinguishing features of expertise is that when expert and novice professionals are presented with a situation that requires action or a problem that needs addressing, their perception of it (including their prioritization for actions to be taken and when) is markedly different. Training junior colleagues, therefore, involves an expert conveying how they grasp a situation – for instance, how they 1. 2. 3. Surgical Decision-Making 359 use just the relevant information and discard irrelevant ‘noise’. In the context of surgery, however, such training is rather hard, as only rarely are surgical decisions of experts explicitly deconstructed into their constituent parts and explained to trainee surgeons. We conducted an interview study with the aim to provide such a method of deconstructing a series of interrelated surgical decisions (Jacklin et al. 2008a). Methods Semi-structured interviews were conducted with ten expert surgeons (three senior specialist registrars and seven consultants), focusing on the care pathway of patients with symptomatic gallstone disease. First of all, participants were instructed to think of a patient attending an outpatient clinic with chronic symptoms of gallstone disease. Secondly, participants were asked to think of a patient presenting acutely at an emergency department, again with symptoms suggestive of a complication of gallstones. Participants were prompted to think about the decisions that they would be required to make throughout the patient’s care (consultation, admission, surgery, recovery and discharge). They identied the decisions made in each setting, along with the relevant cues for each decision, and any strategies or rules of thumb. A coding frame was developed to identify decisions and associated cues and rules from the text. Participants’ responses were coded for content by a surgeon (RJ) and a psychologist (NS) and the relevant decisions identied. Decisions were dened as any nodal choice in the pathway of care. Results The surgeon and the psychologist coders identied similar numbers of decisions per interview (M Surgeon = 16; M Psychologist = 15). A positive correlation between numbers of decisions per interview identied by each coder suggested reliable coding between raters (Spearman’s ρ = 0.61, N = 10, p < 0.06). Interestingly, whereas the surgeon coder considered ‘identication of anatomy’ as a key intra- operative decision, the psychologist coder did not – perhaps due to differences in their training and ensuing focus. Eighteen decisions that were identied in six or more of the interviews were extracted to form a decision ‘hot list’. A sample of these decisions across the care pathways is given below: Pre-operative: are the symptoms due to gallstones or not? Should the patient be offered surgery or not? Intra-operative: open port insertion or Veress needle? Is an intra-operative cholangiogram needed or not? Post-operative: when to take drains out? When to discharge? In addition, two distinct strategies for making these decisions were identied based on the surgeons’ self-reports taken from the coded transcripts. The rst was an intuitive strategy, described by participants as ‘experience based’, which was applied to decisions involving the assessment of a risk (e.g., decision whether the • • • Safer Surgery 360 symptoms were due to gallstones or not; decision whether to operate or not). Such decisions were characterized by multiple features taken into account to balance risks and benets, though how these were integrated into a decision could not be precisely articulated. Formal risk calculation tools were not used. The second, more rule-based strategy was associated with explicit criteria. For example, for the decision when to remove a drain post-operatively, surgeons often specied the absence of bile in the drain along with a volume criterion, the detail of which varied between individuals. Some technical aspects of the procedure (e.g., open port insertion or use of a Veress needle to create the pneumoperitoneum) were also subject to personal rules, again subject to individual variation. (iv-ii) Experimental/modelling approach A key limitation of self-report methodologies, such as knowledge elicitation as described above, is that participants may be unaware of some aspects of their own decision-making process (Nisbett and Wilson 1977). Thus, in addition to direct elicitation from experts, we also attempted in another study to model decision- making quantitatively, without resorting to the participants’ own perceptions of their decision-making process. We used Judgment Analysis (JA) to model surgeons’ estimation of the risk of converting a laparoscopic (key-hole) cholecystectomy to open (Jacklin et al. 2008b, Sevdalis and Jacklin 2008). Methods Thirty junior surgeons (minimum six months’ experience as trainee in general surgery after full registration as a doctor, or completion of the Royal College of Surgeons examination) participated in the study. Participants were presented with 84 case vignettes – 64 design cases and 20 repeat cases to assess reliability. The clinical factors that were manipulated across vignettes were patient’s biliary history, age/co-morbidity, previous surgery, obesity, sex and race (see Figure 21.1). In order to eliminate effects of the participants’ position on the learning curve with respect to operative expertise, participants were instructed that the procedure would be carried out by Mr J, an experienced laparoscopic surgeon. Participants were asked to indicate the likelihood (0–100 percent) that, given the case presentation, the procedure would have to be converted from laparoscopic to open. One key feature of this study was the incorporation of an outcome-based ‘gold standard’ model against which each case could be scored. This provided an evidence-based estimate of the likelihood of conversion for each of the cases in the study, against which to assess participants’ risk judgements. We analysed the use of the available information (cues) across participants. We also analysed participants’ reliability, accuracy and their use of the cues (i.e., cue beta weights in the regression models) with the gold standard model. Within the gold standard model, the biliary history, the patients’ sex and previous abdominal surgery were the only variables that inuenced the likelihood of conversion to open. Surgical Decision-Making 361 Results We observed signicant variation across participants regarding the correlation of their estimates with the gold standard: we obtained a range of Pearson r correlation coefcients between 0.08 and 0.72 – with lower coefcients indicating lower agreement with the gold standard (i.e., lower accuracy). The average observed r was 0.48 (SD = 0.14). Similar variation was observed in the cue utilization across participants (see Figure 21.2). Variability was obtained in model t across individual participants as well. Obtained model t ranged between R 2 adjusted = 0.12 (poor t) to R 2 adjusted = 0.76 Figure 21.1 A model for the study, with the cues that were available to participating surgeons Safer Surgery 362 (very good t), with mean R 2 adjusted = 0.53. The data shows that judgement analysis methodology can be used to model of surgeons’ risk judgements quantitatively, as well as to compare them with a gold standard model based on epidemiological data. Some participants were more consistent in their use of the cues, and, thereby, their responses were more amenable to modelling. The large variation in accuracy between individual surgeons was somewhat concerning clinically; for junior surgeons to be able to undertake tasks such as obtaining informed consent, from patients for procedures, there is a need for their understanding and estimation of surgical risks to improve. (iv-iii) Simulation In the past few years, simulation-based training has been used extensively for the honing of technical skill of surgical trainees (Aggarwal et al. 2004, Dankelman and DiLorenzo 2005, Fried and Feldman 2008). In commercial aviation (and other high risk industries), similar training modules have been developed for training pilots’ non-technical and crisis-management skills (Helmreich et al. 1999, Klampfer et al. 2001). We carried out two series of simulations, aiming to assess surgical teams’ crisis management in a simulated operating theatre (Undre et al. 2007, Figure 21.2 Cue utilization across individual surgeons Surgical Decision-Making 363 Koutantji et al. 2008). The simulated theatre is a fully equipped functional operating theatre separated from a control room by a one-way mirror and containing a standard operating table, operating lights, suction apparatus, anaesthetic machine and other equipment required for standard open or laparoscopic surgery, together with a moderate delity anaesthesia simulator mannequin (SimMan ® Laerdal, UK). The mannequin is connected to a compressor and controlled by a computer from the control room, with software that enables the controller to create an anaesthesia crisis for training and feedback. We modied existing, standardized scales that assess non-technical skills in the context of commercial aviation, assessed their reliability, and used them to assess non-technical skills (including decision-making) in surgical crisis simulation. Methods The rst training series consisted of 20 half-day simulations, in which a full operating theatre team of trainees (surgeon, anaesthetist, scrub nurse and operating department practitioner (these practitioners are technicians trained to assist the anaesthetists performing tasks similar to those performed in some contexts by anaesthetic nurses)) completed a sapheno-femoral junction ligation procedure that involved a number of crises. For the surgeons, the crisis was bleeding halfway through the procedure, leading to cardiac arrest. The second series consisted of nine full-day simulations, very similarly formatted and carried out. The difference was a training module (on safety and crisis management) that the trainees underwent half way through the day – as a pilot intervention. We used the NOn-TECHnical Skills (NOTECHS) observational tool. The NOTECHS was designed for use in the aviation industry to assess non-technical skills in cockpit crews – including decision-making, leadership skills, teamworking skills and situation awareness (Flin et al. 2003, van Avermaete and Kruijsen 1998). We revised the scale for use in surgical teams (Moorthy et al. 2005, Moorthy et al. 2006, Sevdalis et al. 2008). Here we focus on the assessment of surgical trainees’ decision-making. The trainees were observed (via one-way mirror in the control room of the simulated operating theatre) and assessed in real time by expert trainers and at least one psychologist. Upon completion of the procedure, trainers and trainees completed a revised NOTECHS and received feedback on the training exercise. Results Overall, the revised NOTECHS was found to be a reliable tool for the assessment of all non-technical skills in terms of internal consistency of the tool, and between participants and their trainers. Specically for decision-making, we obtained a range of Cronbach α coefcients between 0.82 and 0.92. Findings from the rst series of simulations can be seen in Table 21.1. We did not observe signicant differences between professional groups (surgeons, anaesthetists, scrub nurses and operating department practitioners) in relation to trainers’ decision-making ratings. However, we did obtain signicant differences across skills, with decision-making and leadership being rated signicantly lower than the other skills (F(4, 568) = 24.04, p < 0.001). Finally, trainer surgeons’ . expert participants; Sevdalis and Jacklin 2008). Within surgery, this approach has been used to model surgeons’ prioritization decision-making for cardiac surgery (Kee et al. 1998) and general surgery. experience as trainee in general surgery after full registration as a doctor, or completion of the Royal College of Surgeons examination) participated in the study. Participants were presented with. strategy, described by participants as ‘experience based’, which was applied to decisions involving the assessment of a risk (e.g., decision whether the • • • Safer Surgery 360 symptoms were