Safer Surgery 304 anaesthesia is also only partially represented. The specialized eld of activity for the anaesthesiologists is thus more realistic than for the other occupation groups. Since the content of the communication is determined by specialist activity, in this study we investigated only the anaesthesiologists’ communication, not that of the surgeons or the nursing staff. Anaesthesia simulators, like most ight simulators, are high delity simulators. These offer the advantage of allowing a relatively standardized way of observing how incidents are dealt with. Complete standardization is not possible, because the behaviour of the anaesthesiologists inuences the further course of the incident. The analysis of such scenarios thus faces the same problems as does problem- solving research with highly complex computer-simulated scenarios (see Dörner et al. 1983). Behaviour in simulator scenarios can already deviate from real operation situations because, in calm beginning phases, the participants are more prepared for critical events during an operation. Additionally, at least at the beginning, the participants are aware that they are in an observation situation. For this reason, utterances that often occur in calm phases of real operations, like jokes, lessons, and private conversation (Pettinari 1988), are rarely heard. Despite these limitations, physiologically and as an operation setting for anaesthesiologists, the simulator is at least apparently valid. In the scenarios we used (cf. Section 2.3), the anaesthesiologists exhibited a high degree of involvement which was conrmed in self reports (St Pierre et al. 2004). This high degree of the participants’ involvement during ‘hot phases’ of the scenario suggests that here they used their customary communication strategies, especially to coordinate with the nursing staff and surgeons. Research Questions The study presented here investigated how anaesthesiologists in critical situations in the simulator communicate with their nursing staff and the surgeons. The focus of the investigation is on the analysis of the anaesthesiologists’ utterances arose during the processed scenarios, focussing on communication. This includes the organization of behaviour and the coordination of the team: establishing shared mental models, conveying and requesting information, dening goals, planning, deciding, control, conict management, reection, etc. Special attention is paid to the interaction with the surgeons. Here, we pursued three issues: Description of the Communication (Exploratory, Descriptive Question) Since there are so few studies of communication in operations, we rst investigated what general kinds of utterances arise in the processed scenarios. A focus is on communication related to problem solving. We were also particularly interested in nding out whether clinical experience, gender or the kind of scenario had an inuence on the kinds of utterance. Observing Team Problem Solving and Communication in Critical Incidents 305 Connection between the Categories of Communication and the Quality of Medical Management (Hypothesis-testing and Exploratory Question) The results of human factors research in other occupational elds permits us to deduce the hypothesis that the quality of medical management is connected with communication. We therefore ask: how does the communication behaviour of anaesthesiologists differ under good and bad medical management in the scenarios? Quality of Communication in Critical Situations (Exploratory Question, Normative Approach) During the scenario’s critical situations, the communication was evaluated in terms of previously formulated behavioural expectations (behavioural markers): did the anaesthesiologists exhibit the type of communication behaviour that psychological and medical experts would expect in a team problem-solving process? Method Data Background: The Training Study ‘Human Factors in Anaesthesia’ With cooperation between the Simulator Centre of the Anaesthesia Clinic at Erlangen University Clinic and the Institute for Theoretical Psychology, a curriculum for physicians training for their specialization, ‘Human Factors in Anaesthesia’, was developed (St Pierre et al. 2004). This combined previously introduced simulator training for crisis management and psychological training modules on specic human factors topics. The psychological trainers are also involved in the feedback about the processing of anaesthesiological crisis scenarios in the simulator. For the rst module, ‘communication and cooperation in the OR’, three scenarios were developed that made specic demands on team problem solving and communication while dealing with incidents. This made it possible to evaluate not only the medical competence of the participants, but also their team-related problem-solving competence. Thus, the desirable integration of non-technical abilities (e.g., communication in an interdisciplinary team) and specialized procedures (e.g., stabilizing blood pressure) was achieved. The rst module of the curriculum was evaluated in an experimental design with a test group and a control group. The control group received a lecture on human factors in anaesthesia instead of the training unit. They worked through the same simulator scenarios. For a more detailed presentation of the study and of the training evaluation, (see St Pierre et al. 2004). The scenarios both groups worked through in the course of training were used for the evaluations presented here, because few differences were to be expected within the training (any differences are highlighted in this chapter). Safer Surgery 306 Sample The participants in the study were 34 interns at the University Clinic for Anaesthesiology in Erlangen. This was a random sample, except that women and men were distributed evenly between the two groups and among the training sessions. Because the sample was small, the participating women worked through Scenario 2 whenever possible. This means that the effects of sex and scenarios are confounded, but recognizable. Despite the partly chance allotment, it was possible to obtain homogeneous partial samples, with the exception that the individual scenarios were differently lled in terms of the sex and experience of the participants. Clinical experience ranged from one to six years with a mean of 3.3 years. Men and women did not signicantly differ in their mean length of clinical or in simulator experience. Scenarios Used For the training programme, the following scenarios were developed so as to make specic demands not only on the management of a medical incident, but also on problem solving and communication in the team. Each scenario (detailed below) was designed to take 30 minutes (the actual duration of the scenarios ranged from 16 to 42 minutes). The training programme’s three scenarios were each worked through by one participant, each supported by a real nurse. Simulator staff assumed the role of the surgeon, sometimes supported by a participant. The scenarios are based on a script that calls for fairly standardized communication from the instructed role-players in predetermined critical situations. For example, after a drop in blood pressure, the surgeon asks one of the anaesthesiologists whether he or she ‘isn’t managing back there’. If the participant ignores the question, the script prescribes as the surgeon’s Men Women Total sample N 22 12 34 Years of clinical experience 3.4 3.1 3.3 Proportion of participants with simulator experience 68% 42% 59% Scenario 1 11 1 12 Scenario 2 1 10 11 Scenario 3 10 1 11 Table 18.1 Sample of the sample Observing Team Problem Solving and Communication in Critical Incidents 307 ‘answer strategy’ that he or she ‘exert verbal pressure’. But if the anaesthesiologist communicates a problem, the script instructs to offer cooperation. The participants judged all three scenarios to be adequately realistic and to be stressful. On a ve-step Likert scale (1 = very realistic, 5 = not realistic at all), the means for evaluated realism were between 1.8 and 2.55 (n.s.); on a ten-step Likert scale (1 = boredom, 10 = overburdening), the stress caused by the scenario was reported as between 5.3 and 7.6 (n.s.). While extreme stress would deteriorate participants’ ability to problem solve whereas boredom would mean that they did not experience a critical situation (but instead routine), the medium stress levels reported seems to indicate that participants were challenged but not working at their limit. Scenario 1: Laparoscopic Cholecystectomy with Volume Deciency Reaction and Air Embolism In a laparoscopic cholecystectomy, the abdominal cavity is lled with CO 2 gas to provide the surgeon with better visibility. If the abdomen is inated too much, less blood can ow back from the abdomen to the heart, resulting in lower blood pressure and a faster pulse. This is the rst complication in the scenario. After the therapy, which requires close communication between the surgeon and the anaesthesiologist, operative inattention leads to bleeding in the abdominal cavity. CO 2 gas ows into the bloodstream and results in an air embolism. The anaesthesiologist must recognize this situation, which is acutely life-threatening for the patient, and plan the therapy, in which the surgeon must be integrated. The therapy consists in administering medications that stabilize circulation and, if appropriate, changing the operating procedure, organizing transesophageal ultrasound and transfer to the intensive care unit (ICU). Scenario 2: Occluded Perforated Abdominal Aorta Aneurysm This clinical picture is an aneurysm of the main artery in the upper abdomen (acute intense pain). The aneurysm tears or bursts, resulting in a life-threatening situation. This is the situation in this case. The anaesthesiologist must rapidly coordinate the operating procedure in close discussion with the surgeon and the nursing staff and attempt to stabilize circulation with the aid of providing volume (blood, infusions) and medications supporting circulation (catecholamines). Special communicative demands arise if clamping off is too fast or if the surgeon opens the aorta. In the end, the patient should be sent to the intensive care ward in a stable state. Scenario 3: Lung Embolism after Speculum Examination of the Knee in the Recovery Room This scenario is about a postoperative complication resulting from vascular congestion. The clinical picture develops suddenly when the bloodstream carries a Safer Surgery 308 blood clot (thrombus) into the lung, where it blocks a blood vessel. Thus, a section of the lung is no longer supplied with blood, and no gas exchange occurs here. The blood backs up to the heart and the heart muscle is acutely overburdened, resulting in circulatory failure and intense pain. The anaesthesiologist is called to a patient (who has had a knee operation) as an emergency and must familiarize him or herself with the situation, collect the necessary information and then organize the therapy. Treatment includes rstly, applying medications that support circulation, anaesthesia and respiration and thereafter, medications that reduce blood clotting. But, the use of such a thrombolytic after surgery must be discussed with the surgeon. For the severity of the embolism and the state of therapy to be judged, a number of specialists must be brought in and their judgements discussed. Observation Evaluation Tools The analysis of the scenarios is based on the methods of evaluation described in the following: a system of categories, ‘problem solving in a team’ behavioural markers for specic communication behaviours experts’ judgement of medical management. A Tool for Observing Problem Solving in the Anaesthesia Team A system of categories, ‘problem solving in a team’, was developed to categorize everything uttered in each scenario. It comprises 24 categories organised into ve ‘overarching categories’ labelled: (i) formal characteristics of the statement, (ii) organization of activity, (iii) relation the team and of processes, (iv) conict management and (v) other. The development of the system was oriented toward the phases of action organization, as developed by Dörner (1996), and toward considerations emerging from research on solving complex problems in groups (e.g., Stempe and Badke- Schaub 2002, 2003). It was supplemented by inductive category formation on the basis of video data from the anaesthesia simulator. Every remark was classied on the formal level and in one of the other four overarching categories. Randomness- corrected observer agreement on these categorizations reached 61 percent–80 percent (Cohen’s Kappa). Table 18.2 shows the overarching and subsidiary categories. Behavioural Markers for Specic Communication Behaviour Behavioural markers for communication were developed. Behavioural markers are behaviour patterns whose presence in a stream of behaviour indicates certain skills. For the present evaluations, anaesthesiologists and psychologists developed a set of behavioural expectations based on the scenario scripts. Studies using • • • Observing Team Problem Solving and Communication in Critical Incidents 309 behavioural markers often report low inter-rater reliability, but for our project, which aims to evaluate a training programme, a high concordance between observers was essential. So, we decided to formulate a set of very specic markers. They describe communicative behaviour required to solve a scenario optimally, for example the insistence on a slow de-clamping in the aorta aneurysm scenario. A list of behaviour-oriented observable items was developed that operationalizes the necessary communication competencies. The demands of each scenario were different, so 16 to 22 different markers were dened for each scenario. Two observers judged the presence of each marker in each person in the scenario (possible answers: yes, no, not applicable). The randomness-corrected observer agreement here was 82 percent (Cohen’s Kappa). This shows that it is easier to achieve good inter-rater reliability using more specic markers (but of course, the marker set has to be dened for every scenario that is evaluated). Examples for the behavioural markers used are shown in Table 18.3. Experts’ Judgement of Medical Management Two anaesthesia experts also independently judged the medical management of the scenarios. The experts were not aware that the videos were being evaluated in Overarching category Categories Formal characteristics Question, statement, directive/order, other New unit of activity, addressing the sur geon on own initiative Organization of activity Information gathering, model formation, conveying information (facts), decision, explanation of own activity, commentary on activity , conveying pr oblem and situation, conveying problem and situation with model, redundance, control, conrming understanding, hypothesis, anticipation, goal, plan Relation to team and process Utterances related to team and relationship, process or ganization Reection/emotional utterances/own feelings a Conict management Offer to engage in conict; b anaesthesiologist: objective, escalating, ignoring, de-escalating Other a Because pure utterances of reection were not expected, these categories were bundled together. b This is the only category that considers the surgeon’ s utterances, because a conict always arises from interaction. All utterances that could be considered offers to engage in conict were counted. Table 18.2 Category system ‘Problem solving in a team’ Safer Surgery 310 accordance with the aforementioned tools. Differing observations were discussed until agreement was achieved (communicative validation, e.g., Bauer and Gaskell 2000). For each phase of each scenario, a system was used in which points were given for quality of therapy, diagnostics, and, where applicable, monitoring. Each item could be scored from 0 to 2 points (bad to very good), which resulted in scores between 16 and 24 points for the scenarios. Table 18.4 shows the eight evaluation items for Scenario 1. Some Results As studies on problem solving or the analysis of thinking processes in the medical eld are rare (but see Gaba 1992), we started with explorative questions. We were able to formulate hypotheses concerning the eld of communication. We would like to highlight some of the ndings of our analysis that helped us improve our training programmes. In short: Critical situation in accordance with script Behavioural marker Scenario 1 Before the OP Gives the OK for the OP only after his/her own preparations are completed Changed position (head raised, feet lower ed) Anaesthesiologist conveys concern to the surgeon early Anaesthesiologist asks for a change of position/release of pressure Scenario 2 Cut R equests rapid clamping or conveys problem Asks the surgeon to report Clamping Intermediate brieng with nurse Improvement of circulation conveyed to surgeon Scenario 3 Anaesthesiologist enters r ecovery room Anaesthesiologist asks nurse what has happened Responsible superior is informed Surgeon r ejects heparin Anaesthesiologist remains objective Anaesthesiologist conveys reasons (acute danger to patient, life takes priority over knee … vital problem) Table 18.3 Examples of behavioural markers for evaluating communication in the scenarios used Observing Team Problem Solving and Communication in Critical Incidents 311 Anaesthetists talked more often than they expected they would across all scenarios. Almost half of all utterances help pacing or establishing shared mental models. We found nearly no explicit addressing of the team. There was nearly no talking about aims and plans (of more than one step). There were very few real questions. We found a high correlation (.56) between the quality of clinical management and communication measured with the behavioural markers. In reporting some results, we will give the explorative questions that lead us in the analysis followed by the answer. Description of Communication Amount and Type of Utterances How much do the participants talk, and what kind of remarks do they make? The anaesthesiologists spoke more during the scenarios than even they themselves expected: in preliminary talks, the intention to investigate communication during operations was repeatedly belittled as senseless on the grounds that there is little speaking during an operation (which also contradicts our observations of operations). There was a mean of 228 utterances per person; with an average scenario duration of 28 minutes, that is 8.2 utterances per minute. The sample showed no difference between men and women in the amount spoken. Utterances in the form of orders – an average of 25.4 per scenario – account for almost a tenth of all utterances. There were 31.3 questions asked per scenario. In terms of content, it should be considered that the proportion of genuine questions is much lower, because many directives are clothed in the form of a question (‘Would you hold the bag?’). Table 18.5 shows the distribution of these formal categories in the scenarios. The formal categories showed no signicant differences between the scenarios, sexes, or experience – nor any interaction between the factors. This nding is surprising, because it seems to mean that anaesthesiologists in the simulator • • • • • • Acute phase 1 (pneumoperitoneum with circulatory reaction) Acute phase 2 (discr. venous bleeding) Acute phase 3 (air embolism) A naesthesia introduced (0–2Pt) Differential diagnose (0–2Pt) Therapy (0–2Pt) Therapy (0–2Pt) Diag. standard (0–2Pt) Diag. advanced (0–2Pt) Therapy circulation (0–2Pt) Therapy breathing (0–2Pt) Table 18.4 Items for evaluating medical management (Scenario 1) Safer Surgery 312 utter a certain number of utterances of a specic kind. This should be further investigated. Proportion of Utterances Aiming at Team Coordination and the Establishment of a Shared Mental Model How much of what is said relates to the coordination of team activity and the establishment of shared mental models? Here we looked at the categories: conveying information; thinking out loud; conveying problems (facts only); conveying problems with an explanation or model; explanation of one’s own activity; redundance; conrming understanding; addressing the surgeon (anaesthesiologist’s initiative). An essential factor in successful problem solving is establishing shared mental models. This process cannot be completely observed, but there are utterances that explicitly suggest the intention of improving a shared mental model (e.g., conrming understanding; explanation of one’s own activity) and some that can help the other team members in ‘pacing’ (e.g., thinking aloud; conveying information). The importance of these tasks for problem solving is reected in the frequency of such utterances: the anaesthesiologist says something that can contribute to team coordination a mean of 108 times per scenario, almost four times per minute. This corresponds, in the mean, to almost half of all utterances (47 percent). But a mean of only 18 utterances were explicitly related to establishing shared mental models (conveying problems with explanation; explanation of one’s own activity; conrming understanding). Table 18.6 shows the distribution among categories that we regard as helpful in or as aiming directly at constructing shared mental models. As with the categories of action organization, there are enormous individual differences. The frequency of redundance seems to indicate the anaesthesiologist’s intense safety awareness. In these categories, we found there are no differences in relation to experience or sex. Category Question Directive/ order Statement/ utterance Other/ filler phrases Utterances total Mean 31.6 25.5 162.5 8.2 227.9 Minimum 10 2 89 0 126 Maximum 75 62 349 26 433 Table 18.5 Formal characteristics of utterances in the scenarios Observing Team Problem Solving and Communication in Critical Incidents 313 Utterances Concerning the Team and the Team Problem-solving Process How much of what is said relates to the team and the process of working together? We counted the categories: reection or emotional utterance; references to relationships; process. A large part of the speaking is devoted to coordinating activity (especially with the nurse); but very few utterances are directly related to the team. In the scenarios, only the relationship to the surgeon was thematized, usually to draw boundaries (in the sense of ‘Don’t interfere with my work, I don’t try to tell you what to do, either’), seldom to underscore the shared team task (e.g., ‘Now we have to manage this together’). Reection on the problem-solving process was bundled together with utterances of one’s own emotional state (e.g., ‘Here I’m not so sure, either…’), because we expected (and found) few self-reective utterances in the sense of strategy evaluation. Utterances related to the work process (‘Let’s do this now one step at a time’) accounted for a mean proportion of 5 percent; this is less than one would expect for ‘good team achievement’ (see Table 18.7). Interestingly, there was virtually no communication about goals or plans (less than 1 percent of all utterances). This may be due to the pressure of the situation, or it could indicate a learning need for team problem solving. The individual differences are substantial in the categories of team and problem-solving process, but women and men do not signicantly differ in their use of these categories (p=.360, t=.93; df=32). Nor do experience or the scenario type lead to signicant differences in these categories (F=2.04; p=.15; and F=0.17; p=.84). Category Mean Minimum Maximum Conveying information 16.6 4 42 Thinking out loud 14.0 1 68 Conveying problems (facts) 22.4 5 49 A ddressing surgeon on own initiative 9.4 0 34 Conveying problems with explanation 4.7 0 11 Explanation of own activity 7.5 0 15 Conrming understanding 5.8 0 16 Redundance 27.8 2 49 T otal 1 11.3 49 203 Table 18.6 Utterances related to team coordination and shared mental models . the training (any differences are highlighted in this chapter). Safer Surgery 306 Sample The participants in the study were 34 interns at the University Clinic for Anaesthesiology in Erlangen Safer Surgery 304 anaesthesia is also only partially represented. The specialized eld of activity for the anaesthesiologists. there’. If the participant ignores the question, the script prescribes as the surgeon’s Men Women Total sample N 22 12 34 Years of clinical experience 3.4 3.1 3.3 Proportion of participants with