Safer Surgery 264 Adjustment Variables and Outcomes The American Society of Anesthesiologists (ASA) score assigned by the anaesthesiologist was recorded. The ASA score subjectively categorizes patients into ve sub-groups by preoperative physical tness and appear in Table 16.2 (Mazzocco et al. 2008). The ASA score was devised in 1941 by the ASA as a statistical tool for retrospective analysis of hospital records and has been revised periodically (Walker 2002). In nine patients, the ASA score was not recorded in either the medical record or on the observation sheet. In these cases, an anaesthesiologist independent of the study reviewed information on patient characteristics obtained from the medical record review and assigned an ASA score. The surgical procedures were classied as low, medium or high risk for post-operative complications according to American College of Cardiology and American Heart Association guidelines (Eagle et al. 2002). Low risk procedures included biopsy, excision of mass, hernia repair and laparoscopic cholecystectomy; Behavioural Marker Domain Description Used in univariate analysis and calculation of Behavioural Marker Risk Index Brieng I nformation sharing Situation/relevant background shared; patient, procedure, site/side identied; plans are stated; questions asked; ongoing monitoring and communication encouraged Information is shared; intentions are stated; mutual respect is evident; social conversations are appropriate I nquiry Asks for input and other relevant information V igilance and awareness T asks are prioritized; attention is focused; patient/equipment monitoring is maintained; tunnel vision is avoided; red ags are identied Not used in univariate analysis calculation of Behavioural Marker Risk Index A ssertion The members of the team are speaking up with their observations and recommendations during critical times Contingency management Relevant risks are identied; back-up plans are made and executed Table 16.1 Description of domains behavioural markers of team behaviour assessed by the observers An Empiric Study of Surgical Team Behaviours and Patient Outcomes 265 Measure Definition Example ASA patient classication I Completely healthy patient A t patient II Patient with mild systemic disease Essential hypertension, mild diabetes without end organ damage III Patient with severe systemic disease that is not incapacitating A ngina, moderate to severe C OPD † IV Patient with incapacitating disease that is a constant threat to life Advanced C OPD, cardiac failure V A moribund patient who is not expected to live 24 hours with or without surgery Ruptured aortic aneurysm, massive pulmonary embolism ACC/AHA pr ocedure risk * Low Low risk of non –cardiac complications Biopsy, excision of mass, hernia repair, laparoscopic cholecystectomy Medium Medium risk of non-cardiac complications Mastectomy, thoracotomy, thyroidectomy, exploratory laparotomy H igh High risk of non-cardiac complications Repair of abdominal aortic aneurysm Outcome score 1 N o complications 2 One or more indicators of potential harm 3 Minor complication 4 Major complication 5 Death or permanent disability * Procedures listed as examples for the ACC/AHA procedure risk accounted for 85 percent of all procedures observed in this study. † Chronic Obstructive Pulmonary Disease. Table 16.2 Denitions of measures: patient risk of complications (American Society of Anesthesiologists – ASA – classication), procedure risk (American College of Cardiologists – ACC-score) and outcome (outcome score) Safer Surgery 266 medium risk included open laparotomy, carotid endarterectomy and thyroidectomy; and high risk included aortic aneurysm repair and femoral popliteal bypass. The 30-day outcome of each observed procedure was determined by medical record review using a standard instrument. The medical record reviewer was blinded to the behavioural risk index. Each reviewer had a list of common surgical complications (see the Appendix to this chapter) and these complications and other signicant outcomes were grouped into ve outcome categories: (1) no complications; (2) one or more indicators of potential harm (change in procedure; intubation/reintubation/ BiPap in PACU; non-routine X-ray intra-operative or in PACU; intra-op epinephrine or norepinephrine use; post op Troponin level > 0.5; change anaesthetic during surgery; consult requested in Post Anaesthetic Care Unit (PACU); path report normal or unrelated to diagnosis; and insertion of arterial or central venous line during surgery.); (3) minor complication characterized by one of the following: prolonged, unplanned operative time (e.g., greater than 1.5 × expected time); post- operative transfer to a higher level of care; unplanned return to surgery (within 72 hours); and unplanned ventilatory support for greater than 24 hours or more post-operatively; (4) major intra- or post-operative complication characterized by: prolonged, unplanned operative time (e.g., greater than 1.5 × expected time); post- operative transfer to a higher level of care; unplanned return to surgery (within 72 hours); unplanned ventilatory support for greater than 24 hours or more post- operatively (i.e., inability to extubate); unplanned emergency intervention by the surgical team or code team; and (5) death or permanent disability. Behaviour Risk Index For each procedure/team, the behavioural marker data were summarized using a single score, the Behavioural Marker Risk Index (BMRI), following the approach used by researchers studying group interactions in high risk environments (Dietrich and Childress 2004). Based on inspection of the univariate behavioural marker data, the markers assertion and contingency management were excluded from the BMRI because they were rarely observed in these generally low risk procedures done on mostly low and intermediate risk patients. The BMRI calculates the percent of ratings of behaviour made during the procedure that were less frequent than a rating of 3, or intermittent. BMRI was calculated by assigning a value of 1 if the observer rating for the domain was 0 (behaviour never observed) or 1 (behaviour rarely observed) or 2 (isolated or minimal observation of the behaviour). These values were summed across all phases of surgery for the four behavioural marker domains and then divided by the total number of domains/phases in which an observation was made. The BMRI thus had a range from 0.0 to 1.0 where values closer to 0.0 indicated more frequent observations of team behaviour. Those closer to 1.0 indicated less frequent observations of team behaviour (or as the label implies, ‘riskier’ team behaviour). The valence of the BMRI means that positive correlations of the BMRI with the An Empiric Study of Surgical Team Behaviours and Patient Outcomes 267 patient outcome score reect an association of failure to observe ‘good team behaviour’ with worse outcomes. Analysis Patient characteristics were summarized using means, counts and percent distributions as appropriate to the distribution of the variable. For descriptive analysis, patient outcomes were categorized into two categories – ‘complications or death’ or ‘no complications or death.’ The rst category included patients with both major and minor complications in addition to deaths. The second category included patients with one or more indicators of potential harm in addition to no complications. For each operative phase and BMRI domain, the increased odds of having complications or death associated with lower scores for team behaviour (0–2) were estimated by calculating odds ratios (OR) and 95 percent condence intervals (CI). Multiple logistic regressions were calculated to assess the independence of the associations of the BMRI domains with outcome after taking into account the ASA patient risk score. Two-way interactions involving the BMRI domains with the ASA patient risk were considered but were not signicantly (p>0.20) related to the outcome and not included in the nal adjusted models. Similar unadjusted and adjusted odds ratios and 95 percent condence intervals were calculated by logistic regressions with the BMRI as the predictor variable, the ASA patient risk score as the covariate adjusted for in the adjusted model, and ‘complications or death’ as the predicted outcome. Finally, we used the logistic regression model to calculate the predicted relationship between the BMRI and the OR for complications and death. Statistical analyses were conducted using SPSS version 14.0. Results Observer calibration was achieved to a RWG of 0.9 for the two main observers and a RWG calibration of 0.85 for all observers at the conclusion of training. A total of 300 patients/procedures were observed. The medical records for seven patients could not be located, so their observational data was excluded from the analysis. Table 16.3, reproduced from the prior publication (Mazzocco et al. 2008), shows characteristics of the 293 observed patients and procedures included in the analysis. The patients were mostly middle-aged. The gender and race/ethnicity distribution were generally representative of Kaiser Permanente members undergoing general surgery procedures at the participating hospitals. The patients were mostly low and medium risk; there were no patients in the ASA category V and only ve in the ASA high risk category. All but four of the procedures were ACC/AHA low or intermediate risk. More than one-half of the procedures had ‘no complications’ as the outcome rating. Three patients had an outcome of death or disability. In Safer Surgery 268 Characteristics N % Age range 18–34 44 (15) 35–49 64 (22) 50–74 145 (49) 75+ 40 (14) Race/ethnicity Asian/Pacic Islander 10 (3) A frican American 26 (9) Hispanic 49 (17) Non-H ispanic white 188 (64) Missing 20 (7) Gender F emale 174 (59) Male 119 (41) ASA classication I 47 (16) II 155 (53) III 86 (29) I V 5 (2) V 0 (0) ACC/AHA pr ocedure risk L ow 233 (80) Medium 56 (19) H igh 4 (1) Outcome No complications 158 (54) Table 16.3 Characteristics of 293 patients and procedures An Empiric Study of Surgical Team Behaviours and Patient Outcomes 269 about 25 percent of procedures, the BMRI was more than 0.50 indicating a high proportion of operative phases and domains with infrequent observation of good team behaviours. Table 16.4 (Mazzocco et al. 2008) shows, for each operative phase (induction, intra-operative, hand-off) and behavioural marker domain, the behavioural marker scores after dichotomizing them into categories of less frequent (0–2) or more frequent (3–4) observation of ‘good’ team behaviours along with the percentage of more frequent observation of good team behaviours. The table also shows the number and percentage of patients/procedures with a complication (major or minor) or death according to these scores by operative phase and behavioural marker domain along with the ORs and 95 percent CIs for complication or death for patients/procedures with scores indicating less frequent observation of ‘good’ team behavior. Because the referent in this analysis is patients with scores indicating more frequent observation of ‘good’ team behaviours, an OR above 1.0 indicates an association of less frequent team behaviors with poorer outcome. For most of the phases and domains, good team behaviours were observed frequently or always (scores 3–4) in a substantial percentage of procedures; however, for none of the phases or domains were good teams behaviours observed frequently or always, all of the time. The ORs for complication or death were greater than 1.0 when team behaviours were observed less frequently (scores 0–2) in all operative phases and behavioral domains except the brieng domain of the intra-operative phase and the vigilance domain of the hand-off domain. The OR estimates for complication or death excluded 1.0 in association with low scores for the information sharing domain of the intra-operative phase (OR 2.45; 95 percent CI 1.36–4.42) and for the brieng Characteristics N % One or more indicators of potential harm 71 (24) Minor complication 48 (16) Major complication 13 (4) Death or disability 3 (1) Behavioural Marker Risk Index categorical ranges 0.00–0.24 83 (28) 0.25–0.49 136 (46) 0.50–0.74 56 (19) 0.75–1.00 18 (6) Table 16.3 Concluded Safer Surgery 270 and information sharing domains of the hand-off phase (OR 2.34; 95 percent CI 1.23–4.46 and OR 2.21; 95 percent CI 1/18–4.16, respectively). The elevated OR for complication or death was close to 1.0 in association with a low score for the vigilance domain of the induction phase (OR 2.08; 95 percent CI 0.99–4.35). There were no signicant ndings for the remaining behavioural markers. Table 16.4 Description of behavioural markers scores by operative phase, number and percentage of procedures with complication or death, and odds ratios (OR) and 95 per cent condence intervals (CI) for complication or death for less frequent observation of ‘good’ team behaviours Operative Phase and Behavioral Marker Domain Score Teams/ Procedures Major or Minor Complications or Death N % of Total n (%) OR* 95% C.I. Induction Phase Brieng 0-2† 71 20 (28) 1.59 (0.86-2.93) 3-4‡ 222 (76) 44 (20) referent I nformation sharing 0-2† 48 12 (25) 1.24 ( 0.60-2.55) 3-4‡ 145 (84) 52 (21) referent Inquiry score 0-2† 1 18 28 (24) 1.20 (0.69-2.10) 3-4‡ 175 (60) 36 (21) referent Vigilance 0-2† 38 13 (34) 2.08 (0.99-4.35) 3-4‡ 255 (87) 51 (20) referent N % of Total n (%) OR* 95% C.I. Intraoperative Phase Brieng 0-2† 258 56 (20) 0.94 (0.40-2.17) 3-4‡ 35 (12) 8 (23) referent I nformation sharing 0-2† 76 26 (34) 2.45 ( 1.36-4.42) 3-4‡ 217 (74) 38 (18) referent Inquiry 0-2† 145 34 (23) 1.20 ( 0.69-2.10) 3-4‡ 147 (50) 30 (20) referent Vigilance 0-2† 89 23 (26) 1.39 (0.77-2.49) 3-4‡ 204 (70) 41 (80) referent An Empiric Study of Surgical Team Behaviours and Patient Outcomes 271 Table 16.5 (Mazzocco et al. 2008) shows the results of the logistic regression models using the BMRI and ASA as predictors and surgical outcome as the dependent variable. Odds ratios greater than 1.0 indicate an association of less frequently observed ‘good’ behaviour with poorer outcome. The BMRI was signicantly associated with any complication or death after adjusting for ASA score (adjusted OR 4.82, 95 percent CI 1.30, 17.87). In other words, when teamwork behaviours were relatively infrequent during surgical procedures, patients were more likely to experience death or a major complication. Unadjusted Odds Ratio 95% C.I. on the unadjusted OR p- value (W ald test) Adjusted# Odds Ratio 95% C.I. on the adjusted OR p- value* (Wald test) Risk factor BM RI 5.61 1.53, 20.54 0.009 4.82 1.30, 17.87 0.019 ASA 1.59 1.06, 2.38 0.024 1.51 1.00, 2.27 0.049 Table 16.5 The association of the Behavioural Marker Risk Index with post-operative complications and death N % of Total n (%) OR* 95% C.I. Handoff Phase Brieng 0-2† 54 19 (35) 2.34 (1.23-4.46) 3-4‡ 239 (82) 45 (19) referent I nformation sharing 0-2† 59 20 (34) 2.21 (1.18-4.16) 3-4‡ 234 (80) 44 (19) referent Inquiry 0-2† 175 43 (25) 1.50 (0.84-2.70) 3-4‡ 118 (40) 21 (18) referent Vigilance 0-2† 84 18 (21) 0.97 (0.52-1.79) 3-4‡ 209 (71) 46 (22) referent * Odds ratio for a major or minor complication or death in teams with score of 0–2 for markers of team behavior relative to score of 3-4 for markers of team behaviors † scores of 0-2 indicate that markers of ‘good’ team behavior were never or rarely observed or there was isolated or minimal observation of the behaviors ‡ scores of 3-4 indicate that markers of ‘good’ team behavior were observed often or always Table 16.4 Concluded Safer Surgery 272 Figure 16.1 (Mazzocco et al. 2008) graphically shows the positive association between the BMRI (with a higher score indicating fewer instances of teamwork behaviour) and poorer patient outcome as predicted by our logistic regression model. Discussion Principal Findings and Conclusions from the Published Study We found that patients whose surgical teams exhibited less teamwork behaviours were at higher risk for death or complications, even after adjusting for ASA risk category. We believed that was an important addition to the international conversation on teamwork in healthcare, providing quantitative evidence of a direct link between teamwork during the surgical case and subsequent patient outcome. This discussion reiterates the strengths and limitations of the prior study (Mazzocco et al. 2008) and expands our previous publication by an in-depth discussion of previous research and by describing team training programmes that followed this study. Figure 16.1 The predicted relationship between Behavioral Marker Risk Index and post-operative complications and death 1.0 1.8 2.7 3.5 4.3 5.2 6.0 0.0 0.2 0.4 0.6 0.8 1.0 Risk Index Adjusted ORs An Empiric Study of Surgical Team Behaviours and Patient Outcomes 273 Strengths and Limitations Our study had several strengths. It was conducted in a community setting that is likely to be representative of surgical procedures. A variety of procedures were observed and the teams were diverse. The outcomes were ascertained with the reviewer blinded to the team behaviour scores. Behavioural markers have been applied to healthcare settings such as neonatal resuscitation (Thomas et al. 2006), and this study builds on that work. We modied the behavioural markers and the observation tool to apply to the operating room environment and used the same calibration techniques for our nurse observers as those used in prior studies. Continuous communication among the observers throughout the study ensured a sustained level of inter-rater reliability. The study has some important limitations. First, the study was observational and we did not establish a cause and effect relationship between good team behaviour and better outcome. Second, it is not possible to conclude which behaviours are most important or whether their inuence varies by operative stage (induction, etc.). Developing an intervention solely based on these ndings would not be straightforward. Third, to obtain cooperation in conducting the study, we had to protect the identities of the members of the team and we were thus not able to describe team characteristics (e.g., training, experience) in detail. Research, including an extensive qualitative analysis based on observer comments, is ongoing with these data. Fourth, some of our analyses, notably our grouping of the outcomes into a dichotomous variable, were conducted post-hoc. Comparisons to Other Research Previous studies of operating room teams have focused on characteristics of surgeons such as ‘individual excellence’ (McDonald et al. 1995) and technical competence (Gawande et al. 2003). They have also examined the impact of major and minor human failures upon patient outcomes; Carthey et al. (2003) conducted qualitative analyses of major system features that inuence team performance and patient safety (Davenport et al. 2007, Greenberg et al. 2007) and performed retrospective reviews of malpractice claims les (Gawande et al. 2003). Our methods and results complement and extend this literature in several ways. For example, we used direct observation of procedures and then used different study personnel to prospectively collect patient outcome data. This addresses limitations of malpractice claims le analyses such as hindsight bias (knowledge of the bad outcome can bias reviewers to rate teamwork as lower) and sole reliance on the documents in claims les to make judgements about complicated and dynamic team behaviours. Compared to Carthey et al (2003) we studied a more generalizable and common group of surgical procedures, thus extending their ndings to other types of surgeries. Greenberg et al. studied the entire spectrum of surgical care, not just intra-operative care, and identied communication breakdowns during surgeon communication with other caregivers (Greenberg et al. 2007). They . score) Safer Surgery 266 medium risk included open laparotomy, carotid endarterectomy and thyroidectomy; and high risk included aortic aneurysm repair and femoral popliteal bypass. The 30- day. Safer Surgery 264 Adjustment Variables and Outcomes The American Society of Anesthesiologists (ASA). anaesthetic during surgery; consult requested in Post Anaesthetic Care Unit (PACU); path report normal or unrelated to diagnosis; and insertion of arterial or central venous line during surgery. );