http://dx.doi.org/10.1016/j.rca.2014.09.006

Scientific and Technological Research

Effectiveness of a program for improving teamwork in Operanting Rooms^{1, 2}

Efectividad de un programa para mejorar el trabajo en equipo en salas de cirugía

Ana Carolina Amaya-Añas^a,*, Douglas Idarraga^b, Vanessa Giraldo^c, Luz María Gómez^c

^a Psychologist, MSc Clinical Epidemiology, Head of Promotion y Prevention, Sociedad Colombiana de Anestesiología y Reanimación (S.C.A.R.E.), Bogotá, Colombia
^b Psychologist, Specialist in Clinical Psychology, Medical Student at the Universidad Industrial de Santander, Bucaramanga, Colombia
^c Anthropologist, MSc Public Health, Sociedad Colombiana de Anestesiología y Reanimación (S.C.A.R.E.), Bogotá, Colombia
^d MD, Anesthesiologist, MSc Clinical Epidemiology, Scientific Vice-Director, Sociedad Colombiana de Anestesiología y Reanimación (S.C.A.R.E.), Bogotá, Colombia

¹ Please cite this article as: Arias ACA, Idarraga D, Giraldo V, Gómez LM. Efectividad de un programa para mejorar el trabajo en equipo en salas de cirugía. Rev Colomb Anestesiol. 2015;43:68-75.
² Departments and Institutions to recognize the work: Sociedad Colombiana de Anestesiología y Reanimación, S.C.A.R.E.

* Corresponding author at: Sociedad Colombiana de Anestesiología y Reanimación, S.C.A.R.E. Correspondencia: Cra 15 A N° 120 - 74, Bogotá, Colombia.
E-mail addresses: a.amaya@scare.org.co, acamayaa@unal.edu.co (A.C. Amaya-Arias), douglasidarraga@gmail.com (D. Idarraga), vgiraldoga@gmail.com (V. Giraldo), lm.gomez@scare.org.co (L.M. Gómez).

Article history: Received 2 August 2014 Accepted 29 September 2014 Available online 20 November 2014

Abstract

Introduction: The purpose of the trial was to establish whether a training program could improve teamwork in the Operating Rooms and the Obstetric Suite at a healthcare institution.

Method: Quasi-experimental before and after study - A multiphase and multi-method training program was implemented for improving teamwork, measured with the OTAS-S instrument in 40 surgical procedures, pre and post intervention. Training was given to 80% of the population. The descriptive analysis was abased on the data collected from each stage and the effectiveness was determined based on the comparison of the scores obtained.

Results: Post-intervention measurements revealed differences in the scores obtained by the teamwork between the two stages (MW U-test; z = 48.879, P = .0000). This effectiveness was shown in the surgical and obstetrics area. Two or more points of improvement in the average OTAS-S scores was identified in every phase, behaviors and sub-teams.

Conclusion: Intervention was effective for improving teamwork in the operating rooms and in the obstetrics suites in the short term. Further research is suggested including a control group and a more extended follow-up to establish long-term effectiveness.

Keywords: Patient Safety Operating Rooms Obstetrics General Surgery Work

Resumen

Introducción: El objetivo fue determinar si un programa de entrenamiento podía mejorar el trabajo en equipo en las salas de cirugía y obstetricia de una institución de salud.

Método: Estudio cuasi experimental de antes y después. Se aplicó un programa de entrenamiento multi-fase y multi-método para mejorar el trabajo en equipo, el cual se midió aplicando el instrumento OTAS-S en 40 procedimientos quirúrgicos en las etapas pre y post intervención. La capacitación fue dada al 80% de la población. Se realizaron análisis descriptivos de los datos obtenidos en cada etapa y se determinó la efectividad a partir de la comparación de los puntajes obtenidos.

Resultados: La medición post-intervención reveló diferencias en los puntajes obtenidos en el trabajo en equipo entre ambas etapas (MW U-test; z = 48.879, P = .0000). Esta efectividad se dio en el área de cirugía y obstetricia. Todas las fases, comportamientos y sub-equipos presentaron mejora de dos o más puntos en los puntajes promedios del OTAS-S.

Conclusión: La intervención fue efectiva para mejorar el trabajo en equipo en las unidades de cirugía y obstetricia a corto plazo, se sugiere realizar investigación futura que incluya grupo de control y hacer seguimiento más prolongado en el tiempo para determinar efectividad a largo plazo.

Palabras clave: Seguridad del Paciente, Quirófanos, Obstetricia, Cirugía General, Trabajo.

Introduction

Poor communication and teamwork deficiencies have been identified as contributing factors to medical errors or adverse events¹; some trials indicate that poor communication among the team is responsible for about 20% of the mistakes in healthcare in our country.² Morey et al.³ found that the implementation of initiatives for improving teamwork reduced the number of events from 30% to 4.4%. However, quality evidence is not yet available to support these findings or to determine which of these intervention plans are the major contributors to the reduction of those mistakes.^4,5

One of the most popular initiatives to improve teamwork is Team Stepps based on some of the Crew Resource Management (CRM) resources developed for aviation after the 80s, acknowledging the importance of non-technical skills in the case of mistakes leading to accidents. The training includes the development of collective intelligence, understood as the active ability of the team members to learn, teach, communicate, reason and think together, regardless of their position in the hierarchy, focused on shared objectives and a mission.⁶

There are different instruments for measuring teamwork in the OR. Some of these are the Anesthetists Non-Technical Skills (ANTS),⁷ the Non-Technical Skills (NOTECHS),⁸ the Non-Technical Surgical Skills for Surgeons (NOTSS)⁹ and the Observational Teamwork Assessment for Surgery (OTAS).¹⁰ Only the latter has a validated Spanish version.

This paper intends to answer the question: What is the effectiveness of a training program for improving teamwork skills of staff members working in surgery and obstetrics at a healthcare institution in Colombia?

Method

Quasi experimental before and after study - A multiphase and multi-method training program was implemented based on the TeamSTEPPS® model,^11,12 in addition to the CRM (Crew Resource Management)¹³ intervention models, and on the motivational theories of health psychology.¹⁴

The intervention included the following components (Fig. 1):

(1) 4-h workshop: Previously prepared audiovisual material was used (PowerPoint slides, awareness video and videos of demonstrative cases). The topics included: Patient Safety Systemic Model, teamwork, non-technical skills (communication, cooperation, coordination, leadership and situational awareness), Time-out and effective use of the WHO checklist.

(2) Five virtual modules were uploaded to a Moodle® platform on a weekly basis. The topics corresponded to an in depth analysis of the workshop discussions.

(3) Time-Out training and correct use of the WHO surgical checklist. By consensus with the surgical, obstetrics, safety and quality leaders, an adaptation was made to the WHO checklist and then the professionals from both areas were trained in its application. Additionally, training was given on the use of Briefing and Debriefing meetings, Time-out and Sign-out procedures that are consistent with the surgical checklist. However, in our case there is a difference because the Time-Out was done prior to the induction of anesthesia and not prior to the incision and was not limited to the items on the list, but included the planning and evaluation of every aspect in the procedure.

(4) Institutional Actions - Meetings aimed at getting the Institution's commitment to the change process and with the strategies needed to maintain the improvement processes in the long term.

Procedure

Stage 1. Baseline

An initial OTAS-S-based teamwork measurement was made.^15,16 A previously trained psychologist administered the instrument, following the procedure established in the instrument's handbook.

The sample size for paired data was estimated, resulting in a minimum sample size of 16 pairs, for a minimum detectable difference of 1.5 points in the pre and post intervention stages, a standard deviation of 1.2, a 95% confidence interval and 80% power.

Stage 2. Application of the intervention

The implementation of the intervention lasted two months. The physical workshops were repeated for three weeks, and each staff member attended only once. The on-line workshops began once the physical workshops were over. Training in time-out was done directly in the ORs.

Stage 3. Measurement of effectiveness

One week after the intervention was completed, a post-measurement was done using the OTAS-S instrument again.

Instruments

The OTAS-S measures five teamwork dimensions: communication, coordination, cooperation/support, leadership and supervision/awareness of the situation. It is rated with a 7-point Kinkert scale; it measures the performance of 4 sub-teams - anesthesia, surgery, nursing and instrumentation - in one of the three surgical phases (pre, intra and postoperative).^15,16

The participants evaluated the workshop responding to statements about three topics: contents, quality of delivery as a whole and overall satisfaction. This evaluation format was adapted from Hull et al.¹⁷ Each statement was rated on a 5-point Likert scale (1 = totally disagree, 5 = totally agree).

Results

Stage 1. Baseline

Teamwork was measured in 40 surgical procedures in total, distributed by units, as shown on Table 1.

The minimum number planed for each unit sample was met; in the surgical unit 8 procedures beyond the number planned were observed.

The average surgical time was 51 min (SD = 26min), with a minimum of 18 and a maximum of 352 min. The teams were made up by an average of 7 people ±2; the smallest team size was 3 people and the maximum 11. With regard to the procedures observed, 94% of the obstetric surgeries were cesarean sections, while in the surgical unit there were general procedures and specialized surgeries such as orthopedics, plastic surgery and gynecological surgeries.

Table 2 shows the total and per unit results prior to the intervention.

The test scores range from 0 to 6, where 0 indicates a problem behavior and a serious hurdle for the team's performance and 6 indicates an exemplary behavior, extremely effective for improving teamwork performance, while 3 means that the performance of the team does not improve or deteriorate by the particular behavior. The results show that the teams had a baseline neutral behavior.

Furthermore, no significant difference was noted in the units (MW U-test; z = -1.781, P = .075).

Table 3 presents the results initially obtained based on behavior, sub-team and phase.

Significant statistical differences by phase were identified (KW-test; Chi-square = 338.138, P =.000), in terms of behavior (KW-test; Chi-square = 43.978, P = .000) and in terms of sub-team (KW-test; Chi-square= 63.702, P=.000). The behaviors with the highest opportunity for improvement were leadership and situational awareness; overall however, all of the behaviors could be improved with improved teamwork. As a whole, the nursing team showed the best teamwork scores, as well as the pre-operative phase.

Stage 2. Administration ofthe intervention

18 workshops were organized to train 186 staff members in person, from the areas of intervention (76% of the staff). The results of the evaluation made by the participants are illustrated on Table 4.

In all of the areas evaluated, the workshop was considered very positive (range 4.90-4.97 on a 5-point scale), so it can be considered a valuable tool to create awareness and educate the staff on the importance of teamwork.

The design included 5 modules that were uploaded to S.C.A.R.E.'s central platform "Siempre Educando" (Constantly Educating). There were 134 staff members participating (86% of the workshop participants); the groups with the lowest participation were surgeons and anesthesiologist. The major difficulty experienced was poor knowledge of some staff members about how to use these platforms and this required establishing fixed timetables to provide assistance and hands-on training.

Training on the correct use of the WHO checklist and the implementation of time-out extended over four weeks. 73 interventions were assessed with 47 different teams trained, including 95% of the nurses and scrub nurses, 80% of the anesthesiologists and approximately 70% of the specialized surgeons.

Stage 3. Effectiveness measurement

For the second measurement OTAS-S was used again in 40 procedures, following the same percentages as the preliminary stage (60% surgery, 40% obstetrics).

The surgical time in the second round was 65 min in average (SD = 32 min), range 29 through 182 min. In average, the teams consisted of 8 people, with a standard deviation of 1, ranging from 5 to 10.

Measurement of the post-intervention stage revealed differences in the teamwork scores, as illustrated on Tables 5 and 6.

The final average score was 5.3, representing considerable improvement in the team's performance.

All stages, behaviors and sub-teams showed improvements of two or more points in the average scores obtained. Significant differences were identified in every comparison: total score, by unit, surgical phase, behavior, and sub-teams (surgery and obstetrics) all showed a considerable and similar improvement (post means: 5.4-5.2).

Discussion

This trial is considered a forerunner in the development of human factors-related research and patient safety in our country. As stated by Martin et al.¹⁸ "... following the publication "Errar es humano" (To err is human) from the Institute of Medicine in 1999, there have been numerous grants awarded for research, publications, editorials, letters and reviews on the topic of Human Factors in medicine"; however, no trials have yet been completed in Colombia showing the effectiveness of interventions to improve non-technical skills among healthcare surgical practitioners. The aim of the study was to establish the effectiveness of a training program for enhancing team skills of the surgical and obstetrics staff at a healthcare institution in Colombia.

The results show that the baseline team measurements indicated a mild trend toward teamwork, but if failed to impact patient safety. The behaviors with higher probability of improvement were leadership and situational awareness, but as a whole, every one of them could be improved toward enhanced teamwork. As a whole, the nursing team showed higher teamwork scores, as well as the preoperative phase. The post-operative period showed the lowest teamwork behaviors, with an average score of 2, indicating a mild disruption in the team's performance, whether due to omissions or inadequate behavior. This result could be associated with the lack of a debriefing of the surgical process since some of the team members often left the OR even before the patient had been transferred to the gurney and delivered to the recovery room.

One of the key surgical variables to reduce the number of medical errors is on-the-job training, in addition to strategies and team roles.¹⁹ Teamwork includes technical skills such as handling and use of surgical equipment and non-technical skills such as communication, decision making and shared situational awareness.²⁰ Survey analysis and observations, as well as near misses and adverse events, have evidenced the key role of miscommunication among the team members.

Likewise, the analysis of communications and healthcare team work has enabled the identification of recurrent issues due to cultural barriers and a rigid hierarchical structure.²¹

Overall, the final scores measured in this trial in the various groups show teamwork-oriented behaviors after the implementation of the intervention program. The one to two points difference in the pre and post-intervention indicates that the intervention was effective to improve team work in the surgical and obstetric units at the institution in the short term.

Some of the limitations of this trial were the quasi-experimental method with just one institution involved and the lack of a control group. The pre and post-test designs limit the ability of the researchers to identify causal relationships, although the results may be statistically significant.²¹

Literature reviews have been made on the relationship between training in non-technical skills and technical performance and the clinical results. One review revealed that poor quality of the trials was due to masking issues, subjective measurements and potential Hawthorne effect. The results reported were improved attitude, improved teamwork, improved technical performance, efficiency or decline in the number of errors. None of the randomized clinical trials found any evidence of technical or clinical improvement.²¹

Another review considered 12 trials where significant changes in the team practices were reported. The training strategies were: Briefing-debriefing, WHO checklists, in-person workshops and simulation environments. These results suggest that these types of training help to improve communications and group unity. However, no obvious differences were found in terms of the number of errors or incidents.²⁰

The results of our investigation are based on the observations on teamwork and a survey on intervention satisfaction. Measurements to assess the clinical effect, as well as the drop in adverse events or improved performance of the technical team are lacking. A measurement was made one week after the intervention but longer follow-up is required to ensure the stability of the changes identified.

The strengths of the trial were meeting the requirement of the minimum sample size per unit, the use of observational instruments adapted and validated to our environment and the implementation of an evidence-based program.

The intervention lasted for two months and 186 staff members were trained in the areas of intervention (76% of the staff). To comply with the interdisciplinary nature of the team as suggested by the TeamSTEPPS program, the participants included physicians, nurses, scrub nurses and support staff of the institution.²²

Additionally, the CRM intervention models were taken into account.¹³ Besides urging the participants to acknowledge the importance of a safety culture and teamwork, tools were used to standardize procedures, ensure effective communication, and improve awareness of all the environmental variables.

One of these tools in particular was the use of the checklists introduced in aviation back in 1935. Its recent use in surgery has proven to reduce the number of complications by 36% and deaths by 47%. The implementation of the WHO checklist has shown to influence the perception about a safety culture. At a University Hospital in Norway, a limited impact on safety culture was identified, with a 77% checklist compliance and 85% positive changes in the frequency of reported events and improvement plans.²³ A research done in Colombia found a reduction in the number of reported surgical adverse events, following the implementation of the checklist (7.26% in 2009 vs. 3.29% in 2010) and a perception of enhanced quality and patient satisfaction.²⁴ These findings show the importance of adopting the checklist as a routine in healthcare centers in our country; however, as Gómez (2013) said "... the implementation of the checklist is not an easy task and requires leadership, teamwork, flexibility and adaptation to change, both by the institution itself and by the healthcare professionals".²⁵ So the recommendation when adopting the checklist is to simultaneously introduce programs to enhance these types of non-technical skills such as the one herein discussed.

Some additional strategies that can be combined with the use of the checklist are the Time-Out and Briefings and Debriefings that were also used in this intervention. These are pre and post-surgery team discussions that usually last from 3 to 5 min following a checklist. Prior to the procedure, the list shall include: (1) introduction of team members, (2) goals of surgery, (3) clarification of roles, (4) review of the procedure, and (5) identification of potential hazards. At the end of the procedure, the list includes: (1) checking and counting gausses, needles and instruments, (2) confirmation and labeling of surgical specimens, (3) agreement on post-operative care, and (4) discussion around the team's performance: positive and negative aspects and improvement strategies.

Before the program the post-operative was the surgical phase with fewer teamwork behaviors, associated with not having a debriefing, including transferring the patient and hand-off to the recovery suite. The phase with the highest post-intervention score was the pre-operative phase, but there were significant differences between the before and after measurements for all the surgical phases, including the post-op.

The available evidence about the technical or clinical benefit of teamwork training interventions is weak. The studies, which are mainly descriptive, are of poor quality and exhibit flaws in the psychometric characteristics of the instruments. Many trials base their results on the change in perceptions and attitudes, but this does not guarantee an effective behavioral change.

Studies have focused on communication and teamwork using tools such as briefings-debriefings and checklists. However, there is no information in the literature about other factors such as shared situational awareness, leadership, management of workload and decision-making. There is also a lack of follow-up to ensure the stability of the changes identified, with post-intervention measures for up to 12-18 months.

Most research focuses on surgeons and anesthesiologists, but the other professionals involved in the team, including nurses, scrub nurses, etc., have not been taken into account, or their results have not been frequently reported. In contrast, the participation of surgeons and anesthesiologists was minimal in this trial. Probably as a result of a cultural bias, a strong hierarchy characterizes the medical environment. This is quite concerning because the performance of the surgical team depends on the successful interaction of many people from diverse disciplines that work together in the OR.²⁶

The suggestion is to use a more stringent methodology in future research projects, with the participation of a larger number of institutions, including a control group, the use of validated instruments with evidence in our environment, both for observation and for measurement of the impact on attitudes, team satisfaction and other indicators; finally, a longer follow-up would be appropriate.

Funding

The financing was done by Colombian Society of Anesthesiology (S.C.A.R.E.).

Conflict of interest

The authors Ana Carolina Amaya and María Luz Gómez are currently working in the Deputy Medical Advisory Specialized S.C.A.R.E.

References

1. Amaya Arias AC, Narvaéz R, Eslava-Schmalbach JH. Trabajo en equipo como factor contribuyente en la ocurrencia de errores médicos o eventos adversos. Rev Colomb Cir. 2013;28:297-310.         [ Links ]

2. Gaitán-Duarte H, Eslava-Schmalbach J, Rodríguez-Malagon N, Forero-Supelano V, Santofimio-Sierra D, Altahona H. Incidencia y Evitabilidad de Eventos Adversos en Pacientes Hospitalizados en tres Instituciones Hospitalarias en Colombia, 2006. Rev Salud Pública. 2008;10:215-26.         [ Links ]

3. Morey J, Simon R, Jay G, Jay GD, Wears RL, Salisbury M, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37:1553-81.         [ Links ]

4. Ahmed K, Khan N, Anderson D, Watkiss J, Challacombe B, Khan M, et al. The productive operating theatre (TPOT) programme - is it effective? J Urol. 2012;187:e112.         [ Links ]

5. Wauben LS, Dekker-van Doorn CM, van Wijngaarden JD, Goossens RH, Huijsman R, Klein J, et al. Discrepant perceptions of communication, teamwork and situation awareness among surgical team members. Int J Qual Health Care. 2011;23:159-66.         [ Links ]

6. Gordon S, Mendenhall P, O'Connor BB. Beyond the checklist: what else health care can learn from aviation teamwork and safety. Ithaca, NY: Cornell University Press; 2012.         [ Links ]

7. Flin R, Patey R. Non-technical skills for anaesthetists: developing and applying ANTS. Best Pract Res Clin Anesthesiol. 2011;25:215-27.         [ Links ]

8. Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS system: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care. 2009;18:104-8.         [ Links ]

9. Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg. 2008;32:548-56.         [ Links ]

10. Hull L, Arora S, Kassab E, Kneebone R, Sevdalis N. Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg. 2011;212:234-43, e231-235.         [ Links ]

11. Forse RA, Bramble JD, McQuillan R. Team training can improve operating room performance. Surgery. 2011;150:771-8.         [ Links ]

12. Mayer CM, Cluff L, Lin WT, Willis TS, Stafford RE, Williams C, et al. Evaluating efforts to optimize TeamSTEPPS implementation in surgical and pediatric intensive care units. Jt Commun J Qual Patient Saf. 2011;37:365-74.         [ Links ]

13. Grogan EL, Stiles RA, France DJ, Speroff T, Morris JA Jr, Nixon B, et al. The impact of aviation-based teamwork training on the attitudes of health-care professionals. J Am Coll Surg. 2004;199:843-8.         [ Links ]

14. Flórez L. Psicología Social de la Salud: Promoción y Prevención. Bogotá: Manual Moderno; 2007.         [ Links ]

15. Amaya Arias AC, Eslava-Schmalbach J, Barajas R, Gaitán-Duarte H, Hull L, Wheelock A, et al. Psychometric properties of Observational Teamwork Assessment for Surgery (OTAS) for observational assess teamwork in Colombian surgical teams. Seoul, Korea: 10th Health Technology Assessment international (HTAi); 2013.         [ Links ]

16. Amaya Arias AC, Barajas R, Eslava-Schmalbach J, Wheelock A, Gaitán-Duarte H, Hull L, et al. Translation, cultural adaptation and content re-validation of the Observational Teamwork Assessment for Surgery tool. Int J Surg. 2014, http://dx.doi.org/10.10167j.ijsu.2014.10.001 [in press]         [ Links ].

17. Hull L, Arora S, Amaya Arias AC, Wheelock A, Gaitán- Duarte H, Vincent C, et al. Building and strengthening global capacity for patient safety: a training programme incorporating the WHO core competencies for patient safety research. Int J Surg. 2012;10:493-9.         [ Links ]

18. Martin LD, Rampersad SE, Low DKW, Reed MA. Mejoramiento de los procesos en el quirófano mediante la aplicación de la metodología Lean de Toyota. Rev Colomb Anestesiol. 2014;42:220-8, http://dx.doi.org/10.10167j.rca.2014.02.007.         [ Links ]

19. Bagnulo H, Barbato M, Godino M, Basso J. Evaluación del riesgo en eventos adversos. Rev Med Urug. 2010;26:55-7.         [ Links ]

20. Gillespie BM, Chaboyer W, Murray P. Enhancing communication in surgery through team training interventions: a systematic literature F review. AORN J. 2010;92:642-57.         [ Links ]

21. McCulloch P, Rathbone J, Catchpole K. Interventions to improve teamwork and communications among healthcare staff. Br J Surg. 2011;98:469-79.         [ Links ]

22. Hull L, Arora S, Aggarwal R, Darzi A, Vincent C, Sevdalis N. The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg. 2012;214:214-30.         [ Links ]

23. Haugen AS, S0fteland E, Eide GE, Sevdalis N, Vincent CA, Nortvedt MW, et al. Impact of the World Health Organization's Surgical Safety Checklist on safety culture in the operating theatre: a controlled intervention study. Br J Anaesth. 2013;110:807-15.         [ Links ]

24. Collazos C, Bermudez L, Quintero A, Quintero L, Díaz M. Checklist verification for surgery safety from the patient's perspective. Rev Colomb Anestesiol.         [ Links ]

25. Gómez Buitrago LM. La lista de chequeo: un estándar de cuidado. Rev Colomb Anestesiol. 2013;41:182-3, http://dx.doi.org/10.1016/j.rca.2013.07.001.         [ Links ]

26. Vertino KA. Evaluation of a TeamSTEPPS© Initiative on Staff Attitudes Toward Teamwork. J Nurs Adm. 2014;44: 97-102.         [ Links ]