text in 
English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
What do we know about this issue?
Deficits in situational awareness (SA) account for 82% of human error-related adverse events in anesthesiology.
Despite its importance, SA training is informal, inconsistent, and is not formally integrated into anesthesiology programs.
While SA training has been shown to improve performance in other fields, there is no consensus on how to effectively teach it in anesthesiology.
What does this study contribute?
It identifies SA training needs for residents based on their year, in line with core competencies relevant to any anesthesiology program.
It uses eye-tracking glasses as well as interviews to identify SA gaps.
It suggests that a formal SA training program could enhance resident education and improve patient safety.
INTRODUCTION
Human errors are a leading cause of adverse events in healthcare 1, with SA identified as the predominant factor 2. SA is defined as "the perception of elements in the environment within a volume of time and space (level 1), the comprehension of their meaning (level 2), and the projection of their status in the near future (level 3)" 3. Notably, the loss of SA in anesthesiology accounts for approximately 82% of human error-related incidents 4.
In anesthesiology, SA involves identifying critical patient data (perception), interpreting them in context (comprehension), and anticipating complications (projection). These levels align with the training needs of the residents as they progressively develop SA skills. As one of the categories of the Anesthetists' Non-Technical Skills (ANTS), SA also influences the other three, namely, decision-making, task management, and teamwork 5. It precedes decision-making, where comprehension and projection enable effective choices in high-stakes environments 6, while task management relies on comprehension for prioritization and planning, and teamwork is supported by the communication and coordination elements of the SA model 7 in maintaining shared mental models. This study focuses exclusively on SA as a critical component of resident training.
SA in anesthesiology has traditionally been learned informally through clinical practice, with limited curricular inclusion 8. Evidence from other fields suggests that SA training enhances individual and team performance 9. However, anesthesiology lacks a structured SA training approach, emphasizing the need to analyze residents' training requirements. Our aim was to conduct a Training Needs Analysis (TNA) to identify SA training requirements for anesthesiology residents, and to stablish a framework for effective training strategies.
METHODS
Study design
This is a qualitative exploratory study with an observational and descriptive approach designed to identify and define SA training needs for anesthesiology residents. This approach enabled an in-depth understanding of SA deficiencies through direct observation, interviews, and focus groups, ensuring comprehensive identification of training needs.
A Training Needs Analysis (TNA) was applied based on the McGehee and Thayer model, from a three-level perspective: the organization, the task and the individual 10. Conducted within a post-positivist paradigm, this study sought to identify objective patterns in SA errors among residents by observing and classifying observable undesired events (OUEs) using a SA error taxonomy 11. The ultimate goal was to identify SA training needs.
Ethical approval
The study was approved by the research and ethics committees of the Engineering (FID 190, July 2, 2019) and Medicine (FM-CIE-0514-19, August 23, 2019) schools at Pontificia Universidad Javeriana Data collection was affected by COVID-19 restrictions, which limited research activities. However, these challenges resulted in a more thorough analysis and alignment with the existing literature, enhancing the validity and relevance of the findings.
The research adhered to the principles of the Declaration of Helsinki 12. Data anonymity was ensured, and informed consents were obtained from all participants, including residents, faculty, patients, companions, and healthcare team members. No incentives were provided.
Study population: 1. Faculty members: All 36 faculty members from the Department of Anesthesiology at Pontificia Universidad Javeriana were invited; 10 - each with at least 10 years of experience supervising residents - participated in focus groups. Additionally, the four faculty members who supervised the observed surgical procedures participated in the retrospective interviews. 2. Residents: The five anesthesiology residents from the Hospital Universitario San Ignacio who participated were grouped by residency year (first, second, and third year) according to the complexity level of the observed surgical procedures, and later took part in the retrospective interviews.
Inclusion and exclusion criteria. We included faculty members with >10 years of experience supervising the residents who participated in the focus groups. Only those involved in the observation phase of the surgical procedures were included in the retrospective interviews. First-, second-, and third-year anesthesiology residents responsible for the observed surgeries participated in both procedure observations as well as retrospective interviews. Faculty members or residents who did not provide informed consent, lacked the required experience, or were not responsible for the observed surgical procedures were excluded.
Data collection
Data collection was organized into three main phases:
Phase 1 - Focus groups: Two structured focus groups were conducted with faculty members, each lasting ~60 minutes. The first included six faculty members, while the second, conducted as a continuation, included seven. Due to scheduling constraints, some attended both sessions, while others participated in only one.
Faculty members were selected through convenience sampling, considering their availability and ≥10 years of experience supervising anesthesiology residents. A standardized discussion guide explored SA competencies at different training stages. Faculty described common SA loss scenarios among residents, and responses were recorded on sticky notes, later organized into themes using affinity diagrams 13.
Phase 2 - Observation of surgical procedures: Non-participant observation was carried out in five surgical procedures at Hospital Universitario San Ignacio. The sample was selected intentionally using a non-probabilistic approach, grouping anesthesiology residents by year and surgery complexity: R1 (first-year residents): routine, low-complexity procedures; R2 (second-year residents): neurosurgeries; R3 (third-year residents): high-complexity cardiac surgeries. During the induction phase, residents wore eye-tracking glasses (Tobii Pro Glasses 2, Tobii, Sweden) to record visual behavior (SA Level 1) and interactions (SA Level 3).
Phase 3 - Retrospective interviews: Retrospective semi-structured interviews were conducted with four residents and five faculty members involved in the observed procedures. A predefined question set ensured consistency while allowing flexibility to explore individual experiences. A protocol was developed to elicit mental models in the residents and assess their comprehension using eye-tracking video segments from each OUE.
To maintain a psychologically safe learning environment, sessions were framed as explorations rather than evaluations, emphasizing errors as learning opportunities. The investigators facilitated structured reflections, promoting self-awareness and critical thinking instead of providing corrective feedback. Open discussions were encouraged through guiding questions, avoiding direct corrections.
The protocol included prompts such as: I'm going to show you a series of videos of case situations that I would like to discuss with you: a. What do you think was the issue during this episode? b. What do you think were the causes of this issue? c. If you had to teach an intern how to perform this task step by step, how would you do it? d. In which step do you think you had difficulties, and why? e. Do you think you could break down that step further to teach it to an intern?
For faculty members, interviews incorporated eye-tracking video segments of OUEs, allowing them to identify error frequency, analyze OUE causes, determine expected actions, and propose improvement strategies
Data analysis
Focus group notes were organized using affinity diagrams to identify the most demanding phases, procedures with common errors, and key SA skills for each residency year. The research team, comprising one anesthesiologist with a doctorate in medical education, and three experts in human factors and ergonomics (two with engineering doctorates), reviewed emerging themes to ensure coherence and representativeness.
The concept of "Observable Undesired Events (OUEs)" was introduced to identify situations where resident performance did not meet the expected standards, thereby compromising patient safety as a result of SA deficiencies. Two anesthesia residents were trained to identify and code OUEs in the videos using the SA error taxonomy 11. A peer review strategy 14 where researchers discussed and verified the results was applied.
Interviews were transcribed and responses were compared in Excel to identify differences and similarities in participant perceptions. Additionally, narrated experiences were recorded to support findings. Data analysis followed a mixed inductive-deductive approach, supported by investigator triangulation 14. Interviewee validation 15 ensured accuracy by comparing researcher interpretations with participant perspectives. Thematic analysis 16, guided by the SA error taxonomy 11, triangulated findings from eye-tracking analysis, interview data, and faculty assessments to validate SA deficiencies and identify gaps between the residents' SA skill levels and the expected competencies.
Multiple strategies were employed to minimize bias. Selection bias: only faculty and residents involved in observed cases participated, ensuring relevance. Observer bias: triangulation was used, incorporating eye-tracking, interviews, and focus groups. Analysis bias: a peer review process ensured two researchers independently coded OUEs before reaching consensus. Confirmation bias: interviewee validation compared researcher interpretations with participant perspectives, reinforcing the triangulation process.
RESULTS
Integrating findings from the three levels of analysis provided a comprehensive view of SA training needs. The results were discussed with anesthesiology program coordinators to design targeted educational interventions addressing identified gaps, with the ultimate goal of improving safety and clinical efficacy.
Phase 1 - Focus groups and affinity diagrams
During focus groups, the anesthesiology faculty identified anesthetic induction as one of the most critical task phases. This phase involves essential tasks such as medication administration and vital sign monitoring, and poses challenges derived from patients' unpredictable responses, increasing the risk of adverse events. Consequently, it was prioritized in assessing training needs.
Faculty defined the SA skills that residents must develop at each training level, in line with Endsley's SA model (Perception, Comprehension, Projection). These cognitive abilities are essential for safe anesthesia practice. (See Table 1.)
Table 1 Core situation awareness (SA) skills expected in anesthesiology residents by year of training.
| Year of Residency | SA Level | Required SA Skills |
|---|---|---|
| R1 (First Year) | Perception | Attention management during signal monitoring (e.g., vital signs, alarms) |
| Comprehension | Development of mental models (e.g., mental models for anesthetic induction | |
| Projection | Strategies to cope with environmental pressures (e.g., systematic scans, alarm recognition) | |
| R2 (Second Year) | Perception | Attention management during manual tasks (e.g., nerve blocks and arterial line insertion) |
| R3 (Third Year) | Projection | Contingency planning and crisis management (e.g., excessive bleeding, pump malfunction) |
Source: Authors.
Phase 2 - Observation and identification of OUEs in Eye-tracking videos
Across five observed procedures, 14 Observable Undesired Events (OUEs) were identified. Each event compromised at least two SA levels, highlighting the complexity of these errors and their potential impact on patient safety. The distribution of OUEs by surgical procedure and residency year is shown in Table 2.
Table 2 Distribution of observable undesired events (OUEs) by surgical procedure and resident year.
| Surgery type | Resident type | # OUEs | % of Total OUEs (n=14) |
|---|---|---|---|
| Otorhinolaryngology | R1 | 6 | 42.9% |
| Urology | R1 | 2 | 14.3% |
| Neurosurgery | R2 | 5 | 35.7% |
| Cardiology | R3 | 1 | 7.1% |
| Total | 14 | 100% | |
Source: Authors.
According to Endsley 17, these interactions follow top-down or bottom-up processing Top-down processing is goal-directed, guiding attention and shaping the mental model used. In contrast, bottom-up processing is data-driven, where cues or signals trigger a cognitive schema directingactions. Of the 14 identified OUEs, 9 were top-down errors, while 5 followed a bottom-up approach. Representative examples of both processing types are shown in Table 3.
Table 3 Representative examples of top-down and bottom-up processing errors in situation awareness (SA).
| OUE description | Analysis from SA perspective | Relationship between compromised SA levels |
|---|---|---|
| A second-year anesthesiology resident performs peripheral vein cannulation with a large-caliber catheter, basing selection on vein thickness rather than location (distal), depth, and visibility. Accidentally pressing the release button causes vein perforation and excessive bleeding, necessitating the procedure to be repeated in a more proximal vein. | The resident uses an incorrect mental model for vein selection, which leads to erroneous expectations and a search for inaccurate data. |
Top-down ▼ The use of the incorrect mental model (SA II), conditions what to observe, misperceiving data when selecting the vein (SA I). |
| During neurosurgery, an anesthesiology resident notices the patient is re-inhaling CO2, surpassing the known threshold. The resident stares at the CO2 data on the monitor and the CAL container for a few minutes without acting until the nurse suggests changing the CAL. The resident perceives the CO2 data correctly but the mental model being incomplete, cannot understand the possible causes of the increase in CO2. | The resident perceives the CO2 data correctly but the mental model being incomplete, cannot understand the possible causes of the increase in CO2. |
Bottom-up ▲ By not monitoring the data continuously, the resident fails to observe changes in inhaled CO2 (SA I). The incomplete mental model makes the resident think it is a machine error (SA II), and fails to act (SA III). |
Source: Authors.
The analysis of OUE distribution by residency year showed that most R1 OUEs (6/8) arose from top-down processing ▼. In these cases, processing was goal-directed, but incomplete mental models led to failures in information-seeking, identification, and interpretation.
In contrast, R2 residents had more bottom-up OUEs ▲ (3/5), where incomplete or incorrect mental models resulted in errors in risk assessment and contingency planning. The single R3 OUE was top-down ▼, where an incomplete mental model hindered task organization and prioritization.
Most OUEs (11/14) stemmed from comprehension deficiencies, highlighting the need to develop and maintain adequate mental models in resident training. These failures affected critical cue detection and resource allocation, contributing to risk assessment and contingency planning errors.
Phase 3 - Retrospective interviews with eye-tracking videos
The retrospective interviews allowed the research team to identify the causal factors of SA errors based on Endsley's taxonomy 11. Table 4 presents the frequency and distribution of these errors according to their causal classification.
Tabla 4 Frecuencia y distribución de errores de CS por factor causal según el modelo de Endsley.
| SA error causal factor category | Frequency (n=14) | Resident level | % of total errors |
|---|---|---|---|
| Level 1 SA - Failure to correctly perceive situation | - | ||
| A. Data not available | - | ||
| B. Data difficult to detect/perceive | - | ||
| C. Failure to scan or observe data | - | ||
| 1. Omission | - | ||
| 2. Attentional narrowing/distraction | - | ||
| 3. High task load | - | ||
| D. Misperception of data | - | ||
| E. Memory failure | - | ||
| Level 2 SA - Failure to comprehend situation | 12 | 85% | |
| A. Lack of/poor mental model | 8 | R1 | 57.1% |
| B. Use of incorrect mental model | 4 | R2 | 28.6% |
| C. Over-reliance on default values in the model | - | ||
| D. Memory failure | - | ||
| E. Other | - | ||
| Level 3 SA - Failure to project situation into the future | 2 | 14.3% | |
| A. Lack of/poor mental model | - | ||
| B. Other | - | ||
| Maintaining multiple goals | 1 | R3 | 7.1% |
| Habitual schema | 1 | R2 | 7.1% |
Percentages are based on the 14 identified OUEs. Most errors were related to comprehension failures, particularly due to incomplete or incorrect mental models.
Source: Authors.
Seven of the eight OUEs among R1 residents were related to a lack of or poorly developed mental models; six were due to limited familiarity with the equipment or the procedures, and two were due to an inability to prioritize tasks. Missing mental models included recognizing haptic and auditory cues, which are crucial for monitoring signals during hand-eye coordination tasks. The five OUEs among R2 residents were associated with incorrect mental models that hindered adequate patient risk assessment. Only one OUE was found among R3 residents, related to maintaining multiple simultaneous goals and the difficulty in prioritizing various tasks.
Communication failures were also detected as causes of OUEs, including misinterpretation of supervisor instructions, lack of confirmatory questioning when in doubt, and unclear roles and responsibilities. These communication issues affected comprehension, leading to the construction of erroneous cognitive schemas and poor decision-making.
Training needs by SA levels
None of the residents had received prior training in SA, making it necessary to familiarize them with the basic model of both individual and team SA. This involves understanding the definitions of schemas and mental models, as well as the factors that can influence SA and strategies to mitigate them, along with assertive communication and task coordination, following the recommendations of Thomas 18.
Perception level: At this level, it is crucial for residents to identify which information is critical for each task and to develop effective scanning and search strategies to locate these data. Given the range of visual, auditory, and tactile cues they must attend to, it is essential for them to develop skills in attention allocation and to discriminate between auditory and haptic signals, especially during tasks requiring hand-eye coordination while monitoring auditory signals.
Comprehension level: Most errors at this level were due to missing, incomplete, or incorrect mental models. Thus, developing and maintaining accurate mental models is crucial for task success at all resident training levels. Additionally, residents must be able to identify common complications that may arise during different anesthetic phases and be familiar with specific equipment, formats, or procedures appropriate to their residency level.
Projection Level: Task management and planning are of paramount importance at this level to ensure that critical tasks are performed correctly and on time, thereby reducing the risk of errors and optimizing cognitive resources through the development of habits and routines.
In critical situations, the ability to prioritize and manage tasks can be decisive for patient safety. Most failures at this level occurred because residents did not evaluate the consequences of action or inaction, thereby putting the patient at risk. Therefore, a key training need is risk assessment and error management, including strategies and practices for preventing, identifying, analyzing, and correcting errors. Additionally, residents need strategies to cope with environmental pressures such as time constraints and pressures from surgeons or supervisors.
The identified SA training needs were structured according to Endsley's SA model (perception, comprehension, and projection) and categorized based on the residency year to align with the progressive complexity of anesthesiology training. Table 5 summarizes these training needs by SA level and residency year.
Table 5 Identified training needs in situation awareness (SA) for anesthesiology residents by SA level and year of residency.
| Training need | R1 | R2 | R3 |
|---|---|---|---|
| Basic theoretical knowledge | X | X | X |
| Individual and team SA models | X | X | X |
| Schemes and mental models | X | X | X |
| Factors affecting SA and strategies to avoid them | X | X | X |
| Assertive communication and task coordination | X | X | X |
| Perception | |||
| Identification of relevant pieces of information | X | ||
| Development of attention allocation skills | X | X | |
| Development of auditory discrimination skills | X | ||
| Development of haptic discrimination skills | X | ||
| Comprehension | |||
| Development and maintenance of mental models | X | X | |
| Identification of common complications | X | X | |
| Training in specific procedures | X | ||
| Familiarization with equipment and procedures | X | ||
| Projection | |||
| Task management and planning | X | ||
| Contingency planning | X | ||
| Risk assessment | X | ||
| Error management | X | ||
| Coping with environmental pressures | X |
Source: Authors.
Training needs by residency year
The main challenges for R1 residents have to do with developing mental models related to Level 2 of SA (comprehension) and monitoring cues, including identifying auditory signals at Level 1 of SA (perception). However, as highlighted in the interviews, it is essential to first develop mental models that will facilitate monitoring tasks. Therefore, in consultation with program coordinators, the proposed pilot training should emphasize the development and maintenance of mental models, focusing on room preparation and the first ten minutes of induction, as suggested in the focus groups.
R2 residents also experience difficulties with mental model use, but these are associated with specific procedures they must learn at this level. Additionally, challenges in attention allocation were observed during these procedures, particularly those involving hand-eye coordination, which can lead to attentional tunneling. For this reason, program coordinators agreed to conduct a pilot training focusing on Level 1 of SA - perception - specifically in developing attention allocation skills. Multitasking scenarios will be used, requiring residents to perform specialized procedures, such as using ultrasound for arterial vascular access.
For R3 residents, observations and interviews revealed challenges in task management and prioritization. Focus groups emphasized the importance of risk assessment and error management during crisis situations, given the complexity of cases these residents handle. Therefore, program coordinators agreed to conduct a pilot training focusing on the projection level, specifically in contingency planning, risk management in decision-making, coping with pressures, and strategies for preventing, identifying, analyzing, and correcting errors. Scenarios will involve crisis situations, managing others’ errors, and complex decision-making.
DISCUSSION
The findings of this study highlight the critical importance of SA in anesthesiology, where SA loss accounts for a high percentage of adverse events. By identifying 17 SA training needs through focus groups, eyetracking observations, and retrospective interviews, this study provides a structured framework for developing SA educational programs. Isolating SA within Anaesthetists' Non-Technical Skills (ANTS) and aligning these needs with Endsley's model reinforces its theoretical foundation and offers practical insights for improving resident training and patient safety, not only in anesthesiology programs but also in other medical fields.
This study spanned a considerable period due to the challenges posed by the COVID-19 pandemic, with restricted access to surgical environments and limited data collection. However, these constraints were mitigated by focusing on in-depth data analysis and aligning findings with the established frameworks and terminology in the literature. This approach ensured the relevance and rigor of the results despite the limitations.
Additionally, this project stands out as a pioneering effort in the field, given the uncommon access to real surgical settings and the use of eye-tracking technology to capture Observable Undesired Events (OUEs). Unlike most studies in SA in anesthesiology, which rely on adverse event reports, this study provides insights into events that, while not resulting in adverse outcomes, compromised patient safety. These findings contribute to a broader understanding of SA deficiencies and their implications for training.
In this study, OUEs associated with SA failures were identified and categorized according to Endsley's SA error taxonomy 11, organized by compromised SA levels and types of either "data-driven" or "goal-driven" information processing 17. This approach is in line with studies by Mica R. Endsley and Garland 19 and Mica R. Endsley and Robertson 20 on pilots, where they also analyzed SA failures and causal factors to identify SA training needs.
Furthermore, most of the needs identified in this study are consistent with strategies used in pilot training to improve SA, including the development and maintenance of mental models and critical cue identification 18. These strategies also align with higher-order skills specific to the anesthesia domain, as identified by Gaba et al. 21, which include environmental scanning, attention allocation, pattern recognition, and subtle signal detection.
Comparing our findings with previous literature, we find that, while some studies in anesthesiology emphasize the prevalence of perception errors 4, our study shows that comprehension-level errors are more common among residents. This difference may be attributed to the unique aspects of the teaching-learning process in anesthesiology, where residents are developing their own mental models in high-pressure, complex environments.
Although eye tracking is typically used to quantitatively assess visual behavior patterns 22-24, in this study it was used qualitatively to: 1) objectively detect OUEs and their association with perception and projection levels, 2) evaluate resident comprehension through self-confrontation with their own videos during the interviews, and 3) identify gaps between faculty expectations and the residents' actual performance.
This study focused on identifying SA training needs for anesthesiology residents. Although validation was not part of the original scope, a subsequent Delphi process conducted after a curriculum change confirmed that the identified needs required no modifications. To maintain the clarity and focus of this manuscript, these new findings, along with the development of a structured SA training curriculum, will be presented in a separate publication.
This study has limitations, including the limited number of observations and the lack of procedure diversity, which may affect the representativeness of the results. However, the extended timeline allowed for more thorough data analysis and methodology refinements. While initial data collection was constrained, later validation efforts with experts confirmed the relevance and applicability of the identified training needs. Additionally, the lack of recent literature addressing structured SA training underscores the ongoing need for studies like this one, which provide an evidence-based framework for guiding educational interventions in anesthesiology. These findings laid the groundwork for further developments, including training programs and evaluation systems, which are being presented in separate publications. Future studies should consider a larger sample of residents and a wider range of procedures to enhance the robustness and transferability of the findings.
Further research is essential to deepen our understanding of SA development in the teaching-learning process of anesthesiology and explore its characteristics and challenges. Additionally, a validation of the methodology used in this study could offer an innovative approach to identifying knowledge gaps in complex settings.
Despite the time elapsed since the ethical approval, the study remains relevant due to the limited integration of structured SA training in anesthesiology curricula. A review of the recent literature shows that research on SA in anesthesiology continues to focus primarily on retrospective analyses of adverse events rather than on proactive identification of training needs. The lack of studies addressing structured SA training reinforces the novelty of our approach. By addressing this gap, our study provides an evidence-based framework that remains essential in the current educational context.
These findings open the door to the design of specific SA training programs that integrate the training needs identified in this project, thus contributing to advancements in patient safety and improvements in clinical performance.
CONCLUSIONS
This study identified 17 SA training needs for anesthesiology residents, organized by year of training. The results highlight that first-year residents need to strengthen their perception and comprehension skills; second-year residents should improve their attentional skills; and third-year residents need to develop competencies in projection. Despite the challenges faced during its timeline, the iterative approach and thorough analysis underpin the robustness of these findings, providing a replicable framework for formal SA training to progressively address these gaps.
Validating these findings in broader and more diverse contexts will be crucial to enhance their applicability and robustness. The implementation of specific training programs using high-fidelity clinical simulations and eye-tracking glasses is recommended to continually evaluate and adapt SA training. This approach has the potential to significantly contribute to patient safety and optimize clinical performance in anesthesiology and beyond.
ETHICAL DISCLOSURES
Ethics committee approval
The study was approved by the research and ethics committees of the Schools of Engineering (FID 190, July 2, 2019) and Medicine (FM-CIE-0514-19, August 23, 2019) Schools at Pontificia Universidad Javeriana
Protection of human and animal subjects
The authors declare that no experiments were performed on humans or animals for this study. They also affirm that all procedures complied with the regulations of the relevant clinical research ethics committee and with the principles of the World Medical Association's Code of Ethics (Declaration of Helsinki).
ACKNOWLEDGMENTS
Author contributions
CDB: Conception of the original project as part of her doctoral thesis, study design, data acquisition, analysis and interpretation of results, initial and final drafting of the manuscript, and final approval.
FMOV: Study design, data collection, result interpretation, initial drafting of the manuscript, and final approval.
MPCG: Supervision of the doctoral thesis, study design, data acquisition, analysis and interpretation, initial drafting of the manuscript, and final approval.
EML: Data acquisition, analysis, and interpretation, assistance with initial manuscript drafting, and final approval.
DRS: General study supervision, study design, critical review of the manuscript, and final approval.
