ARTÍCULO ORIGINAL / ORIGINAL ARTICLE/ ARTIGO ORIGINAL

Anxiety and performance of nursing students in regard to assessment via clinical simulations in the classroom versus filmed assessments

Ansiedad y rendimiento de estudiantes de enfermería ante la evaluación presencial versus evaluación filmada de una práctica de simulación clínica

Ansiedade e rendimento de estudantes de enfermagem ante a avaliação presencial versus filmada de uma prática de simulação clínica

Carla Regina de Souza Teixeira¹; Luciana Kusumota²; Marta Cristiane Alves Pereira³; Fernanda Titareli Merizio Martins Braga⁴; Vanessa Pirani Gaioso⁵; Cristina Mara Zamarioli⁶; Emilia Campos de Carvalho⁷

¹RN, Ph.D. Professor, Escola de Enfermagem de Ribeirão Preto, Universidade de São Paulo (EERP-USP), Brazil. email: carlarst@eerp.usp.br.

²RN, Ph.D. Professor, EERP-USP, Brazil. email: kusumota@eerp.usp.br.

³RN, Ph.D. Professor, EERP-USP, Brazil. email: martacris@eerp.usp.br.

⁴RN, Ph.D. Professor, EERP-USP, Brazil. email: titareli@eerp.usp.br.

⁵RN, Ph.D. University of Alabama at Birmingham School of Nursing, United States of America. email: vgaioso@uab.edu.

⁶RN, Marter candidate. EERP-USP, Brazil. email: cristinazamarioli@usp.br.

⁷RN, Ph.D. Professor, EERP-USP, Brazil. email: ecdcava@usp.br.

Receipt date: May 7, 2013. Approval date: Feb 10, 2014.

Article linked to research: Avaliação dos atributos na simulação clinica de enfermagem: comparação entre filmagem e a presença do docente.

Subventions: Estudo financiado pelo CNPq - Edital Universal. Processo N° 474499/2010-3.

Conflicts of interest: none.

ABSTRACT

Objective.To compare the level of anxiety and performance of nursing students when performing a clinical simulation through the traditional method of assessment with the presence of an evaluator and through a filmed assessment without the presence of an evaluator. Methodology.Controlled trial with the participation of Brazilian public university 20 students who were randomly assigned to one of two groups: a) assessment through the traditional method with the presence of an evaluator; or b) filmed assessment. The level of anxiety was assessed using the Zung test and performance was measured based on the number of correct answers. Results.Averages of 32 and 27 were obtained on the anxiety scale by the group assessed through the traditional method before and after the simulation, respectively, while the filmed group obtained averages of 33 and 26; the final scores correspond to mild anxiety. Even though there was a statistically significant reduction in the intra-groups scores before and after the simulation, there was no difference between the groups. As for the performance assessments in the clinical simulation, the groups obtained similar percentages of correct answers (83% in the traditional assessment and 84% in the filmed assessment) without statistically significant differences. Conclusion.Filming can be used and encouraged as a strategy to assess nursing undergraduate students.

Key words:teaching; teaching materials; nursing; simulation; anxiety.

RESUMEN

Objetivo. Comparar el nivel de ansiedad y el rendimiento de estudiantes de enfermería, quienes realizaron una simulación clínica utilizando el método tradicional de evaluación (con la presencia del evaluador) y la evaluación filmada (sin la presencia de este). Metodología.Ensayo controlado con la participación de 20 estudiantes de una universidad pública de Brasil quienes fueron asignados aleatoriamente a dos grupos: a) evaluación por el método tradicional con la presencia del evaluador y b) evaluación filmada -sin la presencia del evaluador-. El nivel de ansiedad se evaluó con el test de Zung y el rendimiento con los aciertos presentados. Resultados.Se obtuvieron promedios en la escala de ansiedad de las evaluaciones de antes y después de la simulación de 32 y de 27, respectivamente, en el grupo del método tradicional versus 33 y 26 en el grupo de filmación después de la simulación, correspondiendo los puntajes finales a ansiedad leve. A pesar de la disminución estadísticamente significativa de las puntuaciones antes y después de ansiedad dentro de cada grupo, no hubo diferencia entre ellos. En cuanto a la evaluación del rendimiento en la simulación clínica, se obtuvieron porcentajes de aciertos similares en los grupos (83% en evaluación tradicional y 84% en la filmada) sin una diferencia estadísticamente significativa. Conclusión.La filmación podría ser utilizada y estimulada como una estrategia de evaluación en los cursos de licenciatura en enfermería.

Palabras clave:enseñanza; materiales de enseñanza; enfermería; simulación; ansiedad.

RESUMO

Objetivo.Comparar o nível de ansiedade e o rendimento de estudantes de enfermagem que realizaram uma simulação clínica, utilizando o método tradicional de avaliação com a presença do avaliador e a avaliação filmada sem a presença deste. Metodologia.Ensaio controlado com a participação de 20 estudantes de uma universidade pública de Brasil que foram atribuídos aleatoriamente aos grupos a) avaliação pelo método tradicional com a presença do avaliador e b) avaliação filmada. O nível de ansiedade se avaliou com o teste de Zung e o rendimento com os acertos apresentados. Resultados.Obtiveram-se médias na escala de ansiedade das avaliações de antes e depois da simulação de 32 e de 27, respectivamente, no grupo do método tradicional contra 33 e 26 no grupo de filmagem depois da simulação, correspondendo as pontuações finais a ansiedade leve. Apesar da diminuição significativa estatisticamente das pontuações antes e depois de ansiedade dentro de cada grupo, não teve diferença entre os grupos. Quanto à avaliação do rendimento na simulação clínica, obtiveram-se porcentagens de acertos similares nos grupos (83% em avaliação tradicional e 84% na filmada) sem uma diferença estatisticamente significativa. Conclusão.A filmagem poderia ser utilizada e estimulada como uma estratégia de avaliação nos cursos de licenciatura em enfermagem

Palavras chave:ensino; materiais de ensino; enfermagem; simulação; ansiedade.

INTRODUCTION

The acquisition of basic or advanced clinical skills by professionals in the health field is related to appropriate learning and repeated practice.¹ The development of a clinical rationale and the technical skills to obtain data and perform nursing interventions represents a challenge for both undergraduate students and educators. Coupled with this challenge is the difficulty assessing the development of these skills among students. Collecting data through physical assessment and interacting with patients, interpreting data using knowledge from different fields, correctly using techniques and instruments together with therapeutic decision-making and the performance of procedures constitute essential skills.

Clinical rationale requires competencies from the intellectual domain, which is expressed in knowledge and mental processes; from the interpersonal domain, mainly manifested in relationships with the patient; and techniques such as precise data collection during anamnesis and physical assessment.² Procedural skills (performance of techniques) belong to domains with varied complexity depending on the professional practice and patient needs, which are portrayed in the nursing intervention. Simulations are efficient in the development of clinical rationale and acquisition of procedural abilities.³ Some aspects, however, require further investigation. One of these aspects, the object of this study, is to investigate whether the presence of an evaluator in the scenario or filming impacts the students' levels of anxiety and clinical performance in a simulated situation of clinical assessment.

A clinical simulation in nursing is defined as an activity that mimics the reality of a clinical setting and is used to demonstrate procedures, decision-making and critical thinking through the use of interactive videos, games, manikins or simulated by patient-actors.⁴ Simulations can be used in different stages of training with different purposes in the teaching-learning process. When used in the teaching of nursing, simulations enable the assessment and measurement of various competencies necessary for a professional, such as the identification of deficiencies and training competencies,⁵ developing critical thinking, acquiring specific knowledge or even assessing students,⁶ such as assessments of behavior, decision-making skills, and performance in tests. The interest in research dealing with the acquisition of these essential skills and its process up to the development of proficiency is justified by the essence of professional practice. In this process, simulations are relevant for professional performance without posing risks to patients.⁷

Studies addressing simulations show that students deem their use to be a realistic strategy and that simulations make them more confident and skilled in a given competence, reducing their fear in performing procedures directly on patients. These studies also show that simulations confer greater safety for patients. Therefore, the use of simulations is indicated as a facilitator in the teaching of nursing.^8,9

The method of teaching through simulation enables a participative and interactive environment, using clinical scenarios that replicate real life experiences. It is obvious that teaching by means of simulation does not replace direct contact between patients and health professionals, which is a necessary condition for the development of skills among professionals. Teaching using simulations allows previously selected, planned and validated situations to be practiced until a high level of proficiency is achieved. Such a process avoids the situation where the interventions of health professionals are accompanied by deleterious or undesirable effects. The use of simulations allows the presentation of diverse scenarios (clinical cases), including those considered rare, within a controlled environment so that all students have the same opportunity to acquire practical learning. Another relevant aspect is the learning acquired in a situation in which the acquisition of knowledge is much more efficient, enabling the required knowledge and skills to be recalled more rapidly and efficiently.^10,11

The literature shows that learning how to perform a clinical assessment is one of many experiences that cause anxiety.¹² Even though anxiety is beneficial in certain tasks, it can also interfere in the learning process. Nursing students, in general, experience stress and anxiety over the course of their education and training.¹³ Therefore, differentiated forms of teaching and assessment that promote lower levels of stress or anxiety, as well as the identification of potential sources of stress in the teaching-learning process, should be considered during the training of nursing professionals.

The assessment methods used in teaching institutions are related to few advancements due to weaknesses observed in the faculty's pedagogical training, higher education policies, the structure of the university itself, in addition to aspects of the curricula and disciplines that do not take into account the expectations of students and current requirements concerning nursing education.^14,15 One study investigating variables that potentially impact anxiety and cognitive results in the context of learning in a high-fidelity simulation among 124 nursing students reports low levels of anxiety.¹⁶ The type of assessment was not the authors' objective of study.

This study's objective was to verify the potential influence of the presence of the evaluator or filming on the students' level of anxiety and performance in a simulated situation of clinical assessment. Therefore, the results are expected to: contribute to advancement in research in the field of nursing teaching; aid the identification of new assessment methods to promote comfort among students; and contribute to the optimization of human and financial resources in High Education Institutions. The purpose of this study was to compare the level of anxiety and practical performance of students when assessed through the traditional method, with the presence of an evaluator in the scenario, and the method in which the simulation is filmed without the presence of an evaluator in the scenario.

METHODOLOGY

This experimental, randomized and controlled study was conducted with students from the Nursing Undergraduate Program of a public university in Brazil. Two assessment strategies were compared during a clinical simulation: in one, assessment included the presence of an evaluator and in the other, assessment was filmed without the presence of an evaluator. The inclusion criterion for the selection of participants was being a student regularly enrolled in the fourth year of the program. This choice is based on the fact that students in this stage of the program have already encountered the disciplines of symptomatology, physical examination techniques and clinical practice in the hospital scenario. For this activity, the students should have knowledge concerning the basic principles of the administration of medications and basic Biosafety principles. All the students completed a test of such content containing statements for which the students should check False or True. (The maximum score was 6). A total of 73 students who met the inclusion criterion were invited. Of these, 20 students, corresponding to 27% of the study's population, manifested their interest and participated in the study. All the students correctly answered 84% or more of the test questions. The percentage of students who participated in the study is in agreement with data of a large portion of the primary studies included in the literature review on simulation as a teaching strategy in Nursing.¹¹

The clinical situation chosen for the study scenario was a patient with a history of a condition of pain with the recommendation to use analgesics if necessary. In this situation, students were required to assess the characteristics of pain and, based on this situation, they should identify the need to administer medication, prepare and administer the medication and later assess the results of the intervention. A medium-fidelity Advanced Nursing Simulator was used to implement the scenario. Among the various resources available in this simulator, the topics to be used in clinical assessment (data collection) of pain condition and nursing intervention were selected after the researchers programmed the simulator.

The simulation was planned according to the 11 dimensions that represent its characteristics¹⁷ in the context of a hospital setting to care for adult and/or elderly patients with the content of the clinical assessment of pain (data collection) and intervention, including the preparation and IV administration of medication. This choice was due to the personal experience of the researchers in the topic.

Table 1.Dimensions and objectives of the simulation developed in accordance with Gaba^17:127

Table 1.

Comparison between the two strategies of assessment (presence of an evaluator and filmed assessment) was performed with the following variables: level of anxiety and procedural requirements. Level of anxiety was assessed through scores obtained in the scale for self-assessment of anxiety¹⁸ before and after the procedure. To assess the intervention, the performance of the students was analyzed according to a list of items that included clinical assessment, decision-making to implement the intervention, preparation and administration of medication.

Three instruments were used in this study. The first is a validated scale for the self-assessment of anxiety with 20 items that characterize one's level of anxiety in a given period of time.¹⁸ The possible scores for each item range from 1 to 4. The minimum total score is 20 and the maximum is 80. The higher the score, the worse the symptoms associated with anxiety. The possible levels of anxiety when scores are weighted are: I - no anxiety (0 to 25); II - mild anxiety (25 to 50); III - moderate anxiety (50 to 75), and IV - severe anxiety (80 to 100). This instrument was completed before the simulation to measure the students' level of anxiety in the week that preceded data collection (format I). The same instrument was applied after the simulation but was modified for the specific situation of post-clinical simulation. Therefore, two of the 20 original items that were related to sleep quality were discarded (format II).

The second instrument was specifically developed for this study to characterize the individuals in regard to sex and age. The performance of each student was computed according to the number of correct answers (scores from 0 to 6). The third instrument, also developed for this study, was an assessment script with 18 items concerning data collection and preparation of medication and another 19 items concerning the administration of medication. The items were characterized as satisfactorily (correctly) performed, unsatisfactorily performed or not performed.

The apparent and content validity of the instruments used in this study (in regard to clarity and ease of understanding, comprehensiveness of content and presentation of items) were verified by three judges, who taught courses with clinical nursing content. A pre-test was also performed with five undergraduate students to identify the best way to apply it and implement necessary changes. Data were collected between February and August 2012 after approval was obtained from the Institutional Review Board. The researcher explained the study's objectives to the students and requested their participation. The researchers invited the participants either personally or through e-mail, which was sent three times. Data collection was scheduled with students outside the Nursing Program's regular hours.

After inclusion criteria were verified, the students were randomly assigned to a group of ten individuals: one manila envelope was drawn from among ten envelopes. (Five contained the word FILMING and the other five contained the word EVALUATOR).

All the students, regardless of the group to which they belonged, were assessed in the Laboratory of Nursing Communication, which has audio and video equipment (filming), where a clinical simulation scenario was set up for data collection. Initially, all the participants completed the characterization and self-report scale of anxiety¹⁸ in format I. Afterward, they watched a video with a clinical situation similar to the one used in the study. At this point, they were instructed about the clinical case for which they would provide nursing care.

The data collection and nursing intervention concerning the students from Group 1 was performed in the clinical simulation laboratory without filming and an evaluator/researcher assessed their performance in loco. The students from Group 2 also performed the nursing intervention and data were collected in the clinical simulation laboratory, but these students' performances were filmed (audio and video) without the presence of an evaluator/researcher in the scenario. A support professional was responsible for the filming and remained in a control room with unidirectional glass, which enabled him to view the entire laboratory. Afterwards, the researchers assessed the performance of students analyzing the film.

At the end of the simulation, the self-report anxiety scale in the format II was applied to all the students and they were debriefed. Collected data were organized and stored in a single spreadsheet in Microsoft Excel. The data were then analyzed and processed using the Statistical Analysis System® (SAS). Descriptive statistics were used and data were presented in tables, charts and descriptive measures. The mixed effects linear model (random and fixed effects) was used for data analysis¹⁹. The individuals were considered random effects and the groups, times and interactions between groups were considered fixed effects. Such a model assumes that the residual obtained through the difference among the values predicted by the model and the observed values have a normal distribution with an average 0 and constant variation. The model was estimated using the PROC MIXED procedure from the statistical software SAS 9.0. The level of significance adopted was α =5% (p<0.05).

The study project was submitted to and approved by the Institutional Review Board where the study was conducted (protocol No. 110503/2011) and the study's subjects signed free and informed consent forms. Note that the invitation and activities of the study assessment were conducted by research assistants who did not have any relationship of authority over the participants and did not teach any classes during the semester, in order to ensure the students could exercise free power of choice and to avoid undermining the voluntary nature of the students' decision or that it should be characterized by conflict.

RESULTS

A total of 20 students, 19 women and one man, participated in the study. The 10 students in group 1 (simulation with the presence of an evaluator) were 23.3 years old (SD=2.9) on average and the other 10 students in group 2 (filmed simulation) were 22.7 (SD=1.2) years old on average.

When comparing the levels of anxiety among all the students, the scores before the simulation ranged from 27 to 45, characterizing mild anxiety. After the simulation, the scores ranged from 22 (lack of anxiety) to 34 (mild anxiety). The group assessed through the traditional method with the presence of an evaluator obtained an average of 31.7 before and 26.5 after the clinical simulation. The group whose performance was filmed without the presence of an evaluator obtained an average of 33.4 before and 26.0 after the clinical simulation. The results were not statistically significant when the averages of anxiety obtained by both groups (inter-group) were compared (p=0.272 before the simulation and p=0.765 after the simulation). In turn, differences concerning intra-group average scores of anxiety obtained before and after the intervention were significant in the group whose assessment included filming (p<0.0001). Similar results were obtained with the group assessed with the presence of an evaluator (p<0.0001) (Table 2).

Table 2.Distribution of individuals according to the level of anxiety before and after clinical simulation. Ribeirão Preto, Brazil 2012

No significant differences were found in the averages concerning the level of anxiety between the groups before (p=0.272) and after (0.765) the simulation. In contrast, a reduction was observed in the intra-group average of stress before and after the simulation, the likelihood of which was below 0.001.

In regard to the assessment of procedures performed by the students in the clinical simulation, we note that there was a variation in the number of activities correctly performed by the students in both groups. All the students performed the nursing intervention; that is, all prepared and administered medication. The filmed group obtained averages of 84% and 83%, respectively, correct answers, while the group with the presence of an evaluator obtained averages of 85% and 78%, respectively, though no statistical differences were found between the averages (Table 3).

Table 3.Distribution of the performance of students in the clinical simulation in both groups (filmed and assessed with the presence of an evaluator), Ribeirão Preto, SP, Brazil 2012.

DISCUSSION

The literature²⁰ draws attention to the need for strategies that enable the mobilization of knowledge for decision-making, that is, the interface of knowledge, skills and attitudes for professional practice. In this sense, we highlight the contribution of simulation because it enables the examination of attitudes, skills and clinical competencies employing diverse scenarios, material and resources. Nursing students experience difficulties in learning techniques, in dealing with the receptivity of patients, and in controlling feelings of insecurity, fear or shyness. In this respect, the use of simulators favors the development of skills necessary for practice. ²¹

We know that a high level of anxiety can hinder learning, reduce students' ability to apply knowledge and interfere in the practice of critical rationale.^22,23 Even though clinical simulation laboratories are essential for the training of nursing students, a descriptive study conducted with 129 nursing students to describe the existence of stress associated with a practice laboratory identified 100% of the interviewees as having experienced some level of stress, while moderate stress was the most frequent.²⁴

The use of filming to assess simulations of clinical practice has been investigated in the literature and incorporated in undergraduate nursing programs. The objective is to advance assessment strategies that reduce stress among nursing students. There are, however, gaps in experimental studies exploring this subject in Brazil. Our results revealed mild anxiety during simulated procedures; the average scores of anxiety were not different between students who were filmed and those who had the presence of an evaluator, despite the lower scores after clinical simulations. These findings are in agreement with those of authors using this assessment strategy in the nursing field in both national²⁵ and international studies. ²⁶

One study assessing the effect of a video in the development of cognitive and technical competencies of nursing undergraduate students²⁵ reports similar findings, i.e., the level of anxiety among the students was low. Therefore, we can state with confidence that filming the performance of students is an appropriate strategy given the low anxiolytic power awakened in students. Such a result may be associated with the fact that these study's students experienced practical assessments throughout the program, which is perceived in cognitive ability without statistical differences between the groups and also based on the fact that the clinical simulation addressed subjects already explored during the undergraduate program.

When we assessed the clinical performance of students in the simulated situation regarding the preparation and administration of medication, an average of 80% correct answers was verified in both groups. Similarity among groups tested with educational strategies was also observed in one study²⁷ comparing the use of low-fidelity simulations and traditional classes concerning the administration of medication; that is, no differences were found between the groups and both groups (experimental and control) achieved a rate of 90% correct answers. The level of correct answers during the simulation draws attention and shows the importance of monitoring the development of skills as the students progress through the Nursing curricula.

CONCLUSION

This study is expected to contribute to the discussion concerning the identification of strategies to assess nursing students. This trial enables us to conclude that the presence of an evaluator in the scenario and the filmed assessment without the presence of an evaluator do not result in distinct levels of anxiety among students. The level of anxiety before the simulation was low and the clinical performance was similar between the groups and considered to be satisfactory. Therefore, this study's results suggest that the filming did not impact the students' level of anxiety or clinical performance and can be used and encouraged for use as a strategy to assess students in undergraduate nursing programs.

Some limitations have to be taken into account such as the low number of participants, which limits generalization of data, however, the similarity between the groups shows the study was appropriate to achieve its objective. Even though, the evaluators did not teach these students, it is possible they experienced difficulty agreeing to participate in the study.

Acknowledgment:the authors thank Dr. Jacqueline Ann Moss, PhD, RN, FAAN (Chair, Professor and Assistant Dean for Clinical Simulation & Technology) and Penni Watts, MSN, RN, CHSE (Director of Clinical Simulation and Training) from the University of Alabama at Birmingham, School of Nursing, USA.

REFERENCES

1. Troncon LEA. Utilização de pacientes simulados no ensino e na avaliação de habilidades clínicas. Med Ribeirão Preto. 2007; 40(2):180-9.         [ Links ]

2. Lunney M. Coleta de dados, julgamento clínico e diagnósticos de enfermagem: como determinar diagnósticos precisos. In: Diagnósticos de enfermagem da NANDA: definições e classificações 2012-2014 / [NANDA International] Porto Alegre: Artmed; 2013. p.113-33.         [ Links ]

3. Stroup C. Simulation usage in nursing fundamentals: integrative literature review. Clin Simul Nurs. 2013; 10(3):155-64        [ Links ]

4. Jeffries PR. A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nurs Educ Perspect. 2005; 26(2):96-103.         [ Links ]

5. Santos MC, Leite MCL. A avaliação das aprendizagens na prática da simulação em enfermagem como feedback de ensino. Rev Gaúcha Enferm. 2010; 31(3):552-6.         [ Links ]

6. Jeffries PR, Mcnelis AM, Wheeler CA. Simulation as a vehicle for enhancing collaborative practice models. Crit Care Nurse Clin North Am. 2008; 20(4):471-80.         [ Links ]

7. Ziv A, Ben-David S, Ziv M. Simulation based medical education: an opportunity to learn from errors. Med Teach. 2005; 27:193-9.         [ Links ]

8. Reilly A, Spratt S. The perceptions of under graduate student nurses of high-fidelity simulation based learning: a case report from the University of Tasmania. Nurse Educ Today. 2007; 27(6):542-50.         [ Links ]

9. Barsuk JH, Mcgaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009; 37(1):2697-701.         [ Links ]

10. Mizoi CS, Kaneko RMU, Filho CAM. A simulação realística como estratégia de treinamento para profissionais da saúde. Educ Contin Saúde. 2007; 5(32):100-1.         [ Links ]

11. Teixeira INDO, Felix JVC. Simulação como estratégia de ensino em enfermagem: revisão de literatura. Interface. 2011; 15( 39):1173-84.         [ Links ]

12. Pug CM, Iannitelli KB, Rooney D, Salud L. Use of manikin-based simulation to decrease student anxiety prior to interacting with male teaching associates. Teach Learn Med. 2012, 24(2):122-7.         [ Links ]

13. Suliman WA, Halabi J. Critical thinking, self-esteem, and state anxiety of nursing students. Nurs Educ Today. 2007; 27(2):162-8.         [ Links ]

14. Barbosa SL, Ferreira MER, Teixeira BMG. Promoción de un ambiente de aprendizaje positivo. Invest Educ Enferm. 2008; 26(2):106-12.         [ Links ]

15. Mitre SM, Siqueira-Batista R, Girardi-Mendonça JM, Morais-Pinto NM, Meirelles CAB, Pinto-Porto C et al. Metodologias ativas de ensino-aprendizagem na formação profissional em saúde: debates atuais. Ciênc. Saúde coletiva. 2008; 13(2):2133-44.         [ Links ]

16. Beischel KP. Variables affecting learning in a simulation experience: a mixed methods study. West J Nurs Res. 2013; 35(2):226-47.         [ Links ].

17. Gaba DV. The future vision of simulation in healthcare. Qual Saf Health Care. 2004;13(Suppl 1):i2-10.         [ Links ]

18. Zung WW. A rating instrument for anxiety disorders," Psychosomatics. 1971; 12(6):371-9.         [ Links ]

19. Schall R. Estimation in generalized linear models with random effects. Biometrika. 1991; 78:719-7.         [ Links ]

20. Almeida ML, Peres AM. Conhecimentos, habilidades e atitudes sobre a gestão dos formados de enfermagem de uma universidade pública brasileira. Invest Educ Enferm. 2012; 30(1):66-76.         [ Links ]

21. Silva CMC, Sabóia VM, Teixeira ER. O ensino do exame físico em suas dimensões técnicas e subjetivas. Texto Contexto Enferm. 2010; 18(3):458-65.         [ Links ]

22. Moscaritolo LM. Interventional strategies to decrease nursing student anxiety in the clínical learning environment. J Nurs Educ. 2009; 48(1):17-23.         [ Links ]

23. Hutchinson TL, Goodin HJ. Nursing Student Anxiety as a Context for Teaching/Learning. J Holist Nurs. 2013; 31(1):19-24.         [ Links ]

24. Musso LB, Vargas BA, Torres MB, Canto MJC, Meléndez CG, Balloqui MFK, Cornejo AS. Fatores derivados dos laboratórios intra-hospitalares que provocam estresse nos estudantes de enfermagem. Rev Latino-Am Enfermagem. 2008; 16(5):805-11.         [ Links ]

25. Cardoso AF, Moreli L, Braga FT, Vasques CI, Carvalho EC. Effect of a video on developing skills in undergraduate nursing students for the management of totally implantable central venous access ports. Nurs Educ Today. 2012; 32:709-13.         [ Links ]

26. Williams B, French J, Brown T. Can interprofessional education DVD simulations provide an alternative method for clínical placements in nursing? Nurs Educ Today. 2009; 29(6):666-70.         [ Links ]

27. Kinney S, Henderson D. Comparison of Low Fidelity Simulation Learning Strategy With Traditional Lecture. Clin Simul Nurs. 2008; 4(2):15-8.         [ Links ]