ARTÍCULO ORIGINAL / ORIGINAL ARTICLE/ ARTIGO ORIGINAL

Level of involvement of clinical nurses in the evaluation of competence of nursing students

Nivel de implicación de enfermeras clínicas en la evaluación de estudiantes de grado en enfermería

Nível de envolvimento de enfermeiras clínicas na avaliação de estudantes de graduação em enfermagem

¹RN, Ph.D. Professor, Universitat Jaume I -;UJI-, Spain. email: macia@uji.es.

²RN, Ph.D. Professor, UJI, Spain. email: vchorda@uji.es.

³RN, Master. Professor, UJI, Spain. email: psalas@uji.es.

⁴RN, Master. Professor, UJI, Spain. email: dmena@uji.es.

⁵RN, Master. Professor, UJI, Spain. email: cerveraa@uji.es.

⁶RN, Ph.D. Professor, UJI, Spain. email: isabel.orts@uji.es.

Receipt date: March 2, 2014. Approval date: June 3, 2014.

Article linked to research: none.

Subventions: none.

Conflicts of interest: none.

How to cite this article: Maciá-Soler ML, González VM, Salas P, Mena D, Cervera A, Orts MI. Level of involvement of clinical nurses in the evaluation of competence of nursing students. Invest Educ Enferm. 2014; 32(3): 461-470.

ABSTRACT

Objective. To determine the level of involvement of clinical nurses accredited by the Universitat Jaume I (Spain) as mentors of practice (Reference Nurses) in the evaluation of competence of nursing students. Methodolgy. Cross-sectional study, in which the "Clinical Practice Assessment Manual'' (CPAM) reported by reference 41 nurses (n=55) were analyzed. Four quality criteria for completion were established: with information at least 80% of the required data, the presence of the signature and final grade in the right place. Verification of learning activities was also conducted. Data collection was performed concurrently reference for nurses and teachers of the subjects in the formative evaluations of clinical clerkship period in the matter "Nursing Care in Healthcare Processes ", from March to June 2013. Results. 63% of CPAM were completed correctly, without reaching the quality threshold established (80%). The absence of the signature is the main criteria of incorrect completion (21%). Nine learning activities do not meet the quality threshold set (80%) (p < 0.05). There are significant differences according to clinical units p < 0.05. From the 30 learning activities evaluated in the CPAM, it can be stated that nine of them do not reach the verification threshold established (80%), therefore it cannot be assumed that these activities had been completed by students and evaluated by the RefN throughout the clinical clerkship period. Conclusion. The level of involvement of Reference Nurse cannot be considered adequate, although strategies to encourage involvement through collaboration and training must be developed. 

Keywords: students, nursing; education, nursing; mentors; clinical clerkship.

RESUMEN

Palabras clave: estudiantes de enfermería; educación en enfermería; tutores; prácticas clínicas.

RESUMO

Objetivo. Identificar o nível de envolvimento das enfermeiras clínicas credenciadas pela Universitat Jaume I (Espanha) como tutoras de práticas (enfermeiras de referência) na avaliação de competências dos estudantes de enfermagem. Metodologia. Estudo de corte transversal, no que se analisaram as ''Guia de Avaliação de Práticas Clínicas'' reportadas por 41 enfermeiras de referência (n=55). Definiram-se quatro critérios de qualidade do preenchimento: que tenha informação pelo menos de 80% dos dados requeridos, presença da assinatura e qualificação final no lugar adequado. Também se realizou a verificação das atividades de aprendizagem. A recolhida de dados se realizou de forma concorrente pelas enfermeiras de referência e o professorado das matérias nas avaliações formativas do período de práticas clínicas tuteladas da matéria ''Cuidados de enfermagem em processos assistenciais'' de Março a Junho de 2013. Resultados. 63% das Guias de Avaliação se entregam preenchidas corretamente, sem atingir o umbral de qualidade estabelecido (80%). A ausência da assinatura é o principal critério de preenchimento incorreta (21%). Das 30 atividades de aprendizagem avaliadas nas guias, pode afirmar-se que nove delas não atingem o padrão de verificação estabelecido (80%), de forma que não pode assegurar-se que estas atividades tenham sido realizadas pelos alunos e avaliadas pelas Enfermeiras tutoras ao longo do período de práticas clínicas. Conclusão. O nível de envolvimento das enfermeiras de referência deve melhorar-se e para isto é necessário desenvolver estratégias que fomentem seu envolvimento através da colaboração e a formação.

Palavras chave: estudantes de enfermagem; educação em enfermagem; mentores; estágio clínico.

INTRODUCTION

Nursing education within the EU environment is undergoing major changes promoted by the new educational policies that culminated in the implementation of the European Higher Education Area in 2010. The EU Directive 2005/36/EC¹ established the minimum training requirements for the free movement of nurses within the EU. Nursing education within the clinical setting is an essential component of the undergraduate nursing curriculum, accounting for 50% of the total educational program, being necessary for clinical nurses to be involved in the learning process as practice mentors. In Spain, the Order CIN 2134/2008² stipulates the requirements for the verification of academic degrees that qualify for the practice of nursing, specifying the skills that students must acquire and establishing a 90 ECTS module of supervised clinical practice for graduate nurses. The mentoring of nursing students during clinical practice is essential in nursing education, enabling students' individual learning process, skill acquisition and development of their professional identities.³ Moseley and Davis⁴ define the practice mentor's role as a person who guides students, acting as a role model and helping them to bring theory into practice.⁵

Nursing Degree at Universitat Jaume I (UJI) (Spain) has recently implemented a program of clinical supervision through which coordinated actions among lecturers and clinical nurses that protect students during clinical clerkship. This program, called Reference Nurse Program (RefN Program), has recruited 200 nurses in an environment of three departments of Health, with 3 public hospitals, 2 private hospitals and 23 primary care centers during the first year of implementation. The program is based on the ability to develop competency assessment with clinical nurses from the workplace, are committed to direct supervision and evaluation of the acquisition of skills in the clinical setting,⁶ participating in the teaching-learning process.⁷ Participation in the RefN Program is voluntary, but in order to access, clinical nurses must meet certain requirements including research experience, job tenure are working during the academic year and experience in the healthcare unit. Nurses who meet these requirements reach an initial 40-hour training program, with content related to the educational program of the degree, assessment of competence in clinical practice and evidence-based practice. This training accredits nurse as mentors of clinical clerkship, or Reference Nurse (RefN).

Lecturers and RefN use a clinical practice guideline, previously agreed, to evaluate the acquisition of competence (Clinical Practice Assessment Manual).^8,9 The Clinical Practice Assessment Manual (CPAM) incorporates verification and registration activities for the achievement of defined learning objectives, and the sequence of formative assessment, and summative assessment at the end of supervised clinical clerkship. Authors such as Wilkes¹⁰ and Ownby et al.¹¹ highlight the importance of qualities such as enthusiasm, kindness or availability, and appropriate training of mentors, to achieve an effective clinical learning environment, for which mentors' involvement at a high level is required.^12,13The main objective of this work is to determine the level of involvement of clinical nurses accredited by the Universitat Jaume I (Spain) as mentors of practice (Reference Nurses) in the teaching-learning process and evaluation of competence of nursing students through the completion the CPAM.

METHODOLOGY

Design. An analytical, observational, prospective, cross-sectional study was developed to determine the level of involvement of the Reference Nurse (NR) through the analysis of the completion of the Clinical Practice Assessment Manual (CPAM), with predetermined quality criteria, and three indicators (completion rate of guide, signature and qualification of the ER).

Population and sample. The study population consists of 200 clinical nurses appointed by UJI as RefN, who are involved in the teaching-learning process within the clinical setting and carry out the assessment jointly with the lecturers, through the completion of the CPAM. RefN is defined as a registered nurse trained and accredited by UJI, who voluntarily participates in the teaching-learning process, and assumes the assessment of competence in the clinical setting for undergraduate nursing students, in collaboration with the lecturers of the subjects. Nursing degree at UJI is being implemented, and RefN are available only in the second year. Thus, through intentional sample, the selected sample included the CPAM completed by 41 appointed clinical nurses who mentored 65 second-year students during the clinical practice of the matter  "Nursing Care in Healthcare Processes" (NCHP), conducted in 16 units of 5 public and private hospitals attached to the University during the second semester (from March 20th to June 26th, 2013). The matter NCHP includes the following subjects: ''Primary Care Nursing'', ''Nursing Care in Osteoarticular Processes'', ''Nursing Care in Digestive'', Endocrine and Renal Processes'' and ''Nursing Care in Cardiovascular and Respiratory Processes''.

Variables under study. Three groups of variables were used: a) identification: hospitaland clinical practice units; b) quality criteria for completion of the CPAM: i) Completion Rate -;CR- (number of verified activities divided by 30 and multiplied the result by 100), ii) Guide signed by the RefN, iii) Final grade in its corresponding place; and c) Verification of learning activities: The CPAM includes a total of 30 activities based on the learning outcomes, which are organized sequentially by increasing their complexity throughout the clinical practice. The verification and formative assessment of these learning activities is carried out during the third, fifth, eighth and twelfth week of clinical practice. The summative assessment of the supervised clinical practice is carried out in the last week.

Data Analysis. Analysis was performed according to the groups of variables: a) Completion of the CPAM: descriptive analysis (percentages) was conducted on the completion quality criteria: manual signed by the RefN, final grade of the supervised clinical practice in its corresponding place and CR greater than 80%. The Z test was conducted to a sample in order to confirm whether the 80% of correctly completed CPAM was reached. By means of the Ji square or the Fisher's exact test, it is considered whether there is a dependency between the correct completion and the hospitals and units of supervised clinical practice; b) Verification of learning activities: The learning activities included in the clinical practice assessment manuals were studied descriptively in the third, fifth, eighth and eleventh week. The T-student test was carried out in order to check whether the average of verified activities exceeded the 80% of the planned activities in the third, fifth, eighth and eleventh week. By means of the ANOVA or the non-parametric Kruskal-Wallis test, and based on the implementation conditions, it was analyzed whether there were differences in the number of verified activities between the hospitals and units in every week and for the total activities. Finally, the verification percentage of every activity was calculated by means of the z test for proportions, estimating whether this percentage is set below 80% (n = 45) for the total manuals.  Moreover, by means of the Ji square test of independence or the Fisher exact test it is considered whether there is a dependency between the verification of the learning activities and the practice units when the number of observations per group is n <5. Statistical analysis of the data is performed with R Commander application of the R 3.0.2 software. A 5% bilateral significance level was assumed in hypothesis tests.

Sources of information and data collection. The CPAM used by the RefN for evaluation of the students are the primary source of information. Data collection is performed concurrently by the RefN and lecturers in formative assessments (third, fifth, eighth and twelfth weeks) of the clinical clerkship period of the subject " Nursing Care in Healthcare Processes", from 20^th March to 26^th June, 2013. Upon completion of the clinical clerkship period, lecturers delivered all documentation to the principal investigator.

Ethical considerations. There is an agreement among the professors of the subjects regarding the use of the CPAM to carry out this study. The anonymity of the RefN and the students the CPAM refer to is preserved at all times through a prior anonymization process.

RESULTS

It is observed in Table 1 that only a 62.5% (n=35) of the Clinical Practice Assessment Manuals are delivered correctly completed by the RefN, not reaching the 80% considered as quality threshold (p <0.05).

Descriptive data shows that the number of CPAM correctly completed by nurses only exceed the 80% at two hospitals.  Incorrect completion is mainly due to the RefN's lack of signature in a 21.4% (n = 12) of the CPAM. Moreover, the descriptive data based on the units shows that in a 43.75% of the units (n=7) the RefN delivered all the CPAM correctly completed, while in a 31.25 % of 5 units (n=5) the CPAM did not meet any of the quality criteria established. The results of the inferential analysis confirmed that the 80% of correctly completed manuals was not met, and that there is a statistically significant dependence between the correct completion of the manuals and the clinical practice units (p <0.05).

Verification of learning activities

In Table 2 it is seen that activity verification average for the total CPAM is 24.76 (s=8.14), although the sample behaves heterogeneously with a variation coefficient of 32%. Moreover, it cannot be stated with a 95% confidence that the mean of verified activities for all the manuals is greater than 24 (p<0.05). By studying the verification of learning activities on a weekly basis, it can be said that the verification threshold established was exceeded in the 3rd, 5th and 8th week, whereas it could not be reached in the12th week (p<0.05). It can also be stated that there are statistically significant differences in the verification of learning activities based on both the care units and the hospitals, except in the 3rd week (p <0.05).

DISCUSSION

Clinical learning is one of the main components of nursing education. Mentorship of students during clinical clerkship requires a high level of involvement of professionals who assume this responsibility, as mentors reported in a recent study by Broadbent et al¹⁴, coinciding with other authors such as Miller, Francis and Bonner,¹² or Jokelainnen et al.¹³ In its first year, the RefN Program has recruited and trained 200 clinical nurses. The high participation of clinical nurses in a voluntary program for reporting them increased workload, without any financial incentive, indicates the interest shown by these professionals to mentor and evaluate students.¹⁵ The results confirm that the completion of the CPAM not meet the quality threshold expected, according to the quality criteria (completion rate, signs of RefN and qualification in the right place), revealing that the level of involvement of the RefN in the clinical learning and assessment of competence does not seem appropriate. Moreover, the analysis of the verification of the learning activities reinforces this claim.

The review of the literature have not identified further work to examine the quality of documentation used by mentors to know their level of involvement, being necessary to detect which external or internal factors to the RefN Program may have influenced results. It is true that the RefN are accustomed to a traditional model of clinical practice developed in Spain,¹⁶ in which there is little linkage between educational and health institutions, and where students are incorporated into the routine of the units, with no one clear evaluation criteria and specific documentation which reflect the results.^17,18 In the UK similar to the Spanish situation was until the late 20th century. In 1986, the Central Council for Nursing published the Project 2000, which revolutionized nursing education, and in 1997 initiated the Mentorship Program, whereby each nursing student performs clinical clerkship under the supervision of a clinical nurse who has overcome a specific period of training and she assumes the assessment of competence.¹⁹ A strategy of this magnitude is not known in Spain. In different countries as Finland,³ United States¹¹ or Australia²⁰ are developing similar strategies for clinical mentorship, either at national or local level, providing positive results compared to traditional models.²¹

Factors related to the RefN's work context, which combine the clinical and teaching roles in their shifts, may have influenced the results. Aspects such as high work load, shift work or staff turnover can affect the involvement of RefN, coinciding with the results reported by other international^12,22,23 and Spanish authors.²⁴ These factors require that the RefN often delegate their responsibilities for student learning in auxiliary staff,²³ especially when it comes to coverage of basic care, affecting the quality of learning.

Another external factor that may influence the results is that students at UJI share practical space with two other nursing schools in which there is the figure of the tutor, responsible, or supervisor nurse. However, the main difference with the ER is participation in student assessment,¹⁵ which has generated some resistance and controversy among professionals regarding this figure. Assessing the acquisition of skills in clinical settings is one of the main challenges are facing universities and one of the major handicaps to the professionals that protect students.²⁶ The reasons are mainly related to the weaknesses in teaching methodology of mentors, lack of knowledge of educational programs and the absence of clear evaluation criteria that hinder the tutorial function.^17,26 The training of mentors and the use of objective assessment tools are essential to facilitate the tutorial action,^3,13,27 allowing identification of low-performing students, in order to take corrective measures in the educational process to achieve adequate²⁸ skills acquisition. In our study, RefN received previous training that certifies them to develop the tutorial function, with content about the curriculum, assessment of competence in clinical practice and evidence-based practice. The RefN's satisfaction regarding the training received is evaluated with an elaborate ad hoc survey with 12 items and a Likert-type scale with 5 levels (1=not at all satisfied, 5=very satisfied). The analysis of the surveys report an average overall satisfaction of 3.65 (s = 0.8) points.¹⁵ This fact encourages us to continue working on the same line, identifying possible areas for improvement in training. Moreover, lecturers should maintain a necessary link with clinical practice,^3,25 although the role of the teacher in clinical practice does not appear to be well established,^19,29 In our case, this link is maintained so that each lecturer is responsible for monitoring one or two hospitals during clinical clerkship, and they make weekly visits to RefN, participating in formative assessments, set out in the CPAM, and summative assessment.

The results of this study highlight the need to review the lecturers' proper performance during clinical clerkship, because the quality threshold set at the completion of the CPAM is not reached, in addition, there was a statistically significant dependence between verification learning activities and hospitals where students undertake clinical clerkship. The results should be treated with caution, because a randomization process of the sample has not been carried out, but a convenience sampling in a population of 200 RefN has been used, including single to RefN that met the selection criteria. Moreover, the study was carried out after the first year of implementation of the program RefN, underlining that the results are useful to initiate improvement actions that directly impact the quality of student learning and further evaluation and RefN improvement program, despite these methodological limitations.

Conclusions. The level of involvement of RefN that mentoring students from Nursing Degree at Universitat Jaume I cannot be considered appropriate in terms of the indicators studied. External factors related to the resistance that involves changing educational model and professional work environment that may affect their level of involvement should be considered. Moreover, there are internal factors in the RefN Program should be reviewed to improve their quality. These factors include collaboration with RefN during the period of clinical practice and the training they receive. Monitoring these quality criteria and developing new ones will be useful for the evolution of the RefN Program and the impact of possible improvement actions.

REFERENCES

1. Directiva 2005/36/CE del Parlamento Europeo y del Consejo de 7 de septiembre de 2005, relativa al reconocimiento de cualificaciones profesionales. Unión Europea (Sep 7, 2005).         [ Links ]

2. Orden 2134/2008 de 3 de julio, por la que se establecen los requisitos para la verificación de los títulos universitarios oficiales que habiliten para el ejercicio de la profesión de Enfermero. Boletín Oficial del Estado núm. 174 (Jul 19, 2008)        [ Links ]

3. Jokelainen M, Turumen H, Tossavainen K, Jamookeeah D, Coco K. A systematic review of mentoring nursing sutdents in clinical placements. J Clin Nurs. 2011; 20(19-20):2854-67         [ Links ]

4. Moseley LG, Davies M. What do mentors find difficult? J Clin Nurs. 2008; 17(12):1627-34.         [ Links ]

5. Perry B. Role modelling excellence in clinical nursing practice. Nurse Educ Pract. 2009; 9(1):36-4.         [ Links ]

6. Maciá Soler L, Orts Cortés I, Galiana Sánchez M, Orts Montenegro A. Simultaneous implementation of the Bachelor, Masters and PhD degrees in the Universidad Jaume I. Castellón de la Plana, Spain. Invest Educ Enferm. 2013; 31(2): 305-14.         [ Links ]

7. Orts Cortés MI, Maciá Soler, ML (Editors). Guía de prácticas clínicas. Universitat Jaume I- Departamentos de Salud de Castellón [Internet]. Castellón, España: Repositorio Institucional de la Universitat Jaume I; 2012 (cited Sep  29 2013) Available from: http://hdl.handle.net/10234/47994        [ Links ]

8. González Chordá VM, Salas Medina P, Juan i Porcar M, Orts Montenegro A, Bou Estelles J, Guillermon Gimeno M, et al. Guía de la evaluación de las prácticas clínicas. Bloque II: Cuidados básicos de Enfermería. Cuidados en procesos osteoarticulares, procesos digestivos, renales y endocrinos. Cuidados en procesos cardiovasculares y respiratorios [Internet]. Castellón, España: Repositorio institucional. Universitat Jaume I; 2012 (cited Aug 4 2014). Available from: http://hdl.handle.net/10234/48441          [ Links ]

9. González Chordá, VM; Salas Medina, P; Juan i Porcar, M; Galiana Sánchez, ME; et al. Guía de la evaluación de las prácticas clínicas. Bloque I: Salud y entorno. [Internet] Castellón, España: Repositorio institucional. Universitat Jaume I; 2012 (cited Sep  29 2013). Available from: http://hdl.handle.net/10234/48474        [ Links ]

10. Wilkes Z. The student-mentor relationship: a review of the literatura. Nurs Stand. 2006; 20(37):42-7.         [ Links ]

11. Ownby K, Schumann R, Dune L, Kohne D. A comparison of a traditional clinical experience to a precepted clinical experience for baccalaurate-seeking nursing students in their second semester. Nurs Res Pract. 2012; 2012: 276506.         [ Links ]

12. Milles JE, Francis KL, Bonner A. Mentoring, clinical supervisión and preceptoring: clarifing the conceptual definitions for Australian rural nurse. A review of the literatura. Rural and Remote Health. 2005; 5(3):1-10.         [ Links ]

13. Jokelainen M, Jamookeeah D, Tossavainen K, Turunen K. Finnish and British mentor's conceptions of faciliting nursing students' placement learning and profesional development. Nurs Educ Pract. 2013; 13:61-7.         [ Links ]

14. Broadbent M, Moxham L, Sander T, Walker S, Dwyer T. Supporting bachelor of nursing students within the clinical environment: Perspectives of preceptors. Nurse Educ Pract. 2014. pii: S1471-5953(13)00216-3. [Epub ahead of print]         [ Links ]

15. Cervera-Gasch A; González-Chordá VM; Mena-Tudela D;  Salas-Medina, P; Maciá-Soler ML; Orts-Cortés MI. Satisfaction of clinical nurses with the formation of a university tutorial program. EduRe Journal. International Journal on Advances in Education Research. 2014, 1(3): 119-131 (cited Aug 13 2014) Available from: http://edure.org/EduReJournalVol1N3/EduRe_V1_I3_P9.pdf        [ Links ]

16. Agencia Laín Entralgo. Guía del Tutor de Grado de Enfermería. 3rd ed. Madrid, España: Ed. Laín Entralgo; 2009.         [ Links ]

17. Kim CJ, Park JW, Kang SW. Effects of a collaborative clinical practicum on clinical practice ability and teaching effectiveness among nursing students. J Nurs Educ Pract. 2013; 3(12):143-50.         [ Links ]

18. González J, Wagenaar R (Editors). Reference points for the desing and delivery of degree programmes in Nursing. Tuning Project [Internet]. Bilbao, España: Universidad de Deusto; 2011 (cited Apr 26 2014). Available from: http://www.unideusto.org/tuningeu/publications.html        [ Links ]

19. Barrett D. The clinical role of nurse lecturers: past, present and future. Nurse Educ Today. 2007; 27(5):367-74.         [ Links ]

20. Franklin N. Clinical supervision in undergraduate nursing students: a review of literature. E-JBEST. 2013; 7(1):34-42.         [ Links ]

21. Tagwa YO, Wakifa AS, Lisamma T, Jayashanthimani J. Perception of nursing students to two models of preceptorship in clinical training. Nurse Educ Pract. 2013; 13(3):155-60.         [ Links ]

22. Madhavanpraphakaran GK, Shukri RK, Balachandran S. Preceptors' Perceptions of Clinical Nursing Education. J Contin Educ Nurs. 2014; 45(1):28-34.         [ Links ]

23. Carnwell R, Baker SA, Bellis M, Murray R. Managerial perceptions of mentor, lecturer practitioner and link tutor roles. Nurse Educ Today. 2007; 27(8):923-32.         [ Links ]

24. Bardallo Porras MD. Entre la Enseñanza y el aprendizaje. Un espacio de saberes para compartir [Dissertation]. Barcelona, España: Universidad de Barcelona; 2010.         [ Links ]

25. Huybrecht S, Loeckx W, Quaeyhaegens Y, De Tobel D, Mistiaen W. Mentoring in nursing education: Perceived characteristics of mentors and the consequences of mentorship. Nurse Educ Today. 2011; 31(3):274-8.         [ Links ]

26. Ali PA, Panther W. Professional development and the role of mentorship. Nurs Standard. 2008; 22(42): 35-9.         [ Links ]

27. Gray M, Donaldson J.  National Approach to Practise Assessment for Nurses and Midwives; Exploring Issues in the Use of Grading in Practice e Literature Review. Edimburg: Edimburgh Napier University, Scotland; 2009.         [ Links ]

28. Heaslip V, Scammell JME. Failing underperforming students: The role of grading in practice assessment. Nurse Educ Pract. 2012; 12(2):95-100.         [ Links ]

29. Price L, Hastie L, Duffy K, Ness V, McCallum J. Supporting students in clinical practice: Pre-registration nursing students' views on the role of the lecturer. Nurse Educ Today. 2011; 31(8):780-4.         [ Links ]