Introduction
Clinical reasoning is a concept widely studied among healthcare-related professions.1 In general terms, it refers to the integration of critical thinking and decision-making in clinical situations; besides, its definition may vary depending on the healthcare-related profession in which it is used.2
In the case of physical therapy specialty, clinical reasoning is an iterative and adaptive process that is based on the interaction of the physical therapist with the patient, with the environment in which the patient is treated and with the healthcare team. In this context, physical therapy interventions and goals are defined according to the information reported by the patient, the clinical context, and the clinical assessment of the patient; therefore, clinical reasoning allows making well-grounded decisions that take into account evidence-based recommendations, clinical practice guidelines, the experience of the physical therapist and the patient's expectations.3-7
Furthermore, in the field of physical therapy, clinical reasoning can be also defined as an integrative and cognitive process that considers both movement analysis and the interaction of the patient with their environment, that is, it is an adaptive, iterative, and collaborative process that brings physical therapists closer to using a biopsychosocial model in clinical practice,4,6 which implies being able to design an optimal communication strategy with the patient. With this in mind, clinical reasoning, in this context, is the ability of the physical therapist to argue their clinical hypotheses.7 In addition, it requires a full ethical commitment to science during clinical practice to optimize the achievement of positive health-related outcomes.4,5,7-9
In the field of physical therapy, knowing how to solve problems faced during clinical practice by making responsible and timely clinical decisions is of great importance, which is why clinical reasoning is a fundamental skill for the proper professional performance of physical therapists.9-11 However, physical therapy programs have a hard time to include in their study plans methodologies that promote the development of clinical reasoning since early stages of training,12 given the differences between students and experienced physical therapists in terms of clinical reasoning, that is, the level of their skills to reflect when making clinical decisions.2,3,9,12,13
In this sense, the physical therapy program offered at Universidad Andres Bello is based on the achievement of learning outcomes (LO) by means of instructional methods focused on what physical therapy students need to learn for having an adequate performance during their professional life.14-16 It should be noted that LOs are defined as the knowledge and understanding of a given topic, action or process that a student shows at the end of a training period, that is, the evidence of the student's level of compliance with the training process.17-23
Objective structured clinical examination (OSCE) allows assessing standardized clinical skills and is used for the training and evaluation of health sciences students who perform tasks in healthcare teams in clinical practice scenarios.24,25 Its objective is directly related to developing and integrating critical thinking and planning, as well as to using the acquired knowledge in specific situations that involve carrying out professional healthcare-related tasks that require clinical reasoning.9,22-25 This type of examination strengthens the sense of security, practical ability, operability, and critical judgment of students and prepares them for their professional practice.25,26
The OSCE methodology focuses on evaluating clinical skills that promote the development of clinical reasoning by placing the student in a clinical setting in which they must use the acquired knowledge and their own abilities to solve a clinical situation,1,27 this is why, situations proposed in OSCE stations are designed to measure clinical reasoning-related skills, and not only memory-related or mechanical (repetition of procedures) skills.28
Furthermore, clinical reasoning includes both formal and informal thoughts in the decision-making process; thus, taking into account the students' perception of the OSCE is also important.23 For this reason, evaluating the student's level of satisfaction with the OSCE methodology is recommended, since this will provide feedback regarding the design and execution of the stations used in the OSCE and will allow them to improve their clinical skills during their training, since, through this methodology, they can reflect and introspect on the development of their clinical reasoning during their training as physical therapists.26,29
Therefore, the objective of this study was to describe the implementation of an OSCE to evaluate the clinical skills associated with the development of clinical reasoning in physical therapy students and to determine their level of satisfaction with this methodology.
Material and methods
Study design and sample
Descriptive cross-sectional study. The study population consisted of all the fourth semester students of the Physical Therapy program at the Universidad Andres Bello (Santiago, Chile) who were taking the Reasoning in Physical Therapy course in the second semester of 2018 (N=163).
All students took the course and completed the OSCE, however four of them did not sign the informed consent form, so the final sample was 159 students.
Procedures and instruments
OSCE
A committee made up of seven professors of the Reasoning in Physical Therapy course was created in order to design and implement the OSCE in the course taking into account the following aspects: stations, checklists, content validation and pilot test.30,31
Before the OSCE was administered to the students of the Reasoning in Physical Therapy course, a pilot test was conducted in 30 students with similar characteristics, who, after signing the informed consent form, were asked to complete all the stations of the OSCE. During the pilot test, the 7 professors of the abovementioned committee evaluated the performance of the 30 students in each station and, by means of a checklist, issued their comments regarding the adjustments that needed to be made, namely: 1) to improve the wording of the checklist items; 2) to increase the time each student had to complete each station, and 3) to remove a rest station.
On the other hand, students' clinical reasoning was assessed using checklists based on the indicators established for each station of the OSCE (i.e., the clinical reasoning skill in physical therapy assessed in each station). In order to create these checklists (one per station), the committee considered the LOs of the course.
Regarding the stations used in the OSCE, three types of station were included based on the LOs of the course: a "scenario" station, which takes place in an environment that allows recreating procedures associated with an intervention; a "standardized patient" station, where an actor, with a standardized script, simulates being a patient in a given clinical situation, and a "mailbox" station, in which students must solve a clinical case and submit their solution by depositing it in a mailbox.
Once the pilot test was completed and all the adjustments suggested were made, the final version of the OSCE included 11 stations plus one rest station, each lasting 5 minutes.32,33 The main characteristics of the stations are described in Table 1.
Table 1 Main characteristics of the 11 stations of the Objective structured clinical examination developed for assessing clinical reasoning in physical therapy students.
Station # | Station name | Station type | Station objective | Station description | Assessed clinical reasoning skill in physical therapy |
---|---|---|---|---|---|
S1 | Hand washing | Scenario | To identify the standard measures to prevent healthcare-associated infections | The student must identify in the proposed scenario the distractive elements that could interfere with proper hand washing according to WHO standards34 | Cognitive, psychomotor and affective |
S2 | Safety in the healthcare setting | Scenario | To choose the adequate personal protective equipment based on the type of infectious agent | Based on the clinical case, the student chooses the best personal protective equipment and uses it properly according to the type of infectious agent and following the WHO standards35 | Cognitive, psychomotor, and affective |
S3 | Clinical interview | Standardized patient | To identify and record relevant aspects of the patient based on the clinical interview | The student must conduct an interview to the patient in order to retrieve information about their health condition | Cognitive, psychomotor, and affective |
S4 | Cardiorespiratory assessment # 1 | Standardized patient | To choose a cardiorespiratory monitoring element based on the patient's health condition | The student must monitor and assess the vital signs of the patient based on their health condition | Cognitive, psychomotor, and affective |
S5 | Musculoskeletal assessment | Standardized patient | To identify signs and symptoms of musculoskeletal disorders according to the health condition informed by the patient | The student must examine and evaluate joint movements based on the patient's health condition | Cognitive, psychomotor, and affective |
S6 | Neurological assessment | Standardized patient | To identify neurological signs and symptoms based on the health condition informed by the patient | The student must make an assessment of the central nervous system of the patient based on their health condition | Cognitive, psychomotor, and affective |
S7 | Cardiorespiratory assessment # 2 | Standardized patient | To identify cardiorespiratory disorders signs and symptoms based on the health condition informed by the patient | The student must auscultate the specific body area that needs to be examined taking into account the patient's health condition | Cognitive, psychomotor, and affective |
S8 | History taking | Mailbox | To organize the information collected in the medical record according to the description of the clinical case | The student, based on the description of the clinical case, must identify relevant information, organize it, and record it in a medical record form | Cognitive and affective |
S9 | Physical therapy diagnosis # 1 | Mailbox | To make a physical therapy diagnosis | The student must analyze the information available in the medical record and reach a physical therapy diagnosis using the ICF36,37 | Cognitive and affective |
S10 | Physical therapy diagnosis # 2 | Mailbox | To make a physical therapy diagnosis | The student must analyze the information available in the medical record and reach a physical therapy diagnosis using the ICF36,37 | Cognitive and affective |
S11 | Clinical problemsolving | Mailbox | To identify clinical problems using the RPS-form36,37 and the ICF36,37 and taking into account the available information | The student must analyze the available information to make a physical therapy diagnosis using the RPS-form36,37 and the ICF36,37 | Affective and cognitive |
ICF: International Classification of Functioning, Disability and Health; RPS-form: Rehabilitation problem-solving form; WHO: World Health Organization.
Source: Own elaboration.
On the other hand, students' clinical reasoning was evaluated using the above-mentioned checklists defined for each station. The maximum score to be obtained in the OSCE was 198 points, and the possible score range in every station was determined based on the number of indicators defined for the station and the complexity level of the skills addressed in each indicator. The final score of each student was calculated by adding the scores obtained in each station; a score >134 points (70% of the total score) was considered a passing score (Table 2).
Table 2 Scoring system of the objective structured clinical examination.
Station | Evaluation instrument | Indicators by station | Scores per indicator | Total Score |
---|---|---|---|---|
S1 | Checklist | 10 |
|
0-21 |
S2 | Checklist | 5 |
|
0-17 |
S3 | Checklist | 11 |
|
0-21 |
S4 | Checklist | 12 |
|
0-24 |
S5 | Checklist | 11 |
|
0-23 |
S6 | Checklist | 5 |
|
0-8 |
S7 | Checklist | 9 |
|
0-17 |
S8 | Rating scale | 8 | Each indicator has a score of 0= Not achieved, 1= Partial achievement and 2= Achieved | 0-16 |
S9 | Checklist | 8 |
|
0-15 |
S10 | Checklist | 8 |
|
0-15 |
S11 | Checklist | 14 |
|
0-21 |
Source: Own elaboration.
The OSCE was carried out at the end of course simultaneously in two rooms; students were divided into 7 groups and each group was given a turn to start participating in the examination, so that all students were able to complete it in a single day. In each station, instructions were given to the student, to the professor responsible for the assessment of the student and to the standardized patient (in the case of standardized patient stations); besides, students were given the checklist and materials required for completing the station. As mentioned above, students were asked to complete each station in a maximum time of 5 minutes each, including the rest station, thus, each student had a total of 60 minutes to complete the OSCE.
A total of seven professors (each assigned to one of the seven standardized patient and scenario stations for evaluation purposes), one logistics coordinator and a time keeper were required for carrying out the OSCE in each of the two rooms. Mailbox stations activities were evaluated after the OSCE was completed by all students.
Finally, once the test was completed, students were asked to enter an adjacent room and complete a student satisfaction survey.
OSCE satisfaction survey
In order to evaluate the degree of satisfaction of students with the OSCE methodology, they were asked to complete an OSCE satisfaction survey consisting of five statements, using a five-point Likert scale (1=strongly disagree, 2=disagree, 3=neither agree or disagree, 4=agree and 5=strongly agree).38-40
Statistical analysis
The scores obtained by the students in the OSCE and the satisfaction survey were entered into a Microsoft Excel 2016 spreadsheet. Descriptive statistics were used to analyze the data. Categorical variables were described using relative and absolute frequencies, while quantitative variables, using medians and interquartile ranges (25th and 75th percentile), since these data did not show a normal distribution as determined with the Shapiro-Wilk test of normality (significance level of p<0.05). All statistical analyses were performed using the Stata statistical software, version 13.0.
Ethical considerations
The study followed the ethical principles for conducting medical research involving human subjects established by the Declaration of Helsinki.41 In addition, the Bioethics Committee of the Faculty of Rehabilitation Sciences of the Universidad Andres Bello reviewed it and approved it, as stated in Project Certificate A.138, issued on July 1, 2020. Also, all participants signed an informed consent form.
Results
The median global score was 142 points (IQR: 132-150) and 61.1% of students obtained a passing score.
Upon analyzing the behavior of the scores obtained in each station, only S1 and S8 showed a distribution similar to the normal curve, while the remaining stations had an asymmetric distribution, where a higher frequency of high scores was observed.
Considering that the maximum scores per station varied between 8 and 24 points, median scores ranged from 8 to 21 points (Figure 1 and Table 3).
Perfect score and passing score frequencies by station ranged from 0% to 50.94% and from 2.52% to 96.86%, respectively (Table 3).
Table 3 Distribution of scores obtained in each station of the objective structured clinical examination.
Station | Total (n) | Max. Score (points) | Max. Score (% of students) | Passing score (points) | Passing Score (% of students) | Median (Points) | Interquatile range (p25 - p75) |
---|---|---|---|---|---|---|---|
S1 - Hand washing | 159 | 21 | 3.14 | 15 | 51.57 | 15 | 14 -17 |
S2 - Safety in the healthcare setting | 159 | 17 | 23.27 | 12 | 54.09 | 15 | 0 - 16 |
S3 - Clinical interview | 159 | 21 | 26.42 | 15 | 78.62 | 18 | 15 - 21 |
S4 - Cardiorespiratory assessment # 1 | 159 | 24 | 4.40 | 17 | 58.49 | 18 | 15 - 21 |
S5 - Musculoskeletal assessment | 159 | 23 | 29.56 | 16 | 96.86 | 21 | 19 - 23 |
S6 - Neurological assessment | 159 | 8 | 50.94 | 6 | 50.94 | 8 | 6 - 8 |
S7 - Cardiorespiratory assessment # 2 | 159 | 17 | 20.75 | 12 | 85.53 | 14 | 13 - 16 |
S8 - History taking | 159 | 16 | 0.00 | 12 | 2.52 | 6 | 4 - 7 |
S9 - Physical therapy diagnosis # 1 | 159 | 15 | 7.55 | 11 | 44.03 | 9 | 7 - 11 |
S10 - Physical therapy diagnosis # 2 | 158 | 15 | 8.23 | 11 | 42.14 | 9 | 9 - 11 |
S11 - Clinical Problem-solving | 158 | 21 | 0.63 | 15 | 57.23 | 15 | 13 - 17 |
Source: Own elaboration.
According to these results, S5, S7 and S3 were the stations where most students obtained a passing score (96.86%, 85.53% and 78.62%, respectively); on the contrary, the lowest passing score frequencies were observed in S8 (2.52%), S9 (44.03%), S10 (41.14%) and S11 (57.23%).
Regarding the OSCE satisfaction survey, when asked if the OSCE had been useful for their training as physical therapists, 48.43% and 38.35% of the students chose "agree" and "strongly agree", respectively. In addition, 45.91% agreed and 38.36% strongly agreed that taking similar tests improves their training, and 36.48% and 59.12% agreed and strongly agreed that taking tests that evaluate their clinical skills is important (Table 4).
Table 4 Results of the student satisfaction survey about the objective structured clinical examination (n=159).
Question | Strongly disagree (%) | Disagree | Indifferent (%) | Agree | Strongly agree (%) |
---|---|---|---|---|---|
Q1: The general organization of the OSCE is adequate | 12.58 | 22.64 | 13.21 | 44.03 | 7.55 |
Q2: The proposed stations were appropriate for my level of knowledge | 1.89 | 15.09 | 8.81 | 52.83 | 21.38 |
Q3: The OSCE has been useful for my training as a physical therapist | 2.52 | 4.4 | 6.29 | 48.43 | 38.36 |
Q4: Taking similar tests improves my training | 1.89 | 5.03 | 8.81 | 45.91 | 38.36 |
Q5: Taking tests that assess my clinical skills is important | 2.56 | 0 | 1.89 | 36.48 | 59.12 |
Source: Own elaboration.
Discussion
The objective of the present study was to describe the implementation of an OSCE as an adequate tool to evaluate clinical skills associated with the development of clinical reasoning in physical therapy students and to determine their level of satisfaction with this assessment methodology. However, it should be noted that the OSCE was implemented in second year students, who, given the study plan of the Physical therapy program at Universidad Andres Bello, had just started their clinical practice activities, where they began obtaining a certain degree of real-life professional experience. In this regard, the implementation of the OSCE allows measuring, in an adaptive manner, the clinical skills associated with the development of clinical reasoning according to the achievement of LOs by the student in early training stages.2,3,9,13,23
The OSCE allows for the integration of problem solving and clinical practice, which is crucial to provide physical therapy students with a knowledge basis that enables them to develop clinical reasoning and then to consolidate it throughout their professional training,1,12,42,43 thus the use of this methodology contributes to the strengthening of the training processes of future physical therapists.23,44
In the present study, it was observed that the scores obtained in standardized patient stations were higher than those obtained in other types of stations (S3= median score: 18, maximum score reached: 21; S5= median score: 21, maximum score reached: 23, and S7= median score: 14, maximum score reached: 17), which differs from the findings by Beom Park et al., 28 who, in a study conducted in 65 fourth-year medical students from a South Korean university, reported higher mean and maximum scores in the history taking station (71.6 and 88.7, respectively) compared to the physical examination station (46.6 and 72.1). This could be explained by the fact that asking and memorizing contents, without necessarily involving clinical reasoning, could imply a lower degree of difficulty.28 This reinforces the need to design each station properly, so that the student is able to use clinical reasoning and its components adequately during the implementation of the OSCE according to the LOs addressed in all of its stations.45-47
Several studies in which the OSCE has been used to assess clinical skills and specific skill associated with clinical reasoning have reported that students have better results in theoretical dimensions; for example, Bustamante et al., 48 in a study conducted in 13 Chilean medical interns, reported that the best results were found in stations where cognitive skills were predominant, as it was observed in the cardiac arrest station, in which the performance of students in the execution of the skills needed to deal with cardiac arrest was much lower than their performance when answering theoretical questions about cardiac arrest (66% vs. 98%), which differs from our findings, where the best results were observed in station S7 (Standardized patient - cardio respiratory assessment # 2), in which 85.53% students obtained a passing score; this difference could be explained by the fact that in our study, the design of both the standardized patient and the scenario stations integrated cognitive, psychomotor and affective skill associated with clinical reasoning in physical therapy, which implied higher levels of difficulty for the students, as they were faced with problem solving in clinical practice simulated scenarios.23,49,50
On the other hand, the worst performance was observed in mailbox stations (S8, S9, S10 and S11), which required cognitive or psychomotor skills, where less than 60% of students obtained a passing score (2.52%, 44.03%, 42.14%, and 57.23%, respectively); this situation may be associated with the fact that the reflection and writing process takes longer and is affected by the stress resulting from the evaluation.13,49
Likewise, another factor that could explain the students' poor performance in the mailbox stations is that the activities in these stations required to be performed according to a rehabilitation problem-solving form, the International Classification of Functioning, Disability and Health, and a biopsychosocial model, which implies a greater cognitive challenge.13,51 In addition, the time allocated for the completion of all stations (5 minutes for each station) could have affected the performance of the students in this type of stations, since it is likely they did not have enough time to fully carry out the activities in these stations, given the increased cognitive challenge they implied; however, it should be noted that the time given for the completion of each of the 11 stations of the OSCE in the present study is in line with the recommendations for the development of OCSEs.47,48,52-54
Regarding the students' level of satisfaction with the implementation of the OSCE methodology, 44.03% and 7.55% (i.e. more than 50%) reported they agreed and strongly agreed with the general organization of the tool. This is consistent with the findings of a study carried out in 2019 in Jamaica by Majumder et al. 53 in Bachelor of Medicine-Bachelor of Surgery students, where 66.7% and 14.8% (over 80%) of them agreed and strongly agreed with the use of the OSCE; however, it should be noted that only 54 students participated in the above mentioned study,53 which could have been a factor influencing these results, as the proportion of students who were satisfied with the OSCE in our study was somehow smaller, but our sample size was larger (n=159).
Furthermore, in the present study, 52.83% and 21.38% of students agreed and strongly agreed that the OSCE stations were adequate for their level of knowledge, which is also similar to what Majumder et al. 53 describe (66.7% and 18.5%). These findings would allow inferring that the OSCE is an evaluation tool that successfully addresses the contents learned during the Reasoning in Physical Therapy course of the Physical Therapy program at Universidad Andres Bello.
Regarding its relevance, 95.6% of the students considered that the OSCE was useful for the acquisition of clinical skills associated with the development of clinical reasoning, that is, less than 5% of the sample did not believe the OSCE was a useful tool for this purpose, which differs from the findings reported in the study by Majumder et al. ,53 where 29.7% of the students stated there was no relationship between the OSCE and the development of clinical skills (very little relationship: 20.4%, and no relationship at all: 9.3%), this difference could be explained by the fact that familiarization with the OSCE methodology, as it was the case of our study, favors its positive evaluation by students as an assessment instrument that promotes the acquisition of clinical skills and the development of reflective thought.40,53-56
Limitations
A limitation of the study could be attributed to the allocation of time and date for the execution of the OSCE, because it consisted of a single evaluation moment (with a limited amount of time per station), since assessing the performance of all 159 students in the OSCE in a single day was required due to the characteristics of the course. In this regard, despite the students' performance in the standardized patient and scenario stations was good in general terms, a marked poorer performance was observed in the mailbox stations, which could be due to the fact that these stations required students to engage in more challenging cognitive activities and it is likely that five minutes were not enough for their adequate execution. Therefore, we recommend to allocate more time for the execution of mailbox stations compared to other stations in future implementations of this OSCE, or, in case the overall time for the development of the OSCE is limited, decrease the total number of stations in order to provide students with more time for completing mailbox stations given their complexity.
Conclusions
The OSCE was successfully designed and implemented to evaluate the clinical skills associated with the development of clinical reasoning in physical therapy students, which confirms this is an excellent methodology to train and evaluate these students.
Moreover, a high level of student satisfaction with the use of the OSCE methodology was observed, since most students value its objectivity and the fact that the instrument may bring them closer to the development of clinical skills in situations similar to real-life clinical practice.
Finally, future studies should carry out this type of examination not only in physical therapy students, but also in students enrolled in other undergraduate health sciences programs offered at Universidad Andres Bello, for it will facilitate their professional development. Also, future studies must analyze if there is a correlation between the student's self-perception of their performance in the OSCE and their performance in this type of examination.