Introduction
Information and communication technologies (ICTs) offer the possibility of creating new spaces for interaction that facilitate teaching and learning processes. 1 In the specific case of face-to-face education, and through the application of an online learning environment (OLE), teachers have access to great tools for knowledge management. This is achieved by developing educational resources to evaluate and generate effective communication with students. The joint use of traditional teaching in classrooms with an OLE strategy is known as blended learning. 2
The progress of medical technology in the field of radiology has allowed it to acquire the dynamic characteristics of the technological world. As a result, education in the area of diagnostic imaging is a major challenge, not only because of the variety of techniques but also because of the rapid advances of these techniques. 3
This dynamic scenario implies significant challenges for professors, as they need to respond to cultural, social and technological trends. 4 Part of the challenge is the creation of online educational resources, known as digital learning objects (DLOs), which aim to create effective and innovative links with students. 5
This method is used both in the national and international context with multiple approaches and results; however, the motivation for developing this research project was to answer the question of how to measure the impact of the implementation of a digital learning environment.
In this sense, and as a response to the educational needs of the Introduction to diagnostic imaging course of the medical program offered by the Universidad Nacional de Colombia, ICTs were implemented, giving way to the possibility of measuring the impact on gained knowledge and the time of use of each digital tool. Consequently, the objective of this study was to design, implement and evaluate a blended learning method for teaching radiology to medical students.
Materials and methods
This study was approved by the Ethics Committee of the Faculty of Medicine of the Universidad Nacional de Colombia through Minutes No. 021-280-15 of December 10, 2015, followed the principles of the Declaration of Helsinki 6 and took into account the regulations of Resolution 8430 of 1993 of the Ministry of Health of Colombia. 7 Informed consent was also obtained from the participants to conduct the study.
The application of the blended learning method in the Introduction to diagnostic imaging course of the medical program was carried out in five phases, according to the ADDIE model: analysis of the institutional context, design and development of the DLOs, implementation, and evaluation of the pedagogical intervention.
Study design
A quasi-experimental and retrospective study was performed using a before-and-after design. The research was done with medical students from the Universidad Nacional de Colombia who took the course Introduction to diagnostic imaging during the academic terms 2016-II and 2017-I; they received a mixed learning methodology. The control group included the students who took the subject in the term 2015-II, for which a traditional pedagogical method was used.
Traditional teaching method
The contents of the course were developed through lectures given with an intensity of 1 hour per week; the evaluation was done through partial exams and a final exam with multiple-choice questions.
Blended learning method
The students had access to an online learning environment on the Moodle platform, which contained five learning objectives that were developed as modules. The number of visits to each module per student was assessed.
In addition, one-hour weekly lectures, partial exams and a final exam with multiple-choice questions were held, following the traditional method without changing the characteristics of the 2015-II face-to-face course. The contents and rules of the course, as well as the teacher in charge of designing the evaluation questions were the same before and after the intervention.
Design and development of digital learning objects
The modules were created based on the ADDIE model 8, which consists of an interactive design process where the results of each phase may lead the instructional designer back to any of the previous phases. In this way, errors are identified and corrected without hindering the progress of the project and the results can be constantly evaluated based on the learning objectives. Table 1 shows the definitions and the steps followed in each phase of DLO production.
Design and development
A digital learning environment was created on the Moodle platform with five DLOs (modules):
Module 1 : History of technology progress and clinical application of X-rays.
Module 2: Thoracic anatomy.
Module 3: Normal chest X-ray.
Module 4: Abdominal X-ray.
Module 5: Generalities of neuroimaging.
The features of the DLOs included the institutional image of the Universidad Nacional de Colombia, compatibility with all browsers and operating systems, optimization of navigation on multiple devices (computers, tablets and cell phones), fast download speed and ease for adding new learning modules.
Evaluation design
Two tests were carried out, one at the beginning and another at the end of the course. Both evaluations had 40 multiple-choice questions with one answer, which were designed based on the learning objectives proposed for the course (Figure 1).
Outcomes
In both groups, the primary outcome was the score obtained in the final exam (20 points). Gained knowledge was calculated in the blended learning group as the difference between the score obtained by each student in an exam at the beginning of the course and an exam taken at the end of the course.
The level of satisfaction was assessed through an anonymous end-of-course survey that included eight questions, of which seven were multiple-choice and one was open-ended.
Statistical analysis
Qualitative variables were described using percentages, while quantitative variables were described using measures of central tendency, mean for normal variables and median/interquartile range otherwise. Normality tests for continuous variables were performed using frequency histograms, box and whiskers plots and statistical tests (Kolmogorov-Smirnov, Shapiro-France and Shapiro Wilk).
Academic terms before and after the intervention were contrasted by means of multiple comparisons of the medians using the Kruskall-Wallis test. An analysis of the association between the frequency of use of the intervention and the outcome of knowledge gain by means of linear regression was also carried out, including the evaluation tests for the model (normality of residuals, linearity of the model and homoscedasticity). In addition, the correlation between knowledge gain and the grade of the final exam with the frequency of use of the intervention was evaluated using the Spearman coefficient.
Results
For the study, 294 students were included, of whom 90 subjects were in the traditional teaching method group (control) and 204 in the blended learning method group (intervention); the academic terms were 2015-11 for control and 2016-11 and 2017-1 for intervention.
Themedianofthefinal exam grade in the controlgroup was 15/20points (IQR 13.5-16.5) and in the intervention group, 16.5 (IQR 15.5-17.8). A statistically significant difference was found between the blended learning method groups compared to the traditional teaching method group. Table 2 presents the results for each term.
* Kruskal-Wallis test vs. 2015-11.
† Kruskal-Wallis test vs. 2016-11 -II vs. 2017-1, p=0.117.
Source: Own elaboration.
In the intervention groups, the number of visits to each DLO per student was recorded for each module during the implementation phase. The median number of visits was 21 (IQR 12-23) and the median number of visits for each module was between 3 and 4 (Figure 2).
Knowledge gain followed a normal distribution with an average of 5.9 (SD=2.4). The result of the linear regression between gained knowledge and the number of visits had statistical significance, with an increase of 0.22 points in the final exam for each visit to the digital modules (p<0.001).
The results of the satisfaction survey for blended learning students are presented in Table 3. Of the total number of students, 27 (13%) did not respond the survey; the percentage of respondents satisfied with the achievement of the learning objectives and the usefulness of the online modules was 99.4% for the 2016-II period and 100% for the 2017-I period.
Discussion
The results of this work demonstrate that the implementation of a blended learning method has a significant impact on student's grades and is perceived with high satisfaction rates.
Different studies have reported the positive impact of the blended learning method on academic performance. 2,9 The meta-analysis of Chumley-Jones et al.10 found sufficient evidence on the effectiveness of medical education combined with the use of digital tools. Furthermore, primary studies coincide in affirming that online instruction together with face-to-face training have a positive effect on the achievement of clinical skills and is comparable to traditional forms of teaching, obtaining better results in the final exam. 11-16
In the present study, each module was evaluated as an extrinsic motivation strategy, obtaining a considerable number of visits, higher than those reported by Mahnken et al.17 in a similar study and with greater gained knowledge.
Despite the standardization of knowledge tests, there were two main limitations in this research. First, the retrospective nature of the study did not allow obtaining data to assess knowledge gain in the control group, which would have been useful to control the differential bias of heterogeneity in student knowledge at the beginning of radiology courses. Second, the cohort bias intrinsic to the before-and-after design referred to differential external factors in each group assessed due to the effects of temporality. However, to control this bias, outcomes were measured at times not so distant from the control group, as well as in two different intervention cohorts. Cohort 2016-I was not taken into account as it was considered to be a transition between both learning methods.
The levels of satisfaction with blended learning have been described by authors such as Choules 16 and Carbonaro et al.18, who concluded that it has contributed to student self-development. Both researchers attributed the success of this method to the creative use of computer technology and the practical nature of the material, and proposed that medical educators should consider the blended learning approach to standardize clinical learning.
The high expectations of the students, exposed to constant innovations in all areas of knowledge, were evident in the satisfaction survey carried out within the framework of this study. Some interviewees stated that the DLOs were scarce and demanded the creation of new digital complements. In addition, a greater availability of teachers through virtual communication tools (chat) was suggested for the resolution of doubts.
Other reported benefits of the blended learning method are improved academic performance and student satisfaction, as it increases students' exposure to area content and optimizes teaching times. 19-21
On the other hand, all educational formats have strengths and limitations. Thus, the ideal methodology should aim to improve the perception on educational environments and to promote problem solving, critical thinking and decision-making skills, in a flexible manner, without a specific place or time. 22
When building a blended learning model, teachers should decide which part of the curriculum will be delivered face-to-face and which online, to avoid students who lack computer skills from feeling disadvantaged or frustrated when using OLEs. 23 The balance between face-to-face education and the digital component is highly relevant and depends on factors such as student level, electronic resources and teacher experience.
Although online learning is an established and effective approach in many medical schools, it should not replace traditional learning; therefore, blended learning is probably a better approach than web-based teaching. 24
This project has demonstrated that the integration of online teaching into traditional classes overcomes the constraints of time and place and improves the quality of education as it promotes the development of skills by expanding the concepts and resources provided by textbooks.
Conclusions
The blended learning method has a significant impact on performance during tests compared to the traditional method. The implementation of DLOs that complement face-to-face education makes it possible to strengthen the teaching process with high levels of satisfaction, justifying the time and resources required for their design and production.