DOI: http://dx.doi.org/10.15446/ing.investig.v36n3.54959

Effects of an educational intervention on the technical writing competence of engineering students

Efectos de una intervención educativa en la competencia de escritura técnica de estudiantes de ingeniería

J. J. Ramírez-Echeverry¹, F. A. Olarte Dussán², and A. García-Carillo³

¹ Electronic Engineer, Ph.D (c), Universitat Politècnica de Catalunya, BarcelonaTech (UPC), Spain. Affiliation: Associate Professor, Electrical Engineering Department of Universidad Nacional de Colombia - Bogotá. E-mail: jjramireze@unal.edu.co
² Electronic Engineering, Ph.D., Universidad Nacional de Colombia, Colombia. Affiliation: Assistance Professor, Electrical Engineering Department of Universidad Nacional de Colombia - Bogotá. E-mail: faolarted@unal.edu.co
³ Chemical Engineer, Ph.D., Universitat Politècnica de Catalunya, BarcelonaTech (UPC), Spain. Affiliation: Full Researcher Professor, Universitat Politècnica de Catalunya, Barcelona Tech (UPC), Spain. E-mail: agueda.garcia@upc.edu

How to cite: Ramírez-Echeverry, J. J., Olarte, F., & García-Carillo, A. (2016). Effects of an educational intervention on the technical writing competence ofengineering students. Ingeniería e Investigación, 36(3), 39-49. DOI: 10.15446/ing.investig.v36n3.54959.

ABSTRACT

This article presents the design, implementation and appraisal of an educational intervention developed to explore how education, practice and feedback of cognitive elements, regarding writing summaries, have an effect on the technical writing competence of freshman engineering students. The educational intervention was designed based on the methodology of Writing Across the Curriculum (WAC) and it consisted of three phases: teaching, practice and feedback. In total, 177 students participated, distributed into three groups: 54 Electronic Engineering students (year 2014), 57 Electrical Engineering students (year 2015) and 66 Electronic Engineering students (year 2015). The intervention effects were studied by quantitative and qualitative evidence. Quantitative evidence was collected through an evaluation rubric of the summaries written by the students; this rubric analyzes ten criteria of the writing competence. Qualitative evidence was collected through open-ended questions about the students' learning experience throughout the intervention. Results show that the three participating groups improved their technical writing competence due to their participation in this educational intervention. Sentence construction (g-Hedges = 0,62) and text sequences (g-Hedges = 0,59) criterion presented the highest effect rates. The other writing criteria, evaluated through the rubric, presented a positive gain with statistical significance, but only in some groups of students.

Key words: Information literacy skills of engineering students, information documentation, engineering technical writing, writing across the curriculum, engineering education.

RESUMEN

Este artículo presenta el diseño, implementación y evaluación de una intervención educativa para explorar cómo la instrucción, práctica y retroalimentación de elementos cognitivos para la escritura de resúmenes impactan la competencia de escritura técnica de una población de estudiantes de primer año de ingeniería. El diseño de la intervención educativa se basó en la metodología de escritura a través del currículo (WAC, en inglés) y se compone de tres fases: instrucción, práctica y retroalimentación. En total participaron 177 estudiantes, distribuidos en tres grupos: 54 estudiantes de Ingeniería Electrónica (año 2014), 57 estudiantes de Ingeniería Eléctrica (año 2015) y 66 estudiantes de Ingeniería Electrónica (año 2015). Los efectos de la intervención se exploraron con evidencias cuantitativas y cualitativas. Las evidencias cuantitativas se recolectaron con una rúbrica de evaluación de los resúmenes elaborados por los estudiantes; dicha rúbrica analiza diez indicadores de desempeño para la competencia de escritura. Las evidencias cualitativas se recopilaron con preguntas de respuesta abierta que interrogaban a los estudiantes sobre la experiencia de aprendizaje vivenciada en la intervención. Los resultados indican que los tres grupos de estudiantes participantes mejoraron su competencia de escritura técnica. Los mayores tamaños de efecto de la intervención los presentaron los indicadores construcción de oraciones (g-Hedges = 0,62) y secuencialidad del texto (g-Hedges = 0,59). Los demás indicadores de escritura, evaluados con la rúbrica, presentaron ganancia positiva con significancia estadística, pero sólo en alguno(s) de los grupos de estudiantes participantes.

Palabras clave: Habilidades para el manejo de la información de estudiantes de ingeniería, documentación de la información, escritura técnica de los ingenieros, escritura a través del currículo, educación en ingeniería.

Received: January 4th 2016 Accepted: November 24th 2016

Introduction

The purpose of technical writing is to communicate ideas clearly and concisely to the reader. Therefore, it is an important communication competence for any professional, as written language enables creating ideas and exchanging them with other people (Winsor, 1990). Some studies about engineering writing suggest engineers spend a lot of time writing documents (Pneena & Romanowski, 2001; Gimenez & Thondhlana, 2012). According to Winsor (1990) "writing is what engineers do". In the work context, writing enables the engineer to communicate with their peers, to be effective at their work activities, reach high positions and gain self- confidence (Pneena & Romanowski, 2001). In an educational context, through writing, the engineer achieves many goals: representing physical realities in order to create or exchange knowledge, understanding completely the study topics, developing critical thinking and reflective capability, connecting ideas orderly, among others (Winsor, 1990; Wheeler & McDonald, 2000).

This paper presents the design, implementation and appraisal of an educational intervention, based on the WAC methodology. The intervention was developed to study the effect of education, practice and feedback of cognitive elements with regard to writing summaries, on the technical writing competence of a group of freshman engineering students.

The cognitive elements regarding writing summaries are the thinking activities required to summarize a technical text. Activities such as organizing and ranking information, identifying the main idea of a text, deciding what information to omit or replace, setting out the textual and grammatical structure of the summary appropriately, and making a proper layout of the document (Fregoso & Aguilar, 2013), are some of the thinking activities involved in text summarization. These cognitive elements were taught to students who participated in this educational intervention.

It was decided to work in the elaboration of summaries of engineering technical texts for several reasons: (1) the written summaries are highly used by engineers (Horowitz, 1986). (2) The teaching and practice of writing summaries has shown to foster general aspects of the writing competences, such as wording (Taylor & Beach, 1984), grammar, and spelling (Fernandes, 2012; Yang, 2015). (3) The elaboration of summaries encourages the ability to find, evaluate, and use information. Engineers need to develop these abilities in order to solve open-ended problems and document the design process (Van Epps, Fosmire, Wertz, & Purzer, 2013). (4) The elaboration of summaries encourages the ability of selecting and organizing the information read. This ability, considered as a learning strategy (McKeachie, Pintrich, Lin, & Smith, 1987), fosters the students' meaningful learning, improves the academic performance (Marugán, Martín, Catalina, & Román, 2013) and facilitates the construction of connections between the ideas to be learned (Beltrán, 2003).

In this intervention the students also learned and used bibliographic citation and referencing techniques to elaborate their summaries. Some studies have found that first-year engineering students have difficulty citing several kinds of resources in their bibliographies, for example, citing Web resources in their reports (Wertz, Purzer, Fosmire, & Cardella, 2013). In other cases, the students cite the sources correctly, but without any reference to them in the text. These results indicate that the documentation skills of engineering students are an area that requires further research (Wertz, Purzer, Fosmire, & Cardella, 2013).

Literature review

The engineering education literature classifies into four categories the information literacy: information gathering, information evaluation, information application, and information documentation (Wertz, Purzer, Fosmire, & Cardella, 2013). The technical writing fits into the information documentation category.

For several decades, writing in engineering has been studied. In the literature, there are studies of methodologies developed to include this competence in the curriculum (Beaufort, 2007), studies of how the engineers write (Winsor, 1990), identification of the most common writings of engineering (Zhu, 2004; Horowitz, 1986), the importance of the writing competence in engineering education (Zhu, 2004), among others. However, researchers suggest that graduates of these careers still have insufficiencies when writing (Nair, Patil, & Mertova, 2009; Armstrong, Dannatt, & Evans, 2012; Goldsmith, Willey, & Boud, 2012; Christiansen, et al, 2014; Wertz, Purzer, Fosmire, & Cardella, 2013).

Likewise, researchers set out the necessity of proposing new initiatives to discover empirical and practical evidence of how the engineering students can foster this competence (Loveland, 2014; Amos & McGowan, 2012; Peña-Reyes, 2011). Since several of the problems identified throughout the development of engineering students' writing competence are shared in many educational syllabuses (Fregoso & Aguilar, 2013; Solis & Abad, 2004), it is expected that the methods employed and answers found in this intervention contribute to the design of new initiatives to foster this competence. The authors of this paper made previous studies that set the bases for this educational intervention (Ramírez-Echeverry & Olarte, 2013; Ramírez-Echeverry, Olarte, & García-Carrillo, 2014).

Design of the educational intervention

Technical writing is a complex competence that requires general and specific competences. General skills, common to any discipline, include being aware of the audience, organizing the text sequentially, constructing sentences appropriately, developing structured paragraphs, using grammar and spelling appropriately, among others (Zhu, 2004; Lea & Street, 1999). The specific skills consist of knowing how to use linguistic and rhetorical conventions of a discipline; they involve understanding the communication and thinking processes related to a specific specialty (Zhu, 2004; Hyland, 2000). For example, in engineering it is common to write documents with mathematical nomenclature, graphics, figures, block diagrams, among others (Winsor, 1990; Zhu, 2004).

This implies that an educational strategy, designed to foster technical writing on students, should be based on methodologies that include teaching and practice of general and specific elements of writing (Zhu, 2004; Beaufort, 2007). A methodology that includes both elements is Writing Across the Curriculum (WAC). Some studies on engineering that employed WAC methodology are Pneena & Romanowski (2001), Bazerman et al., (2005) and Fernandes (2012). WAC proposes "...the strategic integration of carefully designed writing tasks in any content area to serve the ends of learning, authentic communication, personal engagement, and reflective authorship" (Brewster & Klump, 2004 quoted by Fernandes, 2012). In other words, the WAC suggests learning how to write in context with the distinctive requirements of the knowledge area. These statements imply that:

• The pedagogical attention to learn general writing competences should be focused on the disciplinary courses and not only on specific courses (Bazerman, et al., 2005).

• The students seeking the subjects to foster the writing competence should count on the participation of experts on general writing and professionals of the discipline the students are learning.

• The writing, teaching and practice should be focused on disciplinary requirements; for example, learning how to elaborate the most common writings of the career (Bazerman et al., 2005).

The educational intervention implemented was an "informed-training" strategy: the teachers made clear for the students the subject goals and the intentionality of the proposed activities. The intervention had three phases: teaching, practice and feedback (see Figure 1). The innovation of this structure is that it connects several activities recently suggested in other studies (in a separated way), to encourage writing technical summaries (Fernandes, 2012; Yang, 2015).

Teaching phase

This phase starts with a contextualization (step 1, Figure 1). Contextualization is made through an introduction, and its purpose is to motivate and persuade the students to develop technical writing competence. The introduction is conducted by an engineer, during one hour, giving examples that evidence the necessity of writing in engineering academic and work contexts; examples to illustrate the discursive nature, and distinctives of writing in engineering, are also shown during this phase.

The second activity in the teaching phase is the introduction to methods for summarizing, making bibliographic citations and referencing in technical documents (step 2, Figure 1). The method for summarizing involves two processes: reading comprehension and textual production. For reading comprehension, students are advised to recognize the text structure and identify the logical connectors. As a result of this process, it is expected to achieve the hierarchy of ideas. Therefore, the following activities were proposed to be developed by the students:

• Conduct an exploratory reading to get the global idea of what the text is about. Likewise, identify the previous knowledge required to comprehend the text.

• Conduct a new reading in order to identify the thematic content of each paragraph, considering it individually. For this reading, the usefulness of classifying each paragraph according to its intentionality in the whole text is highlighted.

• Read again to identify the main idea of each paragraph and the secondary ideas (hierarchy of ideas). For this task, it is suggested to underline and pay attention to the connectors and their function in the text. At the end of this reading, the student should have identified and selected the main idea of the text as a whole, and the ideas that can be omitted or included in the summary.

The activities above are classified as techniques for selecting information from a text. These techniques have been developed from research about reading for the study and understanding of technical texts. Regarding that, the studies developed by Cook & Mayer (1983), Mayer & Bromage (1980), and Derry & Murphy (1986) stand out. It has been proved that high-skilled students at writing summaries used to select and organize the information before writing the summary (Bean & Steenwyk, 1984). Researchers also have found that less-skilled students at writing summaries can be benefited from the direct teaching on strategies for selecting and organizing information (Bean & Steenwyk, 1984; Mayer R. , 2002; Ponce, Mayer, & Mario, 2013).

The second process suggested to the students for elaborating a summary, is the textual production. In this process, the original text is transformed into another one that reflects generally and briefly the main ideas of the text being summarized. It is done by organizing the information selected during the reading comprehension phase and by writing the summary. While organizing the information, the hierarchy and intentionality of the original text should be preserved. The writer can employ techniques while writing, in order to provide coherence to the information; for example: employ substitution methods such as paraphrasing and usage of subordination and conjunction.

The writing pragmatic elements (Fregoso & Aguilar, 2013) were considered by asking the students to be aware of the intended audience, goals, and the context of this communicative activity. Students were asked to use an appropriate language in their summaries, considering they would summarize scientific texts and their readers would be linguists and engineers. The introduction to reading comprehension and production takes an hour of class.

Regarding bibliographic citation and referencing, information about the importance of respecting the intellectual property and methods for bibliographic citation and referencing are presented to the students, during an hour of class.

Step 2 in the teaching phase (Figure 1) concludes with an exercise in which the students receive a text and are asked to summarize it, employing for the first time the method suggested in class. The teachers also summarize it and present their version to all of the students. The students can compare their version to the teachers'; and give their opinion about the quality of their own summary and the one that teachers made. This exercise enables to emphasize that the summary of a text can have more than one acceptable version (Wheeler & McDonald, 2000). This activity takes an hour of class.

The third activity in the teaching phase (step 3, Figure 1) is socializing with the students the criteria for the evaluation of summaries; those criteria are grouped into an evaluation rubric of ten criteria with their level descriptors (see Appendix 1). This introduction takes an hour of class.

Practice and feedback phases

The writing practice was made by the students through the reading and elaboration of summaries of engineering technical texts (step 4, Figure 1). Those texts were about electrical engineering and electronics engineering topics, such as renewable energy, energy solutions in noninterconnected areas of Colombia, smart grids, wireless telecommunications networks, industrial automation, among others. Engineering teachers suggested the texts, and the students could choose the topic to read and make the summary. In order to benefit reading comprehension, it was taken into account that the texts contents matched the students' knowledge level. The texts were sent to the students via e-mail two weeks before the summary submitting date.

The feedback phase started with an academic counseling made for two hours by the linguists and engineers (readers), a week before submitting the summary. In that session the students consulted and solved doubts about the summary elaboration (step 5, Figure 1).Thus, meetings between writers (students) and readers were fostered, so they could discuss the content of the summaries (Fregoso & Aguilar, 2013) . The teaching and feedback given to the students came from experts on general writing competences (linguists) and experts on engineering technical language (engineers).

This first dialogue between writers and readers enabled exchanging points of view about main and secondary ideas of the texts (dialogues about the content). It also helped each student to refer to their concerns about textual and grammatical structure and about the layout proposed for their document (dialogues about writing). From the date of this first qualitative feedback, the student had a week to reflect, correct and/or adjust the writing of their summary (step 6, Figure 1), before submitting it for its evaluation.

Evaluation of the quality of the summaries was conducted by linguists and engineers using a rubric (see Appendix 1). Once the summary was evaluated, each student got the results as written comments and the rubric filled in. The evaluation results were given personally, so the writers and readers could discuss for a second time the summary quality and the written exercise results (step 7, Figure 1). Finally, the student wrote again the summary, making modifications from the results of the qualitative and quantitative evaluations and presented a final version of the summary (step 8, Figure 1).

In this educational intervention the students elaborated three summaries in total, thus, the cycle described above (step 4 to 8) was performed three times.

Methods

Context and participants

This educational intervention was a learning strategy brought together with the objectives of two subjects of the Universidad Nacional de Colombia - Bogotá: Introduction to Electrical Engineering and Introduction to Electronics Engineering. Both subjects belong to the first year and expect the student to recognize contexts and scopes of the engineering professional practice and to foster the written communication competence.

In total, 177 students participated, distributed into three groups: Group 1: 54 students of Electronic Engineering (year 2014) , Group 2: 57 students of Electrical engineering (year 2015) and Group 3: 66 students of Electronic Engineering (year 2015). The participants had an average age of 18,9 years (standard deviation: 2,7 years); 13 were women and 164 were men.

Instruments for collecting evidence

The results of this intervention were analyzed based on quantitative and qualitative evidence. The quantitative evidence was collected through an evaluation rubric of the summaries, and the qualitative evidence was collected through open-ended questions that asked the students about their learning experience throughout this intervention.

The performance level of the students was analyzed through the rubric considering ten criteria of the writing competence. The evaluation rubric was analytical. The criteria were spelling and punctuation (grammatical structure), paragraphs and sentences (textual structure) purpose and complementary ideas (discursive structure), format and sequentiality (layout), audience, tone and bibliographic references (communication ethics). The performance levels and descriptors for each criterion are presented in Appendix 1.

A rubric for each one of the three summaries presented by the students was filled in. This allowed tracking the writing competence progress for each student as they practiced writing summaries. The rubrics results also contributed to the dialogue between students and teachers during the feedback stages (second dialogue). The rubric employed was adjusted and validated in pilot tests by linguists and engineers, before being used in this intervention.

Open-ended questions asked to the students about their learning experience throughout the intervention were answered voluntarily for each summary, and had no influence over the summaries' final scores. The questions are the following:

• Did you use the method suggested in class to elaborate the summary? If you did, what aspects do you consider innovative?

• Do you consider the activities you did to write this summary are useful to increase knowledge on topics of this subject, or to improve your competence on writing?

Data analysis

The performance level of each student on writing criterion was ranked from five to one: five as an "expert" level and one as an "unfamiliar" level. The performance average of each criterion was obtained for the students of each group participating in the intervention (Groups 1, 2 and 3) and for the entire population (all of the groups). Then, for each criterion, the difference (gain) of performance level between the first and the last summary was calculated. It was verified through t-test if the differences discovered were significant (gain with statistical significance). Since one of the interests of this research is to know the level of the writing competence of students at the beginning and the end of the academic semester, the analyses were limited to calculate the differences between two time points. The difference was calculated for the students of each group and the entire population.

Furthermore, the effect size of the intervention was calculated for each criterion through the g-Hedges measure (intrasubject comparison). The g-Hedges was employed because the amount of students that presented the first and last summary in each group, was diverse (Iraurgi, 2009). The effect sizes enabled identifying what writing criteria the students exposed more or less.

The qualitative analysis for the answers related to the opinion-based questions started by sorting them according to the summary in which the student gave their opinion and to the question the student was answering. In total, there were 125 written answers: 33 students answered in the first summary, 36 in the second one and 56 in the third one. The procedure for analyzing the answers was conducted according to the recommendations for the qualitative analysis of open-ended questions (Zhu, 2004).

Results

Quantitative data

Table I presents the tracking results, for each criterion, of the technical writing competence. Initially, the tracking was conducted with the entire participating population, without discriminating by groups (Groups 1, 2 and 3). In the first summary (column Average first summary, row All) the students' performance was set at an "intermediate" level. The sentences criterion had the lowest value, with an average of 3,40. The criterion of audience and tone had the highest value, with an average of 4,10. In the last summary (column Average last summary, row All) it was found that the students increased their performance, in most of the criteria, to the "proficient" level. Averages between 3,74 (punctuation criterion) and 4,44 (format criterion) were found. By analyzing the performance gain within the criteria and the p-value of those gains for all the students (columns Gain and p-value, row All), without discriminating by groups, it was concluded that all the writing criteria increased significantly.

TABLA 1

Notwithstanding, if the analysis of gain in the criteria performance and the p-value of the gain for each group participating (column Gain and rows Group 1, 2 and 3) is conducted, it is evident that in some criteria a number of groups of students didn't have an increase with statistical significance. This result suggests that to get an accurate identification of the educational intervention effect, it is convenient to analyze the results not only with the entire population, but also by groups.

It was decided to track the criteria discriminating by groups of students (Groups 1, 2 and 3). The results for each criterion, for each group, are shown in columns Average first summary and Average last summary in Table 1. With the purpose of exploring global conclusions, it was verified in how many groups, of three possible, the criteria presented performances with significant gain (p-value of the t-test). The results are shown in the column Number of groups with significant gain. It was found that the three groups participating in the intervention improved, with statistical significance, in the criteria of sentences and sequentiality. Two of the three groups improved in the criteria of paragraphs, format, audience and tone, spelling and complementary ideas. The criteria of purpose, references and punctuation were improved only in certain students of some groups. The analysis above could suggest that teaching, practice and giving feedback to the summaries written by the students during this educational intervention contributed to the writing competence in different proportions.

The column g-Hedges in Table I presents the effect size for each criterion, calculated with the gain for groups and for all the students. The last column in Table 1 presents the rank of each criterion, according to the effect size calculated for all the students. Based on these columns, it can be concluded that the criteria of sentences (g- Hedges = 0,62) and sequentiality (g-Hedges = 0,59), the same all the groups participating improved, revealed the largest effect sizes; while the criteria of complementary ideas (g-Hedges = 0,38) and punctuation (g-Hedges = 0,35) revealed the smallest effect sizes. In general, the effect sizes that resulted from this intervention were set between moderate and excellent (Ponce, Mayer, & Mario, 2013; Hattie, 2013).

Qualitative data

Students mentioned several aspects while sharing their perceptions about the learning experience in this class. Due to the scope of this research, only some of the perceptions related to the research question will be exposed. Some segments are literal quotes of comments made by the students.

Teaching impacts: The students recognized the importance of introducing a method for summarizing. They suggested that before participating in this experience, they summarized texts without implementing techniques, which led to a complexity for doing the summarizing exercise or to a low quality of the text. The students rated the method positively, because they considered it useful and it allowed them to recognize that previous knowledge of how to do summaries was insufficient. An example of this perception was as follows:

"If I had not had a guide about how to elaborate a summary and its parts, I would not have gotten this result, which I consider good. After learning this method, I consider I did not know how to summarize; thanks to what I have learned in class, the exercise has been pretty easy for me..." (Student 7, summary 1).

Most of the perceptions that students had about the teaching phase were related to bibliographic citation and referencing techniques, the method for selecting and organizing information (hierarchy of ideas) and techniques to provide sequentiality to the text. The students mentioned they did not know those techniques and their implementation allowed avoiding plagiarism and made writing easier. For example:

"The topic related to plagiarism and how to avoid it is among the aspects I considered more innovative. Also, I found very interesting learning how to reference correctly." (Student 47, summary 3).

"...It's complicated to read three or four documents and then identify the ideas of everything I read; it's easier and more organized if I pull out the ideas as I read, and then get the hierarchy, it also concludes in a better analysis of the text." (Student 81, summary 1).

In regard to the evaluation presented during the teaching phase, the students said it was useful as a self-evaluation tool. They informed that while checking their evaluations, with the rubric criteria, they could identify their errors and modify their writings before submitting the summary for the teachers to read and evaluate it.

Practice and Feedback impacts: Most of the comments made by the students were about the practice phase. They indicated that by implementing and practicing strategies to select (hierarchy of ideas) and organize information, they could acquire competences to synthetize information and write better summaries. They also concluded practice allows improving their general writing skills, such as wording (construct sentences and paragraphs), sequentiality of the texts, punctuation and bibliographic references. For example:

"These practices helped me strengthen my writing competences, since they improved my ability to find relevant parts within a text, to use punctuation marks correctly, to have an accurate sequentiality throughout writing, and contributed to broaden my critical thinking." (Student 51, summary 3)

Through the qualitative analysis an emergent category was found that was codified as "writing to learn". The students frequently expressed that by summarizing they could learn deeply the topic they read and summarized. Some students suggested the key process for this deep learning was the reading comprehension, succeeded through implementing strategies to select and organize information from the texts. This result could confirm the influence of the summarizing technique on the learning processes.

With regard to the feedback phase, the students expressed that the dialogues between readers (teachers) and writers (students), as well as the quality evaluation through the rubric, helped them to identify their flaws and errors in the writings. Likewise, they argued that most of the guidance requested to the teachers, was related to constructing sentences and paragraphs, sequentiality of the text and punctuation.

The analysis of perceptions related to the feedback also produced an emergent category: "metacognitive experience/ feeling control" (Paz-Penagos, 2011). This category means that students gained experience to regulate their cognitive processes in order to be efficient in the elaboration of summaries. Students emphasized the benefits of making three summaries to gain the mentioned experience. The first feedback for the first summary helped the students planning, tracking and adjusting the way they would do the next two summaries. For example: they planned more efficiently the time spent on elaborating the second and third summary, identified weaknesses in their writing competence and analyzed if they improved as they did the exercises during the semester (learning tracking). The students perceived constructing sentences and paragraphs as the criteria they promoted the most; likewise, they expressed that during the second and third summary they felt more confident with the technical language related to the topic, due to the fact that they got familiarized with the vocabulary by writing the first summary (learning tracking).

Conclusions

This experience demonstrated that students promoted (positive gain with statistical significance) aspects belonging to their technical writing competence in different proportions, after participating during the process of teaching, practice and feedback of cognitive elements, related to writing summaries. This result would indicate that the educational intervention designed benefits some writing aspects more than others. Also, the current design may require adjustments to improve the performance of some criteria.

Constructing sentences correctly and organizing sequentially were the criteria with the highest gain. They also were the only criteria improved by all the students of every participating group. The effect size for the criteria of sentences (g-Hedges = 0,62) and sequentiality (g-Hedges = 0,59) are similar to the results found by other researchers. An example of the above is the effect sizes widely analyzed by Graham & Perin (2007) in their meta-analysis on educational interventions around the world to encourage the writing competence.

The positive effect in constructing clear sentences, paragraphs with appropriate oneness, ideas connected with polished transitions, and the appropriate sequentiality of the text, was a result, to a great extent, of the method of reading comprehension suggested. Students emphasized that knowing and practicing a reading method that enables them to select and organize ideas, as well as recognizing connectors and their function in the text, was useful to learn how to construct their writings. This result suggests the students learned how to write summaries from learning to read (reading comprehension). This finding is consistent with other research about writing (Alharbi, 2015; Cunningham, 1994).

The criteria of purpose, references and punctuation improved only in one of the three participating groups. These were the criteria with the lowest gain. These results indicate that the present intervention must be modified in order to strengthen these criteria.

In regard to bibliographic citation and referencing, it was found that only the students of one group improved significantly this criterion. The students' perceptions led to the conclusion that, for most of the students, using references was a new aspect for constructing writings. The rubrics results indicated that the reason for deficient quantitative results was that the students couldn't implement correctly the method of citation suggested (citation format of Institute of Electrical and Electronics Engineer, IEEE). Students used bibliographic citations to avoid plagiarism, but made mistakes while using the format. A common error was to reference their information sources without providing fields necessary to trace the original sources. These kind of difficulties were also found by Wertz, Purzer, Fosmire & Cardella (2013) among engineering freshmen students. It can be concluded that educational intervention allowed the students to recognize the importance of using bibliographic references; however, it is necessary to reinforce teaching related to the citation and referencing format at any phase of the intervention.

In general, the results proved the use of strategies to select and organize ideas eased the significant learning on the students. The participants expressed they learned deeply about the engineering topic they summarized, because the methods to select (hierarchy) and organize ideas provided them with a high level of reading comprehension. Furthermore, the participants considered that writing summaries is useful not only to learn how to write well, but as a strategy to study more deeply topics from other subjects. These perceptions are consistent to the conclusions in other researches (Wheeler & McDonald, 2000; Beltrán, 2003; Alharbi, 2015).

As a practical contribution of this intervention, it was found that the teaching phase enabled the students to learn a method to summarize. The practice phase helped the students to improve some criteria on their technical writing competence and the significant learning of the topics they wrote about. The feedback phase encouraged the metacognitive reflection on the students; the dialogues between writers and readers (social nature of writing) helped the student to identify and recognize their weaknesses and strengths on the writing competence. The tracking conducted through the evaluation rubric helped the students to adjust their reading and writing methods and notice if they had improved throughout the semester.

Finally, engineering teachers participating in this research could evidence the advantages of designing and implementing this educational intervention as a team, with professional linguists. Thanks to the exchange of knowledge between the two fields, the satisfactory integration of academic tasks into a subject of the engineering basic knowledge core was possible, in order to encourage the writing competence in students. This collaboration between fields, basis of the WAC methodology, has also been suggested in other research (Zhu, 2004; Cunningham, 1994).

Limitations and Future Research

The effort put into implementing an experience like the one presented in this research made it difficult to take into account some design aspects. For example: (1) the lack of a control group that practiced writing summaries without participating in the instruction and/or feedback phases of this intervention. Measuring the writing evolution of the students who were members of the control group, would clear up better the contribution of each phase (teaching, practice and feedback) to the students' writing. (2)The fidelity of the method implemented in the participating groups of the intervention was not measured, nor controlled. To unify criteria, the teachers guiding the activities conducted several meetings, however, it was the only precaution taken into account.

For future research, the proposal is to extend the scope of this research, by measuring the impact this intervention has over the extra-class writing competence of the students; it would be interesting to know if the students keep using the techniques learned in this class in the following semesters. Another extent would consist on creating initiatives for additional subjects of the curriculum. The students could continue encouraging the writing competence from the competence level reached in their freshman year. The genuine improvement on the technical writing competence is the result of a curriculum designed carefully and systematically, complemented with the practice of characteristic texts for each career (Cunningham, 1994). Finally, it could be useful to incorporate the usage of Information and Communication Technologies (ICT) in any phase of the method proposed.

Acknowledgements

This research was supported by Fundación Carolina (Spain) through a doctoral scholarship granted to the Engineer Ramírez, co-author of this research. The researchers acknowledge the students participating in this research and the linguists for their contributions in the design and implementation of this educational intervention.

References

Alharbi, F. (2015). Writing for Learning to Improve Students' Comprehension at the College Level. English Language Teaching, 8(5), 222-234.         [ Links ]

Amos, K., & McGowan, U. (2012). Integrating academic reading and writing skills development with core content in science and engineering. Journal of Learning Development in Higher Education, Special Edition: Developing Writing in STEM Disciplines, 1-24.         [ Links ]

Armstrong, M., Dannatt, J., & Evans, A. (2012). The development of, and response to, an academic writing module for electrical engineers at the University of Bath. Journal of Learning Development in Higher Education, Special Edition: Developing Writing in STEM Disciplines, 1-15.         [ Links ]

Bazerman, C., Little, J., Bethel, L., Chavkin, T., Fouquette, D., & Garufis, J. (2005). Reference Guide to Writing Across the Curriculum. Parlor Press LLC. Retrieved from http://wac.colostate.edu/books/bazerman_wac/.         [ Links ]

Bean, T., & Steenwyk, F. (1984). The Effect of Three Forms of Summarization Instruction on Sixth Grader's Summary Writing and Comprehension. Journal of Literacy Research, 16(4), 297-306.         [ Links ]

Beaufort, A. (2007). College Writing and Beyond: A New Framework for University Writing Instruction. Logan, UT: Utah State University Press.         [ Links ]

Beltrán, J. (2003). Estrategias de Aprendizaje. Revista de Educación(332), 55-73.         [ Links ]

Brewster, C., & Klump, J. (2004). Writing to learn, learning to write: revisiting writing across the curriculum in northwest secondary schools. Portland: OR: Northwest Regional Educational Laboratory.         [ Links ]

Christiansen, M., Dave, E., Hanson, A., Jenson, J., Ojard, S., Saftner, D., & Teasley, R. (2014). Development of Department Writing Guide for Civil Engineering. ASEE North Midwest Section Conference. Iowa.         [ Links ]

Cook, L., & Mayer, R. (1983). Reading Strategies Training for Meaningful Learning from Prose. In M. Pressley, & J. Levin (Eds.), Cognitive Strategy Research: Educational Applications (pp. 87-126). New York: Springer-Verlag.         [ Links ]

Cunningham, S. (1994). Learning to Write and Writing to Learn: Integrating Communication Skills into the Computing Curriculum. Software Education Conference, (pp. 306-312). Dunedin.         [ Links ]

Derry, S., & Murphy, D. (1986). Designing Systems that Train Learning Ability: from Theory to Practice. 56(1), 1-39.         [ Links ]

Fernandes, A. (2012). Writing to learn writing skills - a case study. 37(2), 179-192.         [ Links ]

Fregoso, G., & Aguilar, L. (2013). Comprensión y producción de textos como estrategias cognitivas y metacognitivas en la educación contemporánea. Tepatitlán de Morelos, México: Universidad de Guadalajara.         [ Links ]

Gimenez, J., & Thondhlana, J. (2012). Collaborative writing in engineering: Perspectives from research and implications for undergraduate education. European Journal of Engineering Education, 37(5), 471-487.         [ Links ]

Goldsmith, R., Willey, K., & Boud, D. (2012). How can writing develop students' deep approaches to learning in the engineering curriculum? Profession of Engineering Education: Advancing Teaching, Research and Careers, The: 23rd Annual Conference of the Australasian Association for Engineering Education 2012. Melbourne.         [ Links ]

Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99(3), 445-476.         [ Links ]

Hattie, J. (2013). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.         [ Links ]

Horowitz, D. (1986). What professors actually require: Academic tasks for the ESL classroom. TESOL Quarterly, 17(3), 445-462.         [ Links ]

Hyland, K. (2000). Disciplinary discourses: Social interactions in academic writing. Essex, England: Pearson Education Limited.         [ Links ]

Iraurgi, I. (2009). Evaluación de resultados clinicos (II): Las medidas de la significación clinica o los tamaños del efecto. Norte de Salud Mental(34), 94-110.         [ Links ]

Lea, M., & Street, B. (1999). Writing as academic literacies: Understanding textual practices in higher education. In C. Condlin, & K. Hyland (Eds.), Writing: texts, processes and practices (pp. 62-81). London: Longman.         [ Links ]

Loveland, T. (2014). Incorporating disciplinary literacy in technology and engineering education. Technology and Engineering Teacher, 8-13.         [ Links ]

Marugán, M., Martin, L., Catalina, J., & Román, J. (2013). Cognitive elaboration strategies and their content nature in university students. Psicología Educativa, 19(1), 13-20.         [ Links ]

Mayer, R. (2002). Psicología de la educación: el aprendizaje en las áreas de conocimiento. Madrid: Pearson Education.         [ Links ]

Mayer, R., & Bromage, B. (1980). Different recall protocols for technical texts due to advance organizers. Journal of Educational Psychology, 72(2), 209-225.         [ Links ]

McKeachie, W., Pintrich, P., Lin, Y.-G., & Smith, D. (1987). McKeachie, W., Pintrich, P., Lin, Y.-G., & Smith, D. (1987). Teaching and Learning in the College Classroom. A Review of the Research Literature. Michigan: National Center for Research to Improve Postsecondary Teaching and Learning, NCRIPTAL.         [ Links ]

Nair, C., Patil, A., & Mertova, P. (2009). Re-Engineering Graduate Skills - A Case Study. European Journal of Engineering Education, 34(2), 131-139.         [ Links ]

Paz-Penagos, H. (2011). ¿Cómo desarrollar la metacognición en la educación superior mediante la resolución de problemas? Ingeniería e Investigación, 31 (1), 213-223.         [ Links ]

Peña-Reyes, J. (2011). Major challenges in engineering and its role in society. Ingeniería e Investigación, 31 (1SUP), 100-111. Retrieved from http://www.revistas.unal.edu.co/index.php/ingeinv/article/view/27931.         [ Links ]

Pneena, S., & Romanowski, C. (2001). A Message from Recent Engineering Graduates in the Workplace: Results of a Survey on Technical Communication Skills. Journal of Engineering Education, 90(4), 685-693.         [ Links ]

Ponce, H., Mayer, R., & Mario, L. (2013). A computer-based spatial learning strategy approach that improves reading comprehension and writing. Educational Technology Research & Development, 61 (5), 819-840.         [ Links ]

Ramírez-Echeverry, J. J., & Olarte, F. A. (2013). Experiencia en la Evaluación de Competencias de Estudiantes de Primer año de Ingeniería Electrónica por medio de Rúbricas. Revista Educación en Ingeniería, 8(16), 12-24.         [ Links ]

Ramírez-Echeverry, J., Olarte, F., & Garcia-Carrillo, Á. (2014). Estrategias de Aprendizaje usadas por Estudiantes de Ingenieria Eléctrica e Ingenieria Electróonica de Primer Semestre. Revista de Educación en Ingeniería, 9(18), 216-227.         [ Links ]

Solis, R., & Abad, C. (2004). Case study in Mexico: the civil engineering students give opinion about their weaknesses at the time they leave the university. Ingeniería e Investigación, 24(2), 27-34.         [ Links ]

Taylor, B., & Beach, R. (1984). The effects of text structure instruction on middle-grade students' comprehension and production of expository text. Reading Research Quarterly, 19(2), 134-146.         [ Links ]

Van Epps, A., Fosmire, M., Wertz, R., & Purzer, S. (2013). Measuring student's ability to find and use high quality information: Developing standardized assessments. IEEE International Professional Communication 2013 Conferencee (IPCC) (pp. 1-5). Vancouver, BC: IEEE.         [ Links ]

Wertz, R., Purzer, Fosmire, M., & Cardella, M. (2013). Assessing Information Literacy Skills Demonstrated in an Engineering Design Task. Journal of Engineering Education, 102(4), 577-602.         [ Links ]

Wheeler, E., & McDonald, R. (2000). Writing in Engineering Courses. Journal of Engineering Education, 89(4), 481-486.         [ Links ]

Winsor, D. (1990). Engineering Writing/Writing Engineering. College Composition and Communication, 41 (1), 58-70.         [ Links ]

Yang, Y. (2015). Transforming and constructing academic knowledge through online peer feedback in summary writing. Computer Assisted Language Learning(ahead-of-print), 1-20.         [ Links ]

Zhu, W. (2004). Faculty views on the importance of writing, the nature of academic writing, and teaching and responding to writing in the disciplines. Journal of Second Language Writing, 13(1), 29-48.         [ Links ]

Appendix 1