RESEARCH ARTICLES

http://dx.doi.org/10.14483/udistrital.jour.calj.2014.1.a01

Assessing trainees' oral performance in a Chilean teacher trainingprogram: A corpus-based study¹

Evaluación del desempeño oral de aprendices en un programachileno de capacitación de maestros: un estudio basado en corpus

Maritza Fernanda Ortega Pérez, M.A.
Universidad de Chile
Santiago de Chile, Chile
maritza.ortega.perez@gmail.com

Received: 16-Oct-2013/ Accepted: 27-Feb-2014

To cite this article
Ortega, M. F. (2014). Assessing trainees' oral performance in a Chilean teacher training program: A corpus-based study. Colombian Applied Linguistics Journal, 16(1). 10-16.

Abstract

The present paper reports the implementation of syllabus innovations in EFL teacher education in Chile after diagnosing a lack of language achievement standards common to all EFL teacher training programs offered in public and private universities alike. The aim of this study is to collect linguistic data in natural and artificial social contexts - EFL trainees' intermediate status between their native language (Spanish) and the target language (English) - in order to create the first Chilean corpus of spoken English as a foreign language, in the interest of analyzing the errors that are most likely to be made and fossilized by native speakers of Chilean Spanish. Once the results of this exercise are available, EFL trainers, professors, and SLA researchers will be able to design a newly sequenced syllabus based on the Content-based Approach and tailored to students' needs so as to enhance oral performance in L2 English.

Key Words: communicative performance, content-based approach, corpora, interlanguage, second language pedagogy.

Resumen

El siguiente proyecto de investigación reporta la implementación de innovaciones curriculares en la formación docente de profesores de inglés en Chile, después de diagnosticar una falta de logros lingüísticos comunes a todos los programas de pedagogía en inglés que ofrecen las universidades públicas y privadas en la actualidad. El objetivo principal de este estudio es recopilar datos lingüísticos de los estudiantes de pedagogía en inglés en contextos sociales naturales y artificiales, con el fin de crear el primer corpus chileno de inglés hablado como idioma extranjero. De este modo, es posible analizar los errores más comunes con tendencia a ser fosilizados por hablantes nativos de español chileno. Una vez que los resultados de este ejercicio se encuentren disponibles, profesores de lengua inglesa e investigadores interesados en el área de la lingüística aplicada serán capaces de diseñar un nuevo plan de estudios secuencial basado en el enfoque de intrucción de contenidos, y al mismo tiempo, adaptado a las necesidades de los estudiantes, para mejorar su desempeño oral en la lengua meta.

Palabras clave: desempeño comunicativo, enfoque basado en contenidos, interlengua, lingüística de corpus, pedagogía en inglés.

Introduction

It is widely known that one of the most studied areas of second language pedagogy involves curriculum design based on several approaches to the teaching of English as a foreign language. The grammar-translation method, the audio-lingual approach, the natural approach, and communicative language teaching, among others, were all created to facilitate the learning and/or acquisition of foreign languages in contexts in which the target language is not spoken by the majority of the population. However, after years of demonstration of these methods, in unrealistic contexts with ideal educational conditions, it is still possible to find students struggling to achieve the goal of improving their second language performance, and to find teachers exerting themselves to provide their students with the most effective techniques to achieve advanced levels of the target language according to international standards.

Chile, being a monolingual country, is no exception to this scenario. International language exams (TOEIC Bridge) taken by high-school students from all schools throughout the country show that only 11% of students obtain a certification in elementary English. Thus, the language teaching methods mentioned above, which are broadly used in the Chilean syllabus for foreign language instruction, are not fulfilling the outcomes expected from language learners after eight years of formal learning. Other claims related to this issue point to the poor L2 language levels achieved by teachers of English as a foreign language by the end of their teacher training programs in Chilean universities. Statistics suggest that only 28% of these teachers can speak English at an advanced level of proficiency, and that only 8% use the target language properly while teaching (Simce inglés 2010, Gobierno de Chile).

Unfortunately, this situation has triggered an almost unbreakable vicious circle. On the one hand, teachers are prevented from growing professionally due to their limited linguistic and communicative abilities in the target language, and on the other, the students' opportunities to learn English are prejudiced due to their teachers' inexperience. Nevertheless, it has been proven that this cycle can be broken by designing a special syllabus that addresses the specific needs of the learners in a given context (Chan, 2008). It is for this reason that this study will aim to improve students' language achievements through the creation of an EFL corpus of Chilean students from the English teacher training program that will be used as an instrument to assist the error analysis process of the learners' interlanguage. Subsequent to the corpus will be the design of a sequenced syllabus tailored to student teachers' needs through the implementation of lessons based on the Content-based method.

Theoretical Framework

In Latin America, educational policies are heavily focused on expanding the limited body of subject teachers, with the central aim being that all beneficiaries of the public system have free access to a minimum set of contents throughout elementary and secondary education. These 'expansion and access' policies, however, ignore completely one of the most important premises of public education: quality and efficiency (Abrahams & Farias, 2010). As in the rest of Latin America, neoliberal policies in Chile have produced a proliferation of English teaching programs at private universities that lack any tradition in research or formation of quality English language teacher experts. Unfortunately, this has resulted in a wealth of universities engaging students in EFL teaching programs in harmful dynamics that tend to result in substandard graduates who are largely unacquainted with the foundations or contents constituting the national syllabi. Although the Chilean government has invested highly in English language teaching since the formation of various free-trade agreements - in-service training, graduate studies abroad, semester abroad program, etc - the poor language achievement standards continue to be a problem amongst EFL teachers. In 2004, 11,000 students took an ESOL exam to assess the levels of English proficiency of Chilean students following four to eight years of formal language instruction at school. The results were shocking. The vast majority of students with eight years of English instruction achieved 'lower breakthrough' and 'breakthrough' levels, with 'lower breakthrough' being a new level created following the discovery that the target language level of the students was below international standards. This diagnosis also showed that only 5% of pupils left school with a level of English sufficient to face academic and professional life, and that they were more likely to understand the target language when they read rather than listened to it (Diagnóstico de inglés 2004, Gobierno de Chile). It is at this point that the spotlight fell upon EFL teachers who attracted criticism for the apparent lack of exposure students receive to the target language, as well as the use of old-fashioned language teaching approaches, such as the grammar-translation method, which are proven to fail if not combined with other approaches related to communicative performance. Since no research has been done in this area within the Chilean context, it would be interesting to see the outcomes of the present project and its implications in the field of second language pedagogy within our country.

One of the areas to be examined in this project is based upon the theory of interlanguage. As Mystkowska-Wiertelak and Pawlak (2012, p.32) suggest, interlanguage theory emerges as a new technique to identify the errors that learners make throughout their foreign/second language acquisition process before they achieve the complexity of the language system used by native speakers of the target language. Brown (2000), Corder (1971), Nemser (1971), and Selinker (1972) named this phenomenon in various ways - interlanguage, intermediate status between the native and target language, approximative system, and idiosyncratic dialect, respectively - nevertheless, they each agree that what language learners produce at this stage is different from their L1 and L2 due to its self-contained nature (Brown, 2000, p.215). A study carried out by Maniruzzaman (2008) at the university of Jahangirnagar examined the relevance of the use of interlanguage theory in the EFL classroom with Bengali speaking learners at the tertiary level. He concluded that interlanguage studies allow researchers to spot the students' weaknesses in the four skills and identify their true linguistic level in order to create a unique syllabus suitable for EFL learners depending on the areas in which they have more difficulty. In this way, Maniruzzaman demonstrated that EFL programs based on interlanguage studies are more effective than those based on assumptions, theories, experiences, and predictions. Nonetheless, it is important to note that his study did not necessarily focus on the four skills, but rather on the production of written language, making the aims of his proposed syllabus for EFL learners solely grammar-oriented. The present investigation will be based on interlanguage theory, but unlike Maniruzzaman's work, the oral communicative performance of students from the English teacher training program in Chile will be assessed.

In order to collect the spoken language needed to analyze the weaknesses of students, corpora will be used. A corpus is "a collection of texts or parts of texts upon which some general linguistic analysis can be conducted" (Meyer, 2002, p.5). There are two different types of corpora: pre-electronic corpora, which correspond to texts that were analyzed manually due to the nonexistence of computers, and electronic corpora, whose use began in the sixties with the creation of the first computerized corpus, the Brown Corpus (Kucera & Francis, 1967).

Corpora possess unique characteristics essential to empirical linguistic research, thus I have several reasons for selecting this linguistic method over others. First of all, the corpus is sampled to maximize the representation of the population, therefore, findings can be easily generalized to a larger population. In addition to this, it is not necessary to go through the process of data collection if the research can be carried out with existing corpora, which is, in most cases, easy to access. Similarly, certain projects require the collection of new data relevant to the research being conducted, thereby creating one's own corpus is a plausible mechanism, if a balanced representative sample is available to the researcher. Finally, the majority of the existing corpora contain part-of-speech annotation, grammatical parsing, and prosodic transcription of texts that provide naturalistic and unmonitored language that is product of real social contexts (McEnery & Wilson, 2001, pp.32-60). The studies on language pedagogy by means of corpora have been implemented largely through the use of existing data so as to critically assess language teaching materials available in the market. As Kennedy's (1987) research shows, the analysis of relevant constructions and frequencies of specific vocabulary items in the sample textbooks and also in standard corpora of English aims to find whether the production of language learning materials contain the most common choices of usage of the English language. However, there is a different modality of corpora that offers a detailed description of the distinctive linguistic features of language produced by learners of a foreign/second language, called learner corpora (Guilquin, Granger, & Paquot, 2007). In Guilquin et al's (2007) study on learner corpora and its contribution to EAP pedagogy, the authors point out that analyzing learner corpus data is an effective way of spotting writing difficulties in novice nonnative writers. They also highlight the advantages of using this type of corpus over native corpora in terms of quantity and quality of the data base itself. Learner corpus data is usually larger and so provides a wider empirical basis for researchers, whilst having the added strength that "it can be submitted to a wide range of automated tools which make it possible to quantify learner data with linguistic annotations" (Guilquin et al, 2007, p. 324). Regarding error analysis, this research posits that the learner corpus data must be analyzed with software packages that include text-handling tools to facilitate analysis. Unfortunately, there are not many learner corpusdriven studies on speaking skills currently available; nonetheless, Sylviane Grange's research group at Université Catholique de Louvain is working on an investigation of fluency profiles in English learner speech in comparison with native speech: Fluency and disfluency in spoken English (UCL-PhD Theses Under Preparation, 2014). It would be interesting to explore the process of creating the EFL corpus needed for this research project, from its design and organization, to its empirical investigation and final compilation.

Another area to be discussed, due to its importance in the development of syllabus design, is the Content-based approach. Unlike other approaches to language teaching, this approach makes use of a subject matter for language teaching purposes, i.e. contents from other disciplines are used in language courses in order to integrate the learning of language with the learning of other contents, usually academic subjects (Larsen-Freeman, 2000). Teachers using this approach must be aware of the importance of building on students' previous knowledge so that they may see the relevance of their language use and also to increase their levels of motivation towards the learning of the target language. As language is used in this context as a medium to convey information, students struggling to properly express themselves should always be assisted by their teachers in constructing what they want to say: examples, redundancy, contextual clues to infer meaning, etc. Additionally, this method offers the opportunity for students to be exposed to cognitively demanding language within the context of authentic material; therefore, the communicative competence that takes place as outcome not only involves speaking, but also discussing, debating, and writing about the content seen (Richards & Rodgers, 2001). The lessons normally taught under these premises are considered a form of language immersion as the academic subjects are learned by means of a foreign language and not the first. Studies on the impact of CBI on student gain in second language development report that students enrolled in CBI classes made a significant improvement on a variety of language tasks and scored higher than students enrolled in skill-based courses (Dupuy, 2000). Similarly, students indicated having enhanced their speaking, reading, and listening skills, besides increasing their self-confidence and motivation to continue studying the target language (Ready & Wesche, 1992).

To conclude, the benefits of the Content-based Approach are remarkably significant for EFL students, since through the model they are able to learn both specific content material and related language skills. As Wesche (1993) states, "'in content-based language teaching instruction, the claim in a sense is that students get "two for one" - both content knowledge and increased language proficiency" (p. 57-58). This approach has never been successfully executed in Chilean public sector EFL classrooms due to the Ministry of Education's concern over the levels of comprehension a student, lacking a solid background in English, can obtain with such an innovative technique.

Methodology and Analysis

This research project presents the following hypothesis: that syllabi based on error analysis of EFL corpora representative of the subjects' spoken interlanguange improves the communicative and linguistic competences of EFL trainees, when working on those errors found with lesson design based on the Content-based approach. Consequently, the research questions behind this proposed hypothesis are the following: What are the main lexical/phraseological errors that Chilean students make when producing the target language orally? Will the Contentbased method be adaptable to the Chilean context in EFL classrooms? Will the combination of corpora and CBI enhance the communicative and linguistic performance of EFL students within the teacher training program?

Due to the non-experimental nature of this research, two student groups made up of 50 second-year students from the EFL teacher training program will be compared. All of the participants speak Spanish as their first language and possess an intermediate level of the target language. Their ages fluctuate between 19 and 28 years old, and they all study at public universities. On the one hand, the control group will be taught their language courses by means of their usual syllabus based on the Communicative Approach. On the other hand, the experimental group will receive language instruction based on the Content-based Approach, which at the same time will be tailored to the language weaknesses found within the learner corpus collected for the purpose of this investigation. It is important to mention that both groups will be attending two-hour language lessons twice a week for a two-month period.

During the first stage of this study, selected subjects will be asked to attend an interview (IELTS speaking module) to assess their spoken level of the target language before carrying out the language lessons based on the Communicative Approach and Content-based Approach, respectively. The data recorded in this first interview will be part of the EFL corpus designed to conduct the error analysis threshold. After collecting the data taken from the spoken texts provided by the EFL trainees, the learner corpus will be organized, transcribed and annotated in order to identify the most common mistakes made by Chilean speakers of English from the EFL teacher training program. Since the aim of learner corpora is to analyze the interlanguage of non-native speakers of the target language, the present learner corpus will be error-tagged in order to visualize the errors in context, and to allow for sorting dependent upon the purpose of the research to be carried out. The error analysis methodology adopted for this study is based on the computer-aided error analysis system developed for English within the French Interlanguage Database project by Granger (1998). Therefore, the steps will be the following: the manual correction of L2 Chilean Spanish corpus, the elaboration of the error tagging system for L2 Chilean Spanish, the insertion of error tags and corrections in the text files, the retrieval of lists of specific error types and error statistics, and finally, concordance-based linguistic analysis of major error types. Accordingly, Contentbased lessons focusing on the findings of the error analysis performed with the use of corpora will be designed. Both experimental and control groups will have lessons based on the Content-based Approach, which will also be founded on the error analysis of the students' own oral performance in the target language. As mentioned above, control groups will continue to work with the standard syllabus from the EFL teacher training program, unlike students who belong to the experimental group. Once the trial period comes to an end, all subjects from both the control and experimental groups will be asked to attend a second interview (IELTS speaking module) in order to assess their level of English. In this way, their la-test results can be compared with their initial results and the outcomes of the experimental group, who will have had lessons tailored to their specific language weaknesses, and can be viewed against those of their peers in the control group. The instruments used for this matter are a standardized speaking test (IELTS speaking module), and the Corpus of Chilean speakers of English as a foreign language (Chilean EFL Corpus). To begin with, the speaking test selected as an instrument for this research corresponds to an adapted version of the IELTS speaking module. Hence, it will be a one-to-one interaction between the subjects and the researcher in which the three parts of the test - answering introductory questions related to oneself, speaking about a given topic, and discussing the aforementioned topic - give the subjects the opportunity to use a range of different speaking skills. The test will be recorded in order to evaluate students' performance. Subsequently, the Corpus of Chilean speakers of English from the EFL teacher training program (EFL Corpus) will aim to collect the selected subjects' interlanguage by means of a semi-structured interview, which will effectively be an adapted version of the first section of the IELTS speaking module. As a means of observing naturally occurring interaction within an atmosphere of social activity, the students' language classes will also be recorded, thus making it possible to compare their performance to that provided in the semi-structured interview.

Conclusion

The outcomes obtained through this study will provide EFL trainers, academics, professors, and researchers with key information on their trainees in respect to their linguistic and communicative competences in L2 oral performance. The knowledge which trainers will obtain of their students' specific drawbacks and weaknesses when going through the process of learning English will empower them to change their syllabus and design a new program tailored to their trainees' linguistic needs. Since this is the first corpus of Chilean EFL speakers from the teacher training program, the project itself seeks to remain as the most important database available for EFL teachers and researchers seeking to improve the quality of the national syllabus and the teaching of English as a foreign language in Chile. Although the Content-based Approach is currently used in Chilean international schools where students speak English fluently, the results from the present study could be expected to spread the use of this method in public education, so as to introduce partial immersion programs and have pupils exposed to the target language from their first year of elementary school onwards.

References

Abrahams, M.J., & Farías, M. (2010). Struggling for change in Chilean EFL teacher education. Colombian Applied Linguistics Journal, 12(2), 110-118.         [ Links ]

Brown, D.H. (2000). Principles of language learning and teaching. New York: Longman.         [ Links ]

Chan, V. (2008). Determining Students' Language Needs in a Tertiary Setting. English Teaching Forum, 39 (3), p.16.         [ Links ]

Corder, S.P. (1971). Idiosyncratic dialects and error analysis. IRAL, 9(2), 147-160.         [ Links ]

Dupuy, B.C. (2000). Content-based instruction: Can it help ease the transition from beginning to advanced foreign language classes? Foreign Language Annals, 33(2),205-222.         [ Links ]

Granger, S. (1998). Learner English on computer. London and New York: Addison Wesley Longman.         [ Links ]

Granger, S. (2003). Error-tagged learner corpora and CALL: A promising synergy. CALICO 20 (3), 465-480.         [ Links ]

Gobierno de Chile. (2004). Diagnóstico de inglés de octavo básico a cuarto medio. Santiago, Chile.         [ Links ]

Gobierno de Chile. (2010). Resultados de SIMCE inglés. Santiago, Chile.         [ Links ]

Guilquin, G., Granger, S., & Paquot, M. (2007). Learner corpora: The missing link in EAP pedagogy. Journal of English for Academic Purposes, 6(4), 319-335.         [ Links ]

Kucera, H., & Francis, W. N. (1967). Computational analysis of present-day American English. Providence: Brown University Press.         [ Links ]

Larsen-Freeman, D. (2000). Techniques and principles in language teaching. Oxford: Oxford University Press.         [ Links ]

Maniruzzaman, M. (2008). Relevance of the study of interlanguage to teaching EFL at the tertiary level: Interlanguage to EFL teaching. Germany: Books on Demand GmbH.         [ Links ]

McEnery, T., & Wilson, A. (2001). Corpus linguistics. Edinburgh: Edinburgh University Press.         [ Links ]

Meyer, C. F. (2002). English corpus linguistics: An introduction. Cambridge: Cambridge University Press.         [ Links ]

Mystkowska-Wiertelak, A., & Pawlak, M. (2012). Production-oriented and comprehension-based grammar teaching in the foreign language classroom. Berlin: Springer.         [ Links ]

Newser, W. (1971). Approximative systems of foreign language learners. IRAL, 9(2), 115-123.         [ Links ]

Ready, D., & Wesche, M. (1992). An evaluation of the University of Ottawa's Sheltered Program: Language teaching strategies that work. In R. Courchene, J. Glidden, J. St. John, & C. Thérien (Eds.). Comprehension-based second language teaching / L'enseignement des langues asxé sur la compréhension. Ottawa: University of Ottawa Press.         [ Links ]

Richards, J. C., & Rodgers, T. (2001). Approaches and methods in language teaching. Cambridge: Cambridge University Press.         [ Links ]

Selinker, L. (1972). Interlanguage. International Review of Applied Linguistics, 10, 209-241.         [ Links ]

University Catholique de Louvain Site Officiel. (2014). PhD Theses. In Centre for English Corpus Linguistics. Retrieved from www.uclovain.be/en-279056.html.         [ Links ]

Wesche, M. B. (1993). Approaches to language study: Research issues and outcomes. In Krueger, M., & F. Ryan (Eds.) Language content: Discipline-content approaches to language study. Lexington, MA: D.C. Heath and Company.         [ Links ]

THE AUTHOR

MARITZA FERNANDA ORTEGA PÉREZ, holds an MA in English Linguistics from University College London, a BA in English Language Teaching from Universidad de Santiago de Chile, and is an Applied Grammar Lecturer at Universidad de Chile.