This article aims at identifying the kind of American and British literature tests that can be designed to allow students who enter a bilingual education program at a private university in Colombia to have their previous knowledge in these two subjects accredited through a proficiency test. Students' needs, opinions, beliefs, existing commercial tests, the University's (specifically, the one where the study was conducted) literature programs, several anthologies, and competences required in the education program were all taken into consideration. Then the tests were developed, piloted, and validated with a focus group composed of ten students of said program. The results indicated that students require previous knowledge, literary competence, and command of the English language because those are determining factors in successfully passing the tests.

Key words: Previous knowledge, proficiency tests, American and British literature, competences

Introduction

The interest in this particular topic arose from our experience as English teachers and some of the Education students who also possessed a background of previous knowledge in both American and British literature. Since they could not get their previous knowledge accredited, it was proposed that the University let us design the two proficiency tests to be used as a tool to accredit previous knowledge in those subjects.

The researchers' goals were to open doors in the Faculty of Education of the University that would allow students' previous knowledge to be taken into account and allow future generations to be able to sit exams on the subjects which they considered themselves proficient in. Accreditation saves students valuable time and, therefore, enables them to study new subject areas, thus the students of the Bilingual Education Program at the University will optimize their time and their knowledge by getting the credits in these subjects accepted. When adults are studying and working full time, besides being parents and having other family obligations– as is the case of most of the students at this university– time is precious.

Our concern about this topic has evolved, and in order to take appropriate action, it was decided we should investigate this subject in depth. After having studied the problem, the following research question was posed: How can the University's bilingual education students' previous knowledge, the existing literature proficiency tests, and the University's current American and British literature programs be taken into consideration in designing proficiency tests for these two subjects? Hence, we formulated the following specific questions to find a solution to said problem: What previous knowledge of American and British literature do students bring to the university? What type of tests can be designed to evaluate students' previous competence in American and British literature? How can the criteria of the topics be tested and developed?

Folk tales travel wherever people travel.
(Anaya, 2002, p. 10)

After establishing the legal foundations to accredit the previous knowledge through the proficiency tests, we focused on defining the four important constructs of our research: proficiency tests, literature, prior knowledge, and competence.

Proficiency Tests

According to Hughes (1989), "proficiency tests are designed to measure people's ability in a subject regardless of any training they may have had in that subject. The content of a proficiency test, therefore, is not based on the content or objectives of any courses which people taking the test may have followed" (p. 9). If this is so, then theoretically anybody who has ability to perform in a specific subject is entitled to take a test to prove their abilities. In this case, knowledge and previous knowledge are not taken into account. "Rather, proficiency is based on a specification of what candidates have to be able to do in order to be considered proficient" (Hughes, 1989, p. 9). In Webster's Third New International Dictionary (1986), the definition of proficient is the following: "to be well advanced in an art, occupation, skill or a branch of knowledge: unusually efficient".

One of the proficiency tests for literature recog-nized worldwide is the Graduate Record Exa-mination (GRE) subject test (literature in English), which is not for undergraduate students, of course. Even though it is a useful tool for comparative purposes, the researchers decided that the Subject Area Test (SAT) literature tests recommended by College Level Examination Program (CLEP) are more adequate to the level and needs of the participants and the University.

Regarding the abovementioned data, Hilliard (1999) asserts that "All standardized tests have their place, even the weak ones. Without them we would have nothing —no way to hold people accountable for teaching or learning" (p. 38). In connection with this, our goal was to design the proficiency tests in order to account for students' competences in literature through the mentioned tests. Regarding tests, Hilliard (1999, p. 63) asks: "Then why use standardized tests at all?" Schools and districts use them because they provide data and results that are essential for improvement. For these reasons, we took the theory and some models as test references.

Student performance data are essential elements in a healthy and successful improvement effort (Glickman, 1993; Fullan, 1991). These authors state that standardized tests, despite some shortcomings, provide numerical and intelligible data as to how well a learner is performing or improving, and vital information about patterns of strengths and weaknesses among students.

Literature

To define this construct (a subject that focuses on special topics because it is content-based), we can start with Long (2005), who says that literature is the written record of man's best thoughts and feelings, and English literature is the part of that record which belongs to the English people. In its broadest sense, literature includes all writing, but as we commonly define the term, it excludes works which aim at instruction and includes only the works which aim to give pleasure, and which are artistic in that they reflect nature or human life in a way to arouse our sense of beauty (Long, 2005).

According to the Encyclopedia Britannica (1994, p. 398), literature is a body of written works. The name is usually applied to those imaginative works of poetry and prose distinguished by the intentions of their authors and the excellence of their execution. Literature may be classified into a variety of categories, including language, national origin, historical period, genre, and subject matter. When we study literature we deal chiefly with the great, enduring books, which may have been written in an older or a recent time, but which have in them timeless magic. In order to be considered proficient in literature, a person must understand each form of writing and be able to identify each text in its corresponding genre.

As already said, this investigation intends to determine the role prior knowledge plays in determining what type of test will be designed in order for students to be acknowledged proficient in American and British literature.

The researchers agree with him, and think that this is especially applicable when referring to literature. Besides, González (2000) states that there are special tasks focused on activating previous knowledge. In connection with this concept, Hitotuzi (2005) mentions that "Students are not a tabula rasa on which a supercilious teacher can record their knowledge; instead, they have knowledge and experiences of life and language (as well as other subjects, contents, and topics) which can contribute greatly to the learning process" (p. 98). In our case, some students of the Bilingual Education Program had some previous knowledge and experience in literature and thus wanted this knowledge to be accredited through a proficiency test.

Hughes (1989) states that prior knowledge and content seem to have a certain degree of influence regarding proficiency; thus, he supports the idea that if students have previous knowledge of a topic –in our case, literature– it will affect their test results. Similarly and following Roschelle (2004), it can be stated that it would be very difficult to learn without prior knowledge because students construct new concepts from prior knowledge. Since prior knowledge is based on experiences, it can be said that new knowledge is also defined by the new experiences in life.

Rico (2005) adds that "The level of proficiency is seen according to the students' capacity to recognize and master the elements of the system" (p. 99). Furthermore, he links several important concepts related with test design: proficiency, test design, competence, and previous knowledge. These elements supported our main goals in designing the proficiency tests in which, first, we found out what previous knowledge of American and British literature the participants had. In doing so, we followed Hughes (1989), who emphatically points out that the first step in test designing or construction is the statement of the problem. The test must be "perfectly clear about what it is one wants to know and for what purpose" (p. 48).

Competences

According to the Colombian National Ministry of Education, competences are a set of knowledge, attitudes, disposition and skills (cognitive, socialaffective and communicative), all interrelated to help students in the learning process, including the development of a new sense of activity in new and challenging contexts (MEN, 2004). Therefore, competence implies knowing, being, and knowing how to do. From the above, competences can be seen as the battery of knowledge, skills, and attitudes needed to carry out a determined activity and/or to solve problems in an autonomous and creative manner.

According to Smith & Wilson (1990), "Within Chomskyan theory, there are two levels of assessments: the first is called the level of competence, the speaker's knowledge of language which is purely linguistic, and the second is called the level of performance, the speaker's use of language" (p. 20).

Competences play an important role in the program under study. When a student wants to accredit previous knowledge in literature, s/he should not only demonstrate competence in the English language, but in the literature subject. He needs to perform and accomplish tasks in regards to aspects such as the ones mentioned before: national origin, historical period, genres, and outstanding works, among other aspects of the subject matter.

The study was a descriptive qualitative research because "the researchers do not set out to test hypotheses, but rather to observe what is present with their focus, and consequently the data, free to vary during the course of the observation" (Larsen-Freeman & Long, 1993, p. 10). According to these authors, this study follows a cross-sectional approach where the performance of a large group of subjects is studied, and the data are usually collected in one session. Hence, we gathered the data through personal answers of the interviewees by means of questionnaires and oral interviews.

According to the above-mentioned authors, "one of the characteristics of the qualitative paradigm is that it is not generalizable; it is process-oriented, subjective, grounded, discoveryoriented, exploratory, descriptive, and inductive. It is also holistic and assumes a dynamic reality" (Larsen-Freeman & Long, 1993, p. 11). First of all, we wanted to find out if the participants had previous knowledge, and if so, if they were willing to take the tests to accredit their previous knowledge, to then be able to design the literature tests.

The Participants and Setting

Out of sixty four students of the Faculty of Education, ten students were selected as a focus population to answer a prior information questionnaire (See Appendix 1). They replied that they did have previous knowledge in American and/or British literature, and also answered affirmatively to the question of wanting to present a proficiency test to have this knowledge accredited. Their ages ranged from sixteen to fifty five. Some of them studied at bilingual schools in the city; others either studied or lived for a time (varying with each individual) in English speaking countries. Other students gave account of their previous knowledge in these areas due to personal interest and autonomous study.

Instruments and Procedure

As already said, at first we developed a Prior Information Questionnaire (See Appendix 1). We selected a questionnaire to collect the first information because it let us gather data in one or two sections. As Burns asserts, questionnaires are "written sets of questions used to gain responses in non face-to-face situations; questions are usually focused on specific issues and may invite either factual or attitudinal responses" (1995, p. 117).

Those two instruments were chosen after reflecting on the type of question to be answered and the type of research that was being carried out. The information was typed into the computer, then organized; lastly the researchers carried out the analysis by closely following the steps that dealt with levels of analysis.

In the Literature Review section it is stated that one of the main limitations of proficiency exams or standardized tests is the fact that they are designed to be taken by large groups of people at one time and, therefore, must necessarily be machine scored (Barr, Craig, Fisette & Syverson, 1999; Hilliard, 1999; Covey, 1989; ETS, 2002). Thus, commercial standardized tests were not a good option for our purpose. To avoid this limitation, the researchers took it upon themselves to design two "tailor-made" literature proficiency tests. In this task we considered the students' needs, opinions, and beliefs, as well as the Faculty's requirements for the above-mentioned subjects. Then the tests were developed and piloted with the previously chosen candidates.

In the designing of the tests we took into consideration the data provided by the participants through instruments described above, as well as information from several literature textbooks, anthologies, the GRE, the CLEP, and the SAT. All these documents are related with literature subject tests and were used as references to design the two literature tests.

The participants took the tests. Then, both tests were assessed (graded) by the researchers and a final questionnaire was presented to the participants in order to learn their opinions about the literature tests (See Appendix 3).

The category testing time included two subcategories, and was aimed at finding out students' preferences as to when, during their studies, they would want to take the proficiency tests. Of the total of answers given, the majority indicated that they would take the test any semester; and the others indicated that they would like the opportunity of taking it during the first semesters (1- 3) because what they had learned at school was still fresh in their minds. This question was solely for informational purposes because it had no usability. As noted by Brown, it "has to do with the degree to which a procedure is practical to use". However, the university students' handbook states that proficiency examinations may be taken at any time during the undergraduate studies (1995, p. 52).

Taking into account that the objective of the study was to find out what the role of the previous knowledge was for designing a literature proficiency test, this was one of the most important categories. Questions one, two, and eight from the interview provided the greatest amount of information regarding this category (See Appendix 2). Table 2 illustrates this, with answers given by three participants, about previous knowledge.

The students' comments in Table 2 supported the quote from the CLEP (2007) in which it states that only if they had this "broad knowledge of literature" and "a familiarity with the basic literature terminology", the participants were able to obtain good results as was observed when the tests were applied.

The Designing of the Proficiency Test and Its Outcomes

Having analyzed all of the above information from our three sources, students' interviews, theory (University's current programs, standardized tests, and anthologies), and the researchers' points of view, we decided that the tests in American and British Literature should have 70 questions each and one essay. These questions were thirty multiple-choice (30 points), a time line (10), reading comprehension (20), ten open-ended questions (10), and one essay (30). The numbers in parentheses are equivalent to the number of points each type of question would count for a total of one hundred points (100). The minimum passing grade was 70/100.

After having corrected the literature proficiency tests presented by ten students at the University, the general results were as follows: half of the total number of students who took the literature proficiency tests passed them- a majority passing British Literature and a minority passing American Literature.

In British Literature a great number passed the time line, multiple-choice, and reading hension; less than half passed short opened-questions; and everybody did well on the essay. Finally, all of the students achieved a passing grade on the test, (60/100), but since the minimum grade for obtaining the proficiency credits was placed at 70%, most of the participants achieved it, but one person did not. The time allotted was adequate because all of the students used fewer than the three hours given, and only one person took two hours and fifty minutes.

In American Literature nobody passed the time line; one person passed the multiple choice section; one person passed the short open-ended question; and several passed the essay. One person did not complete it, and one person did not do it because she felt unable to do it. One person obtained a perfect score (30/30) on the essay.

The time allotted was enough because all of the people who took the test needed less time to complete it. After the students sat the literature proficiency tests, they were asked to answer a final questionnaire (See Appendix 3) in order for the researchers to obtain feedback on the tests. Some aspects worth mentioning about the feedback results are the following: Six students did not study at all for the test. However, five of them passed the test, which supports the idea that previous knowledge is important when taking a proficiency test. Participant 5 said: "No studying; I just relied on previous knowledge".

Before the students knew their results, more than half thought they had done acceptably and the rest thought they had done poorly. The main reasons mentioned were lack of study and preparation. Nevertheless, the tests results obtained were usually better than what the participants thought. For example, participant No. 2, who answered acceptably, did very well (93/100); likewise, participant No. 9, who answered poorly, obtained the passing grade of 70/100.

Regarding the allotted time, three participants expressed that it was sufficient; four said it was adequate, and two thought it was excessive. In general, the students' acceptance of the test was good even though they suggested some modifications. This can also be observed in the explanations given to question No. 7. Some of these explanations contradict each other. As none of the reasons given was supported by a considerable majority, the researchers concluded that the tests should not be modified.

As researchers, we thought that the participants were going to do very well on the reading comprehension part, as the answers can all be found in the text given if the participant has good reading skills, and because in the interview eight out of ten wanted it to be included in the test. However, this section did not produce the expected results. This may be due to the fact that many students read too fast, and did not pay sufficient attention to details during testing.

The above has a definite relationship with what Glickman (1993) expresses in that proficiency tests are useful in determining patterns of students' strengths and weaknesses in subject areas.

With respect to the short open-ended questions, the results obtained by many of the participants were poor. This was unexpected because the questions included in the tests were designed around very well-known topics. It was also strange because in the interview students answered (see Appendix 2) about the type of questions to be included in the test, three of the participants specifically asked to include this type of questions.

It is important to acknowledge the fact that the profile needed for presenting these tests was that of having prior knowledge in either American or British literature, or in both. Seven of the participants met this requirement either because they studied at bilingual schools or because they were teaching literature. This issue brings out the fact that there were three students who presented the proficiency tests without having the required profile.

Taking into account the students' feedback on the tests, the following aspects should be noted: "It should be based on the University program" (Participant 1); "Some authors are not in the program" (Participant 10). In connection to this, it should be kept in mind that the focus of a proficiency test is to validate the student's previous knowledge in a given subject in order to accredit it; therefore, it must include a general overview of the subject area.

A proficiency test is not intended to be limited to any course, curriculum, or single skill in the language. The following testimonies evidence some participants' positive views about the tests: "It's OK, better than the program" (Participant 2); "It was good, adequate to the topic" (Participant 4).

Although the test was the result of analyzing carefully the number of questions, the time allotted and the topics of several literature commercial tests, other general comments were that it was "too long" (Participants 7, 8, 9, and 10).

All the participants who passed with 70% or more had previous knowledge in literature from other studies either at high school, university or both, as was confirmed by the information obtained from the initial interviews. This finding was in addition to having taken the British literature course at the University in the semester just prior to taking the test. Another reason that influenced the above-mentioned difference was that all the students who took this exam and passed it had a high level of English and could be considered bilingual since they were taking or had already passed English VI at the University. This was a very important factor because the test requires a full command of the English language.

On the other hand, none of the participants had taken an American literature course at the University. Additionally, the three participants with the lowest scores on the test were still doing the first levels of English at the University at the time they took the test. It was inferred, therefore, that they are not totally proficient in the English language which, as stated above, is a determining factor in the final result. This is illustrated with this excerpt: "I did not even try to do the essay because I did not understand what it was asking" (Participant 4).

There is a definite relation between students' profile (background) and previous knowledge in order to obtain the accreditation of the latter, which is important to have in mind when the students ask to have their previous knowledge validated by presenting proficiency tests. If they do not have a good mastery of English, they will probably not be able to answer the essay.

As the majority of the students who took British Literature passed the test, the researchers think that the test may be a useful tool for the University in the future. Although the results were different for the America Literature test, it can be explained through the observations stated above that the reasons for this could be attributed to the students' lack of previous knowledge, and their low command of English, and not to failures in the test design itself.

This research demonstrated that the three aspects mentioned in the initial research question, students' previous knowledge, the existing literature proficiency tests, and the University's current American and British literature programs, were carefully studied and included in the design of the two proficiency tests. Our general objective was also achieved despite what Cohen (1994) says about standardized tests. This author believes there is a difference between proficiency tests, which are used mainly for academic purposes because they focus on theory, and achievement tests, used mainly for administrative purposes because they focus on abilities or competences.

The main conclusion after having carried out this research project was that both previous knowledge and competences are determining factors in the test results. The difference in the test results among the participants who had previous knowledge both in literature as well as mastery of the second language supports what the theories and studies mentioned before state, regarding testing.

Another important finding was that competences, evaluated through reading comprehension and essay writing, can also be taken into consideration when assessing both previous knowledge and competence in literature.

Two literature proficiency tests were designed and validated although, according to TESOL literature, teachers do little collaborative research and most of the literature regarding proficiency tests is limited to language proficiency testing. The tests designed in this study achieved a synthesis of assessment methods, as well as a more communicative approach to literature testing. As proposed by Hilliard (1999), they evaluate previous knowledge (theoretical subject content/literature) in the timeline, multiple choice section, and short open ended questions, as well as competences considered in the reading comprehension and essay writing.

Some limitations we had to deal with in this research project were that ten people comprised a small population compared to the total number of bilingual education students (111 at the time this research was initiated). Nonetheless, this focus group was composed of the participants who met the required profile out of 64 participants who answered the prior information questionnaire.

Due to the lack of model literature tests that covered most of the aspects students wanted to be evaluated, we had to spend more time in designing them. Due to the nature of the tests developed for this project, they had to be teacher corrected and could not be machine scored.

After concluding the research we have the following questions for further study: How can the information obtained from the participants' interviews be used to modify or improve the University's current literature programs? How can the two proficiency tests resulting from this research be actually used by the University? Should they be revised or modified; if so, by whom? Should a question bank be developed in order not to repeat the exact test every semester? How can students' abilities and competences be taken into consideration to improve the University's literature programs? Can there be another way of accrediting previous knowledge?

The American and British literature tests were designed based on the students' needs, the University requirements and program, theory about literature and testing, and commercial tests to accredit previous knowledge. Competences and good command in English were essential factors in passing the tests. The tests were not published in this research to enhance its validity as they are going to be used by the University to accredit students' previous knowledge.

Accredit. (1986). In Webster's Third New International Dictionary, Inc., 1, 13. Proficient, 2, 1811. Chicago: Merriam- Webster.        [ Links ]

Anaya, R. (2002). Folk literature. Timeless Voices, Timeless Themes, Copper Level. New Jersey: Pearson Education, Inc.        [ Links ]

Atlasti- The knowledge Workbench: ATLAS.ti 5.2 (software); www.atlasti.com-sp4.html        [ Links ]

Barr, M. A., Craig, D. A., Fisette, D., & Syverson, M. A., (1999). Assessing literacy with the learning record. Portsmouth, NH: Heinemann.        [ Links ]

Brown, D. H. (1995). The elements of language curriculum. Boston: Heinle & Heinle.        [ Links ]

Burns, A. (1995). Qualitative research studies. Cambridge: Cambridge University Press.        [ Links ]

Carroll, J. A., Feldman, K., Kinsella, K., Stump, C. S., & Wilson, E. (2002). Learn about literature. New Jersey: Pearson Education, Inc.        [ Links ]

Cohen, A. D. (1994). Assessing language ability in the classroom. (2nd ed.). Boston: Newbury House.        [ Links ]

Colombia. Ministerio de Educación Nacional [MEN]. (2002). Decree 0808 April 25, 2002 Articles 3, 4, and 8: Por el cual se establece el credito académico como mecanismo de evaluación de calidad de transferencia Art. 3, 4, 8. Bogotá.        [ Links ]

Colombia. Ministerio de Educación Nacional [MEN]. (2004). ¿Qué son las competencias? Retrieved August 12, 07 from http://www.inesup.com/descargas/conceptodecompetencia.ppt        [ Links ]

Colombian Congress. (1992). Ley 30 de 1992 art. 20, 28, 109: Fundamentos de la Educación Superior. Bogotá: Autor.        [ Links ]

College-Level Examination Program [CLEP] (2007). Retrieved August 7, 2007 from http://www.collegeboard.com/ student/testing/clep/ex_el.html        [ Links ]

Covey, S. (1989). The seven habits of highly effective people. New York: Simon & Schuster in Educational Leadership.        [ Links ]

Esposito, J. J. (1994). Encyclopedia Britannica. Chicago: Encyclopedia Britannica, Inc. 7, 398.        [ Links ]

Educational Testing Service [ETS]. (2002). Literature in English test practice book. Retrieved August 7, 2007 from http:/www.gre.com        [ Links ]

Fullan, M. G. (1991). The new meaning of educational change. New York: Teachers College Press.        [ Links ]

Glickman, C. D. (1993). Renewing America's school: A guide for school base actions. San Francisco: Jossey-Bass.        [ Links ]

González, A. (2000). Lo que no sabemos nos impide aprender. Conocimiento previo y aprendizaje de textos [electronic version]. Universidad de La Rioja, 2001- 2007. Retrieved August 7, 2007 from http://dialnet.unirioja.es/servlet/articulo?codigo=1032524        [ Links ]

Hilliard, A. (1999). Toward a better teaching education: A conversation with Asa Hilliard. New York: Teachers Collage Press.        [ Links ]

Hitotuzi, N. (2005). Teacher talking time in the EFL classroom. PROFILE, Issues in Teachers' Professional Development, 6, 97-106.        [ Links ]

Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.        [ Links ]

Larsen-Freeman, D., & Long, M. (1993). An introduction to second language acquisition research. New York: Longman.        [ Links ]

Long, W. J. (2005). Outlines of English and American literature. An introduction to the chief writers of England and America, to the books they wrote, and to the times in which they lived. Retrieved August 23, 07 from http://www.fullbooks.com/Outlines-of-Englishand-American-Literature1.html        [ Links ]

Popham, W. J. (1999). Why standardized tests don't measure educational quality. New York: Teachers Collage Press.        [ Links ]

Rico, C. (2005). Searching for coherence in language teaching; the issue of teaching competencies. Colombian Applied Linguistics Journal, 7, 95-107.        [ Links ]

Roschelle, J. (2004). Learning in interactive environments: Prior knowledge and new experience. Dartmouth: University of Massachusetts.        [ Links ]

Siedman, I. (1991). Interviewing as qualitative research: A guide for researchers in education and the social sciences. New York: Teachers College Press.        [ Links ]

Smith, N., & Wilson, D. (1990). Modern Linguistics. England: Penguin books.        [ Links ]

About the Authors

The objective of this questionnaire is to establish which students, if any, of those studying Bilingual Education at the University, have any prior knowledge in the subjects of American and British Literature. If they do have this prior knowledge, we want to find out where and when they obtained it; and to ask if they want this knowledge to be accredited through a proficiency test.

]]> If you are interested, it would be very valuable for our research project, if you would answer the following questions:

Personal Data:

Name:______________________________________________________________________

Date:_______________________________________ Semester:________________________

1. Do you have any previous knowledge on American Literature? Yes___ No ___

2. How would you consider your level in this area?

High = able to identify authors, writings, literary periods, and historical context and also to do reading comprehension activities about them. High? ____

Medium = able to identify most authors, writings, literary periods and do some reading comprehension activities about them. Medium? _____

]]> Low = only identify a few authors and their work, but are not sure about literary periods or historical context. Not totally sure about doing reading comprehension activities. Low? _____

3. Where did you obtain this knowledge? School? ___ University? ___ Other? ___

Which? _____________________________________________________________________

4. When did you obtain this knowledge? Approximate Year? ______

5. Do you have any previous knowledge on British Literature? Yes___ No___

6. How would you consider your level in this area? *Use the same criteria as in question 2
High? _____
Medium? _____
Low? _____

7. Where did you obtain this knowledge? School? School? ___ University? ___ Other? ____

]]> Which? _____________________________________________________________________

8. When did you obtain this knowledge? Approximate Year? _____

9. Would you consider taking a proficiency exam to validate this knowledge? Yes___ No ___