SciELO - Scientific Electronic Library Online

 
vol.20 issue1Teacher to Researcher: Reflections on a New Action Research Program for University EFL Teachers author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

  • On index processCited by Google
  • Have no similar articlesSimilars in SciELO
  • On index processSimilars in Google

Share


Profile Issues in Teachers` Professional Development

Print version ISSN 1657-0790

profile vol.20 no.1 Bogotá Jan./June 2018

https://doi.org/10.15446/profile.v20n1.68848 

Editorial introducción

Bibliometric Indicators and Their Accurate Use in the Ranking of Colombian Journals*

Melba Libia Cárdenas **  

María Claudia Nieto Cruz ***  

** Journal Editor, Revista Profile Issues in Teachers’ Professional Development, Universidad Nacional de Colombia, Colombia

*** Journal Director, Revista Profile Issues in Teachers’ Professional Development, Universidad Nacional de Colombia, Colombia


The service [Google Scholar Metrics - GSM] can offer a better alternative than the traditional Google Scholar service to discover and judge the standing of journals through the prism of their citedness. GSM could become a potentially useful complementary resource primarily by virtue of its brand recognition, and the convenience of not requiring the installation of additional software, but currently its bibliometric indicators are often inappropriate for decision making in matters of tenure, promotion, grants and accreditation. (Jacsó, 2012, p. 604)

After a long period of debates and communications from academic groups involved in the production of journals in Colombia, Colciencias published its latest classification of academic journals based on the criteria established in its latest policies (Colciencias, 2016a, 2016b). Colciencias, the Colombian research agency in charge of the classification of the scientific journals edited within the country, ran the model announced in such policies and the results produced different reactions. For some scholars, those results were not surprising for the main parameters are in tune with international trends, that is, the recognition of quality in scientific or academic journals according to measurements established by prestigious databases and indexing systems (the JCR - Journal Citation Reports, led by Thomson or the SJR - Scimago Journal Ranking, managed by Scopus). For others, and as pointed out in our latest edition (Cárdenas & Nieto Cruz, 2017), there is a heavy reliance on number of citations made about or of articles as a means to quantify journals and researchers’ impact.

As stated in the call for applicants, the criteria used by Colciencias (2016a) to evaluate and rank the journals were applied in three phases. The first one embraced evaluation of editorial processing and punctuality in publication; the second had to do with the validation of the evaluation process as well as the journals’ international visibility; and the third one considered the impact of the journals. This last stage was given the highest prominence and was based on the measurement of the journals’ impact, as evidenced in their positions in the four quartiles set out by the JCR or the SJR and the h-index (h5). They, in the end, seem to be the most determining factors in the classification of the journals in one of the four categories: A1 (the utmost), A2, B, or C (the lowest).

Concerning the impact, we can observe a clear discrimination in the value granted to the JCR and SJR reports in contrast to that assigned to the h5. Whereas the former determine the inclusion of the journals in the highest categories (A1 and A2), the h5 determines the inclusion of the publications in categories B or C. Another issue to be raised in connection to the disparity has to do with the fact that the JCR and SJR reports are derived from citations made of journals and articles that belong to their databases, whereas the h5 is intended to measure impact in a wider spectrum, that is to say, it covers journals and articles from different regions no matter if they are present in those two reports.

Let us remember that the h5 index scrutinises the number of citations made per article in a period of 5 years. It also measures researchers and journals impacts.

h-index (h5): The h-index of a publication is the largest number h such that at least h articles in that publication were cited at least h times each. For example, a publication with five articles cited by, respectively, 17, 9, 6, 3, and 2, has the h-index of 3. . . . The h5-index . . . of a publication [is] . . . the h-index . . . of only those of its articles that were published in the last five complete calendar years. (Google Scholar Metrics, n.d., par. 1-4)

On September 15, 2017, Colciencias issued the results of its most recent classification process. The Profile journal was not classified in any of the categories because, according to Colciencias, the h5 of the journal was 1, which is not enough to be included in the National Bibliographical Index. Immediately, we wrote to Colciencias asking for a revision of this item because what this would indicate was that, during the past 5 years, when Profile has published around 120 articles, only one of them has received one citation. We analysed the citation exercise and found out that the h5 of the journal is 12, not 1, and with 12 the journal would be classified. Finally, on October 6, we received from Colciencias an answer to our request in which it acknowledges de validity of an h5 of 12 for the journal. Therefore, the journal was officially classified in Category B.

The situation we went through to get the classification also raised questions about the preciseness of measurements used by Colciencias. As a means to illustrate this, we contrasted the h5 of seven journals specialized in the area of education and ELT (English language teaching), as shown in the table below. The table depicts the results obtained by using Google Scholar Metrics. Interestingly, five journals were classified in category B and the h5 of their printed version ranged between 3 and 12. With respect to this measurement, when comparing, for example, Journal 1 (h5 = 3) and Profile (h5 = 12), we cannot understand why they both got the same classification if there is a big difference in their h5.

Journal h5 Printed version of the journal h5 Electronic version of the journal Classification granted by Colciencias
1 3 2 B
2 9 5 B
3 9 2 B
4 1 C
5 7 3 B
6 3 C
Profile 12 1 B

Regarding the use of the h5 in the latest classification by Colciencias, we agree with Jacsó (2012), who acknowledges the usefulness of Google Scholar Metrics as a complementary source and warns us about the limitations of its bibliometric indicators. We wonder if more detailed information provided by Colciencias with respect to the indicators borne in mind to endow the latest classification would help us ponder such limitations. Nonetheless, we still feel that the recognition of evaluations by local initiatives for Latin America, like SciELO and Redalyc, as well as of many other databases, would add to the examination of our publications.

In the same line of thought, questions have been raised in connection to the convenience of indicators mainly based on JCR and SJR reports, which are very restrictive for journals like ours (Cárdenas, 2016). All in all, the latest classification and Colciencias’ determination to rule the evaluation of journals edited in Colombia, with apparently no regard for scholars’ critiques and suggestions, show that there might be no chances to revise the implications of their policies for decision-making in regard to the journals’ accreditation and related matters-incentives for university professors and recognition of research groups’ products, among others.

Despite what has happened with our recent classification, we thank Colciencias for having considered our arguments. The inclusion of Profile in the national ranking system is vital for accreditation purposes of the Colombian authors who have published in the journal. We also thank our reviewers, authors, and readers for their continuous support. We only hope that now that the results of this new classification model are out, Colciencias would go over the limitations of such a model and listen to the suggestions proposed by editors and scholars so that the Colombian research output, rather than being stifled, is properly recognised and shared. After all, that should be the main concern of a research agency like Colciencias as it is for all academic publications and the staff behind them.

We have gathered twelve articles in this edition; five of them are authored by Colombian scholars and the other seven come from international authors writing from Argentina, Australia, Brazil, Chile, Mexico, and the United Kingdom. One article is the result of joint efforts between Chile and Australia.

The first section, Issues from Teacher Researchers, includes eight articles concerning language action research, English language teacher education, and language learning and identity. We open this issue with an article by joint authors Anne Burns from the University of New South Wales (Australia) and Anne Westmacott from the Universidad Chileno-Británica de Cultura (Chile) about the experiences of teachers who want to become researchers through the implementation of a new action research programme in the mentioned Chilean university. The study describes the action research programme developed with five English as a foreign language teachers in 2016, and discusses some reflections on this first year of the programme. As a direct product of the programme described in the first article, the action research project developed by Paul Anthony Dissington at the Universidad Chileno-Británica de Cultura (Chile) addresses the problem of negative lexical transfer errors in Chilean university students studying English as part of their degree programmes. The purpose of the action research cycle was to raise awareness of lexical transfer through focused attention on common transfer errors. Students valued the experience as positive in order to achieve productive mastery.

Regarding the topic of language learning, Carol Anne Ochoa Alpala and William Ricardo Ortíz García from Universidad Santo Tomás (Colombia) present a study in the development of oral skills in Colombian university students studying English as part of their degree programmes in collaborative environments through the use of different technological gadgets as are videos and 3D virtual worlds. The results showed how students worked collaboratively to achieve their goal of improving their oral presentation skills.

In the fourth article, Argentinian author Darío Luis Banegas, on behalf of the University of Warwick (United Kingdom) and the Ministerio de Educación del Chubut (Argentina), examines the conceptions of research held by English as a foreign language teachers in this South American country. He found that teachers felt marginal as regards research and claim a lack of time for not engaging in/with research. However, the engagement in action research proved to be meaningful to them resulting in teacher development, agency, empowerment, and autonomy.

Edgar Augusto Aguirre Garzón from the Pontificia Universidad Javeriana (Colombia) concentrated his efforts in a case study that dealt with the co-construction and transformation of the curriculum of a public school through the collaborative-reflective inquiry of four unlicensed EFL teachers. Findings suggest that collaborative-reflective inquiry encourages the language teachers to design teaching strategies and materials at the service of their public school. The teachers’ joint efforts resulted in curricular changes to adjust contents and goals of the area with the students’ contextual reality.

Next, we have an article on the topic of identity. Roderick Lander from the UCL Institute of Education (United Kingdom), examines this topic in the Colombian context. The author invited three gay male teachers working in Colombia to participate in a narrative research project framed within a post-structural perspective on identity. The study reveals that the three participants live their queer identity alongside their language teacher identity with ease although they have found instances of homophobia which have impacted their day-to-day lives and their careers.

John Jairo Viáfara and José David Largo from the Universidad Pedagógica y Tecnológica de Colombia, analyze the context of tertiary education taking into consideration the perceptions of master’s program students regarding the influence these graduate courses exert on their professional development. The strongest effects are evident in their practice and in the research skills gained through innovation, reflection, and collaboration. Their administrative and political stands seem to require further integration. In the Argentinian context, María Gimena San Martín from Universidad Nacional de Córdoba, examines an English as a foreign language teacher education programme where a supervisor supports the student-teachers’ learning-to-teach process in the context of one-to-one tutoring sessions. This scaffold help should be understood in relation to the function it serves and how it accommodates the students’ level of understanding.

In the second section, Issues from Novice Teacher-Researchers, we include a study by José Irineo Omar Serna-Gutiérrez and Irasema Mora-Pablo from the Universidad de Guanajuato (Mexico). The study deals with the identity of transnational student-teachers in Central Mexico. It explores the reasons why these student-teachers pursued a career in English language teaching. These reasons, recognized as critical incidents, trace a path to the construction of their identity as English teachers.

The last section, Issues Based on Reflections and Innovations, includes three articles this time. In first place, Diego Fernando Macías from Universidad Surcolombiana, in Colombia, reviews studies in the area of classroom management in foreign language education. The reflection deals with three areas of interest: the distinctive characteristics of foreign language instruction, a description of classroom management issues, and the different alternatives towards reducing the negative impact of classroom management on foreign language classes. In conclusion, the author suggests that there is a need for more research, particularly on the relationship between classroom management and aspects such as target language use and teaching methods.

The second article is authored by Débora Izé Balsemão Oss who works at the Caxias do Sul Municipality in Brazil. She presents her views regarding the relevance of teachers’ practical knowledge in the development of teacher Education programs. The author’s thesis is that teachers’ personal practical knowledge is likely to evolve into professional knowledge provided that it is analyzed, verified, and improved.

In the third article, we find Frank Giraldo’s reflections from Universidad Tecnológica de Pereira (Colombia) with respect to language assessment literacy. This reflection article reviews assessment literacy, understood as the knowledge, skills, and principles needed for assessment, from general education experts and language education scholars and shows how the meaning of language assessment literacy has expanded. The article presents implications for language teachers and proposes a core list of assessment knowledge, skills, and principles for teachers and students.

We would like to close this editorial by giving a warm welcome to Professor Jill Burton (University of South Australia) who has come back to the Editorial Review Board of the journal after a recess when she was busy with other endeavours. As before, her contributions along with those of the other members will greatly help us carry on with this editorial enterprise whose ultimate goal is to contribute to fostering teacher research, better ELT, and quality in education.

References

Cárdenas, M. L. (2016). Challenges of the HOW journal in spreading teachers’ works in times of ranking pressures. HOW, 23(2), 35-57. http://dx.doi.org/10.19183/how.23.2.333. [ Links ]

Cárdenas, M. L., & Nieto Cruz, M. C. (2017). Editorial: Resisting exclusion from core indexing systems. Profile: Issues in Teachers’ Professional Development, 19(2), 7-11. https://doi.org/10.15446/profile.v19n2.64604. [ Links ]

Colciencias. (2016a). Convocatoria No. 768 de 2016 para indexación de revistas científicas colombianas especializadas - Publindex (términos de referencia firmados) [Call No. 768, 2016 for the classification of Colombian specialized scientific journals - Publindex: Approved document] . Bogotá, CO: Author. Retrieved from http://www.colciencias.gov.co/sites/default/files/upload/convocatoria/terminosdereferencia-conv768-2016-firmados.pdf. [ Links ]

Colciencias. (2016b, Mayo). Documento No 1601: política para mejorar la calidad de las publicaciones científicas nacionales (versión para discusión) [Document No. 1601: Policies to improve the quality of the national scientific publications (for discussion)]. Bogotá, CO: Author. [ Links ]

Google Scholar Metrics. (n.d.). https://scholar.google.com/intl/en/scholar/metrics.html#metrics. [ Links ]

Jacsó, P. (2012). Google Scholar Metrics for publications: The software and content features of a new open access bibliometric service. Online Information Review, 36(4), 604-619. https://doi.org/10.1108/14684521211254121. [ Links ]

* How to cite this article (APA, 6th ed.): Cárdenas, M. L., & Nieto Cruz, M. C. (2018). Editorial: Bibliometric indicators and their accurate use in the ranking of Colombian journals. Profile: Issues in Teachers’ Professional Development, 20(1), 7-12. https://doi.org/10.15446/profile.v20n1.68848.

Received: October 30, 2017; Accepted: November 10, 2017

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License