SciELO - Scientific Electronic Library Online

 
vol.31 suppl.1The reference setting for engineering in ColombiaThe quality of engineering education is a key factor in developmental author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

  • On index processCited by Google
  • Have no similar articlesSimilars in SciELO
  • On index processSimilars in Google

Share


Ingeniería e Investigación

Print version ISSN 0120-5609

Ing. Investig. vol.31  suppl.1 Bogotá Aug. 2011

 

The academic impact of the Universidad Nacional de Colombia's Faculty of Engineering on the Bogota campus

Daniel Bogoya M.1

1 Chemical Engineer and Master in Systems Engineering of Universidad Nacional de Colombia. Dean of the Faculty of Natural Sciences and Engineering at the Universidad Jorge Tadeo Lozano. daniel.bogoya@utadeo.edu.co


ABSTRACT

An analysis regarding the quality of higher education programmes in Colombia is presented by comparing students' scores from a given institution on the ECAES 2009 test to students' performance throughout the country. The analysis uses the valueadded concept as applied to the academic world, this being obtained by using a regression model to ascertain the difference between the students' scores from a given faculty programme and the corresponding scores for students throughout Colombia; the higher education admission test was used to control initial results with results achieved by the same individuals on this test., A case study was proposed to illustrate the methodology by considering the Faculty of Engineering programmes at the Universidad Nacional in Bogota. Two different approaches were followed; the first used the generated function between SABER 11 and ECAES test scores and the second added institutional results by using the mean of the scores. The aforementioned Faculty of Engineering ratifies its relevance and importance as all its programmes come within the greater academic valueadded area.

Keywords: ECAES, academic valueadded, educational equity.


Received: march 31th 2011 Accepted: june 27th 2011


Introduction

The ECAES tests which are taken annually by students in the last year of university studies in Colombia (nowadays called SABER PRO) were applied for the seventh time in 2009. After processing the information from the field work, the Colombian Institute for the Assessment of Education (ICFES) released the pertinent databases to facilitate research work within the academic community aimed at acquiring knowledge regarding the quality of Colombian higher education programmes and exploring relationships between results and the variables which each community may choose to investigate.

Hypothesis

This article attempts to develop different elements to support the following hypothesis:

An educational institution is relevant and useful as long as it can offer greater academic added value to its students than that observed in other individuals being educated/trained within other educational institutions.

The programmes offered by the Universidad Nacional's Faculty of Engineering in Bogota were thus chosen to support the hypothesis. The scores obtained by students in such programmes on higher education admission test, (nowa days called SABER eleven) were used as a baseline or input variable for the proposed model and the scores obtained by the same students on the ECAES 2009test (nowadays called SABER PRO) as the output variable.

Background

The academic valueadded (AVA) concept proposed here for higher education comes from its already observed use in primary and secondary education where it has been defined as a statistical process providing parameters for estimating teachers, educational institutions and school systems' influence (Sanders & Horn, 1994; p. 301). By comparing the performance of students from two different schools on a given test, during two different stages, it can be estimated how far ahead or behind a student is compared to the others during both stages if the difference between scores achieved by students from each institution and the corresponding value for the whole country has been established (Goldstein, 2001; p. 251-252).

It is expected that an educational project will be more effective than others when it acquires greater AVA (i.e. students achieve greater levels of understanding shown by the greater difference between test scores obtained by such students at the beginning and end of a given period of time in relation to differences regarding students from other educational projects within the area being studied). Several factors may explain an institution's greater effectiveness, such as setting up networks, shared planning of academic activities as the basis for cooperation, revising shared goals for both education and research and mutual recognition of quality (Mikkola & Snellman, 2006; p. 5354). Nevertheless, applying tests at the beginning and end of an educational project to gather information about its input and output (in the same way as the concept being used in economics) usually leads to strong arguments for both researchers and policymakers regarding value judgments, conceptual assumptions and technical decisions implicit in choosing what should be assessed by a given test (Saunders, 2010; p. 253254).These questions become even more important in the field of education since a whole constellation of factors influence achieving higher levels of quality as has been argued by some researchers at the Jyväskylä University Institute of Educational Research (Välijärvi et al, 2002; pp. 1546).

Colombia

The AVA concept started to be used in Colombia in 2006by means of test scores obtained by students who were finishing their higher education programmes; this was three years after the ECAES project had become implemented throughout the whole of Colombia. A mathematical regression model was used as well as a corresponding graphical representation for test scores obtained on the higher education admission test (on the horizontal axis) as input variable and the corresponding test scores obtained on the ECAES test for academic programmes (on the vertical axis) as output variable: systems engineering, law and physiotherapy in 2003 and physiotherapy again in 2004. The proposal considered two levels of analysis: student sand groups of students by higher education institution.

Both the shape and position of the functions generated by the mathematical regression model showed the magnitude of each programme's AVA in relation to the value of the other programmes, taking into account standardreferenced assessment (i.e. always in relation to the entire target population) (Bogoya, 2006; p. 1823).

Procedure

The following four steps were carried out to support the hypothesis proposed here.

Step 1. Conceptual model

A general model was adopted to represent processes involved within such project (see Figure 1) for applying the AVA concept to educational project relevance and importance analysis, where xij represents input variable whose value corresponds to the score obtained by student (from educational project j) on the higher education admission test; yij represents the output variable whose value corresponds to the score obtained by the very same student ion the ECAES 2009 test. It should be mentioned that scores obtained on such tests in Colombia represent the most stable, objective and standardised source for assessing high school and university students' cognitive development and understanding. To complete the scheme, zij indicates a second input variable referring to the potential influence of student i within project j.

Step 2. Source of information

The official databases released by ICFES (see the website ftp://ftp.icfes.gov.co) provided the main source of information; students' identification codes were extracted, along with their assigned scores on the higher education admission test (input) and the ECAES 2009 test (output). The codes allowed the researcher to identify each student's programme and institution. Information relating assigned test score to students on the tests mentioned above was also used.

Step 3. Graphic representation

The relationship between the associated model' sinput and output variables was represented on a coordinate grid for graphically illustrating a given educational institution's AVA by means of the assigned score on the higher education admission and graduation tests against the relationship found for other institutions forming part of the target population whose students took the same tests. Such representation was done at individual level to explore the diversity of output values corresponding to the same input value and also for each institutional by averaging score son the given tests to appreciate an institution's overall influence.

Step 4. Analysis

Context was omitted from the analysis proposed here, as the objective information for estimating how this variable's mode and magnitude would affect output was not available; such omission was equivalent to assuming a similar context influence having the same value for all educational projects in the universe being observed, or taking an influence into account within the academic impact of such educational projects. Analysis was carried out at student level or by summarising or grouping the data, calculating averaged scores on admission and graduation and comparing the relationship found for an educational institution and the corresponding relationship for the whole target population.

Results

The case study results (supporting the hypothesis mentioned above) are presented in three sections: general statistics, students AVA and institutional AVA for each programme.

General statistics

The number of students evaluated and their assigned scores' means and standard deviations for each programme and throughout Colombia (Tables 1 to 8) provided an overall picture of the results obtained by Universidad Nacional Faculty of Engineering students in Bogota on the ECAES 2009 test. It should be pointed out that countrywide data also contained information for the whole of the Universidad Nacional in Bogota.

In all cases, the overall average for Universidad Nacional Faculty of Engineering students in Bogotá was greater than the national average, difference ranging from 9.97 points for the chemical engineering programme to 15.36 points for the systems engineering programme.

Student AVA

Figure 2 shows the relationship between the model's student input (horizontal axis) and output (vertical axis) for the Universidad Nacional's agricultural engineering programme in Bogotá and for the other agricultural engineering programmes in Colombia (see Figure 3), using just cases for which results on both tests were available (64% Universidad Nacional and 67% for the whole of Colombia), where students' location was highlighted by greater AVA.

The regression-generated tendency lines for input and output using a second degree polynomial have been represented on the same diagram to facilitate comparisons between the results obtained by the group of Universidad Nacional agricultural engineering students in Bogotá and the group made up by students from the other institutions across the country (see Figure 4).

The Universidad Nacional agricultural engineering programme in Bogotá had the greatest AVA compared to the group of other programmes in Colombia, for the whole generated function domain, as output value (students' learning on the vertical axis) was always larger than the corresponding value for students attending other institutions and who had similar admission conditions (when reading the results for the same xvalue on the horizontal axis).

Likewise, students' AVA has been shown for civil, electrical, electronic, chemical, industrial, systems and mechanical engineering programmes at Universidad Nacional in Bogotá in relation to other programmes throughout Colombia (see Figures 2 to 11) using inputgenerated (higher education admission test) and outputgenerated (ECAES 2009 test data) tendency lines.

Again, using just the students for whom it was possible to find results for both tests (51% to 84% of cases), the electrical, electronic, chemical and systems engineering programmes at the Universidad Nacional in Bogotá also had greater AVA than the group of other programmes across the country, for the whole generated function domain (see Figures 6, 7, 8 & 10).Civil, industrial and mechanical engineering programmes (due to the observed intersection of the two represented functions) had two opposing patterns (see Figures 5, 9, & 11): greater AVA was seen in relation to results across Colombia within the low performance area (to the left of the intersection) and AVA was lower than that across Colombia within the high performance area (to the right of the intersection).

Academic added value (AVA) by programme

This section presents the overall academic impact of the Universidad Nacional's engineering programmes in Bogotá for all cases greater than the trend of the set of the other programmes in Colombia, using the test result averages on both tests and grouping the students by higher education institution, using just the information for those students whose results on both tests were available (see Figures 12 to 19).

Conclusions

Even though the factors influencing university students' education are diverse and are related to each other in complex ways, it would be expected that a higher education programme would provide greater academic added value and achieve greater impact as long as it increases the students' understanding level, from the beginning to the end of their studies, to a greater extent than that of other educational institutions regarding the target population.

Agricultural, electrical, electronic, chemical and systems engineering programmes at the Universidad Nacional in Bogotá had greater AVA than the group of other programmes across the country for the whole generated function domain. Civil, industrial and mechanical engineering programmes had two opposing patterns; they had greater academic added value than the group of other programmes across the country within the low performance area where the students presented greater learning difficulty but had lower AVA within the high performance area.

The academic impact, relevance and importance in Colombia of the eight engineering programmes at the Universidad Nacional in Bogotá was seen to be due to the fact that they bestow overall academic added value which was always greater than the corresponding value found by regression for the other programmes offered across the country, given that the input value for the considered model was kept constant (i.e. average score on the higher education admission test).


References

Bogoya, D., Evaluación Educativa en Colombia., Memorias del Seminario Internacional de Evaluación, Cartagena, ICFES, febrero, 2006, pp. N1-N27.

Goldstein, H., League Tables and Schooling. Science in Parliament., Vol. 58, No. 2, 2001, pp. 251-252.

Mikkola, M., Snellman, O., Evaluation of CIMO NorthSouth Higher Education Network Programme., Ministry for Foreign Affairs of Finland &- Department for Development Policy, Helsinki, Hakapaino Oy, 2006, pp. 53-54.

Sanders, J. L., Horn, S. P., The Tennessee Value Added Assessment System (TVAAS): Mixed-Model Methodology in Educational Assessment., Journal of Personnel Evaluation in Education, Vol. 8, 1994, pp. 299 -311.

Saunders, L., A brief History of Educational 'Value Added': How Did We Get To Where We Are? School Effectiveness and Scholl Improvement, Vol. 10, No. 2, 2010, pp. 233-256.

Välijärvi, J., Linnakylä, P., Kupari, P., Reinikainen, P., Arffman, I., The Finnish Success in PISA &- and some reasons behind it., Finlandia, Kirjapaino Oma Oy, 2002, pp. 15-46

Bogoya, D., Evaluación Educativa en Colombia., Memorias del Seminario Internacional de Evaluación, Cartagena, ICFES, febrero, 2006, pp. N1-N27.        [ Links ]

Goldstein, H., League Tables and Schooling. Science in Parliament., Vol. 58, No. 2, 2001, pp. 251-252.        [ Links ]

Mikkola, M., Snellman, O., Evaluation of CIMO NorthSouth Higher Education Network Programme., Ministry for Foreign Affairs of Finland &- Department for Development Policy, Helsinki, Hakapaino Oy, 2006, pp. 53-54.        [ Links ]

Sanders, J. L., Horn, S. P., The Tennessee Value Added Assessment System (TVAAS): Mixed-Model Methodology in Educational Assessment., Journal of Personnel Evaluation in Education, Vol. 8, 1994, pp. 299 -311.        [ Links ]

Saunders, L., A brief History of Educational 'Value Added': How Did We Get To Where We Are? School Effectiveness and Scholl Improvement, Vol. 10, No. 2, 2010, pp. 233-256.        [ Links ]

Välijärvi, J., Linnakylä, P., Kupari, P., Reinikainen, P., Arffman, I., The Finnish Success in PISA &- and some reasons behind it., Finlandia, Kirjapaino Oma Oy, 2002, pp. 15-46        [ Links ]

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License