SciELO - Scientific Electronic Library Online

vol.35 issue64Individual unlearning from the point of view of managers in merger and acquisition events in Brazil*Perceptions of the Colombian business sector regarding its role in the post-conflict author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand



Related links

  • On index processCited by Google
  • Have no similar articlesSimilars in SciELO
  • On index processSimilars in Google


Cuadernos de Administración (Universidad del Valle)

Print version ISSN 0120-4645

cuad.adm. vol.35 no.64 Cali May/Aug. 2019 

Artículo de investigación científica y tecnológica

Sustainability index assessment based on the units of sustainable university development: a case study on a colombian university

Índice de sostenibilidad, evaluado mediante unidades de desarrollo sostenible universitario: caso de estudio universidad colombiana

Indice de durabilité, évalué par les unités universitaires de développement durable: étude de cas d’une université colombienne

0000-0003-1224-3846Edgar Felipe Cortés-León1  , 0000-0003-3184-919XFernando Gutiérrez-Fernández2 

1 Associate Professor, Environmental Engineering Program, Universidad El Bosque, Bogotá, Colombia. Envi-ronmental Engineer, Master’s Degree in Environmental Business Management, Universidad El Bosque, Bogotá, Colombia. e-mail:

2 Full Professor, School of Engineering, Universidad El Bosque, Bogotá, Colombia. Ecologist, Pontificia Universidad Jave-riana, PhD in Sustainable Development and Ecodesign, Universidad Politécnica de Valencia, Spain. Water, Health and Environment Research Group, Universidad El Bosque, Colombia. e-mail:


To build the University Sustainability Index (USI), proposed in this article, a 7-step methodology was designed: Review of indicators (International, national and specific), Selection of indicators by means of the linear weighting method (Scoring), design of the requirements tree (proposed by Alarcón, 2005), panel of experts (according to the method proposed by Cyret and March, 1965), validation of the system (carried out using the Delphi method), distribution of weights using the AHP methodology and finally the construction value functions, to be able to homogenize the values and be able to aggregate them in a single measure, with which to make decisions at the strategic level of the organization, however, if decisions are required at the tactical and operational level, the value can be disaggregated into units of lower levels, which is proposed an index that covers the entire company. The proposed system allows comparisons to be made, and best practices to be analyzed, allowing acceptable values in indicators to be achieved at lower cost and more quickly.

Key words: Sustainable development; Indicators; Index


Para construir el Índice de Sostenibilidad Universitario (USI), propuesto en este artículo se diseñó una metodología de 7 pasos: Revisión de indicadores (Internacionales, nacionales y específicos), Selección de indicadores por medio del método de ponderación lineal (Scoring), diseño del árbol de requerimientos (propuesto por Alarcón, 2005), panel de expertos (de acuerdo al método propuesto por Cyret y March, 1965), validación del sistema (realizado mediante el método Delphi), distribución de pesos mediante la metodología AHP y finalmente la construcción de funciones de valor, para poder homogeneizar los valores y poder agregarlos en una única medida, con la cual tomar decisiones en el nivel estratégico de la organización, sin embargo, si se requieren decisiones en el nivel táctico y operativo se puede desagregar el valor en unidades de niveles inferiores, con lo cual se propone un índice que permite abarcar la totalidad de la empresa. El sistema propuesto permite realizar comparaciones, analizar las mejores prácticas, permitiendo alcanzar con menor costo y mayor prontitud valores aceptables en los indicadores.

Palabras-clave: Desarrollo sostenible; Indicadores; Índice


Afin d’élaborer l’indice de durabilité des universités, proposé dans le présent article, une méthodologie en sept étapes a été élaborée : revue des indicateurs (internationaux, nationaux et spécifiques), sélection des indicateurs par la méthode de pondération linéaire (Scoring), conception de l’arbre des exigences (proposé par Alarcón, 2005), panel d’experts (selon la méthode proposée par Cyret et March 1965), validation du système (réalisée par la méthode Delphi), distribution des poids par la méthodologie AHP et enfin construction des fonctions valeur. Afin d’homogénéiser les valeurs et de pouvoir les additionner en une seule mesure, avec laquelle prendre des décisions au niveau stratégique de l’organisation, cependant, si des décisions sont requises au niveau tactique et opérationnel, la valeur peut être désagrégée en unités aux niveaux inférieurs, avec lesquelles un indice est proposé qui permet de couvrir l’ensemble de l’entreprise. Le système proposé permet d’effectuer des comparaisons, d’analyser les meilleures pratiques et d’atteindre des valeurs acceptables pour les indicateurs à moindre coût et plus rapidement.

Key words: Développement durable; Indicateurs; Indice

1. Introduction

Several authors have discussed the addition of sustainability activities to the teaching-learning processes that are present within the graduate program curricula (Minguet, Ull, Piñero, and Martínez-Agut, 2013), with second-level priority being assigned to accountability due to the low environmental impact attributed to the activities performed by universities (Melle, 2007).

However, this situation has changed in the previous decade during which the community, especially the educated society, dependent on universities, has become considerably sensitive toward achieving sustainability. Therefore, higher education institutions require a tool that will help them to measure the sustainability of the activities that they perform by considering the concept of sustainable development, which can be understood as a “scheme of human, social, and economic development capable of indefinitely remaining in harmony with the biophysical systems of the planet” (Schuschny and Soto, 2009), as the reference.

The index proposed in this study assesses a set of indicators and evaluates the sustainable development at any university, regardless of its type or structure. Currently, there are different kinds of indicators that are proposed by different organizations such as the UI GreenMetric World University Rankings from the University of Indonesia or the university sustainability indicator network (RISU). However, none of these tools uniformly assess and aggregate sustainably into a single measurement, which may indicate the degree of sustainability associated with the activities performed by universities with respect to the senior management or public in general.

For this project, we focus on a university with high-quality certification in the city of Bogotá, which has been developing different projects to improve the environmental issues with respect to the elements of social responsibility. This institution was selected based on the following two central criteria: access to information by the research group and its level of information availability.

The developed index improves the decision-making process of a university at both the strategic and tactical levels. In addition, this index, which is also tested for usability, is an administrative tool that groups the selected indicators by converting them from subjective to objective because they are selected by experts and not based on their application convenience or any other individual criterion of the organization.

The selected indicators were later validated based on their relevance as sustainability measures. Simultaneously, their importance within the proposed system was also determined by means of specific weights. Finally, they were added or indexed by constructing value functions to homogenize different measurement units into a single unit denoted as units of sustainable university development (USUD), thereby laying the foundations of the university sustainability index (USI).

2. Theoretical Framework

Kaplan and Norton (1998) were the first authors who used a tool similar to the one that was proposed in this study. Even though their tool was similar to our tool in rationale, the indicators that were used were completely different. In addition, they believed in measuring the results and simultaneously proposed that the current financial-indicator-based corporate performance measurement approaches were becoming obsolete (Kaplan, Norton, and Santapau, 1997). Their proposal is essentially based on four perspectives, including financial, process, customer, and capability perspectives.

Subsequently, the sustainability balanced scorecard, as proposed by Figge, Hahn, Schaltegger, and Wagner (2002) in their article entitled “The Sustainability Balanced Scorecard-Theory and Application of a Tool for Value-Based Sustainability Management,” became a popular tool for managing value-based sustainability although it was based on the perspectives initially proposed by Kaplan, Norton, and Santapau (1997).

The studies that described multicriteria analysis tools used for selecting indicators as a decision-making exercise to maintain competitiveness in a company constituted the next indicator development milestone. Here, the participants are asked to select one or more alternatives from a list of options, and the results are obtained based on the assessment of desirable properties or attributes that the different alternatives should satisfy, with the simultaneous fulfillment of all the alternatives being mutually exclusive in some cases (Díaz, Ballester, Alcaraz, and Iniesta, 2012).

This inclusion of multicriteria techniques, especially the analytical hierarchical process (AHP) proposed by Saaty in 1980, allowed Martínez, 2007, to “eliminate improvised conjectures, the unexplained, unjustified, and intuitive thinking that sometimes accompanies the majority of our decisions regarding complex problems” (p. 525).

Subsequently, AHP is coupled with third-party consultation, similar to the article entitled “Proposal of a System of Sustainability Indicators for Natural Tourist Attraction Areas, validated by Third-Party Consultation.” As an example, the analysis assessed 19 systems of indicators with different typologies; based on subject-matter-expert consultation and using the three-tier interview proposed by Cloquell Ballester, Cloquel Ballester, Monterde-Diaz, and Santamarina-Siurana, 2006 (Figure 1), the final sustainability indicators were selected, building a model based on a systemic approach to ensure sustainable development (Gutiérrez-Fernández, Cloquell Ballester, and Cloquel Ballester, 2012).

Figure 1 Decision process leading to an ad hoc indicator design  

Recently, the participatory construction of indicators has gained strength, as described in the article entitled “Evaluation of Sustainability Indicators in Los Jazmines, Viñales, Pinar del Río, Cuba,” wherein 166 community members were polled and eight people were interviewed and all of them were directly or indirectly linked with the environmental activity in the investigated community (Pérez, Linares, Marques, Vento, and Díaz Pérez, 2018).

3. Methodology

Based on the review of the state of the art, a seven-step methodology is constructed as per the specifications obtained from Gutiérrez (2011) and Cortés (2016) (Figure 2).

Figure 2 Methodology used for the development of University Sustainability Index (USI) 

Subsequently, each step of the proposed methodology is described.

3.1. Step 1: Review of Indicators

The initial step during project development was to conduct literature review of the indicators used by different organizations and institutions, such as the World Bank, the National Accreditation Council of Colombia, the Colombian Environment Information System (SIMA), the Economic Commission for Latin America and the Caribbean, and the United Nations Environment Program, to measure and monitor the sustainability as well as the indicators used by some European universities.

3.2. Step 2: Selection of Indicators through the Linear Weighting Method (Scoring)

We begin from a consolidated base of indicators obtained from the previous step, which, as can be observed from the results, were 179, to select indicators for index construction. Further, the scoring method, which was one of the discrete multicriteria decision methodologies, was applied to these selected indicators. This method is selected because, as per Berumen and Llamazares (2007), it is probably the best-known and most commonly used method for building a simple index, while simultaneously being robust with respect to the development methodology.

Thirteen elements, including the scientific validity, comparison, representativeness, sensibility against changes, data reliability, validity, relevance, comprehensibility, predictability, objective values, simplicity, geographical coverage, and availability, were initially considered to be the evaluation criteria. However, at the end, only the five elements that adapted to the development of this project and that allowed for its subsequent application to a case study or to any other university were selected.

Once the evaluation criteria were selected, a quantitative assessment scale was constructed based on their qualitative judgments in the following manner.

Criterion 1. Comparison: This indicator should provide a basis for performing comparisons and exchanging information between universities at the national and international levels.

  • One (1): The indicators are not comparable with those of another university.

  • Three (3): The indicators can be compared with those of another university.

  • Five (5): The indicators are comparable with those of any other university.

Criterion 2. Data Reliability: This denotes the probability that the indicators work under given conditions for a given period of time.

  • One (1): The indicators are not reliable.

  • Three (3): The indicators can be reliable.

  • Five (5): The indicators are reliable.

Criterion 3. Relevance: This denotes a quality or condition that can be considered to be important for both the research and the project. The following scores are obtained:

One (1): The information is irrelevant to the research study.

  • Three (3): The information may be relevant to the research study.

  • Five (5): The information is important and necessary for the research study.

Criterion 4. Simplicity: This denotes the ease of understanding the indicators and can be assessed as follows:

  • One (1): The indicator is not understandable for the project.

  • Two (2): The indicator is very difficult to understand for the project.

  • Three (3): The indicator is difficult to understand.

  • Four (4): The indicator is easy to understand.

  • Five (5): The indicator is very easy.

Criterion 5. Availability: This denotes how easily the information can be accessed at higher education institutions and can be rated as follows:

  • One (1): The necessary information does not exist in the organization.

  • Three (3): The information exists but is difficult to access.

  • Five (5): The information exists and can be easily accessed.

  • Finally, it was decided to average the answers given by the polled experts, resulting in an average equal to or higher than 4.7 as the cutoff point, to obtain the selection filters of the ideal indicators for constructing the USI.

3.3. Step 3: Design of the Requirement Tree

For constructing USI, the requirement tree methodology proposed by Alarcón, 2005, was used, which “(...) provides a global and general view of the problem from its hierarchy and deployment at different levels” (p.60).

3.4. Step 4: Selection of Experts

The selection of experts is an essential step with respect to the reliability index because these experts will effectively validate the indicators and the proposed system. Therefore, the method that was developed by Cyret and March (1965) and subsequently used by Grant and Davis (1997), Landeta (1999), Germain (2006), Salas, Rodríguez, Sagué, and Mena (2010), Gutiérrez (2011), and Cruz and Martínez (2012) was selected.

3.5. Step 5: Validation of the Indicator System by a Panel of Experts

In this step, the Delphi method is used for third-party consultation. This method is a structuring technique with respect to a group communication process whose objective is to obtain a degree of consensus from the experts on the proposed problem instead of leaving the decision to a single researcher, providing objectivity and universality to USI consolidation.

3.6. Step 6: Distribution of the Weights using the AHP methodology

This methodology was proposed by Saaty (1980) and has been extensively used in different studies, such as those of Li (2004), Mardle, Pascoe, and Herrero (2004), Wolfslehner (2005), Gutiérrez (2011), Martínez, Gómez, Ibarra, and Moncada (2018), and Paredes (2018), was used to evaluate the weight or importance of each element within the hierarchy (requirement tree).

3.7. Step 7: Construction of the Value Functions

The following steps were followed in accordance with Gutiérrez (2013) for constructing the value functions.

  1. Select the indicator values for the university sustainable development unit (USUD), and determine their range (minimum and maximum possible indicator values).

  2. Define the qualitative characteristics of the relation established using the value function (monotonicity, concavity, and convexity).

  3. Determine the values between function values in terms of USUD for different indicator values.

  4. Construct a curve by adjusting the relation values established in the previous step or interpolation.

  5. Validate the results obtained using the value function.

4. Results

4.1. Step 1: Review of the Indicators

A database containing 998 indicators was developed from the consulted entities based on the literature review. Then, the first filter was applied to assess their similarity by considering that the same indicator could be observed in two or more or even similar indicators with different names when reviewing several proposals. Thus, the initial number of indicators was reduced to 179 using which the index construction process was continued.

4.2. Step 2: Selection of Indicators using the Linear Weighting Method (Scoring)

Based on the consolidated list of indicators (179 indicators) obtained in the previous step, each indicator was rated based on the five criteria mentioned previously (comparison, data reliability, relevance, simplicity, and availability). Based on the initial list and after the application of the scoring method, 22 indicators achieved an average rating equal to or greater than 4.7.

4.3. Step 3: Design of the Requirement Tree

As mentioned in the methodology section, the requirement tree was used to restructure the index in four levels (Figure 3).

Figure 3 Requirement tree 

First level: main topic or USI.

  • Second level: dimensions of sustainable development.

  • Third level: topics or categories of each dimension.

  • Fourth level: management indicators that have been identified with a code for ensuring clarity.

4.4. Step 4: Selection of Experts

In this step, we used the expert selection methodology proposed by Cyret and March (1965). The knowledge coefficient (Kc) and the argumentation coefficient (Ka) were obtained from the initial list of possible experts. Thus, the competence coefficient (K) was obtained for each expert, and we identified 11 people whose K exceeded 0.8.

4.5. Step 5: Validation of the Indicator System by a Panel of Experts

Eleven experts (selected in the previous step) participated in this exercise. These experts assessed the proposed indicator system using a questionnaire (Annex 1), weighted using a Likert scale. In addition, before its application, this questionnaire was tested for consistency using psychometrics, resulting in a Cronbach’s alpha of 0.74.

To determine whether the experts approved the proposed system, their answers were tallied, and an average value of 4.0 was established as the cutoff point for ensuring agreement with the stated requirements. This cutoff point is lower than that proposed in step 2 because it was already sieved once. Besides, a cutoff point of 4.7 was considered to be too high.

It should be noted that the panel of experts included members from senior management, including a former Dean of Engineering and a former member of the Board of Directors.

Finally, values were obtained for the following questions as follows:

  • Question 1. Is the proposed indicator system consistent with the quantified conceptual evaluation (measurement of sustainable development at universities)? Average = 4.33.

  • Question 2. Is the proposed indicator system of practical utility for the pursued objective? Average = 4.33.

  • Question 3. Is there no duplication or overlap between the indicators proposed by the system? Average = 4.67.

  • Question 4. Are all the proposed indicators relevant? Average = 4.33

  • Question 5. Do the proposed indicators sufficiently assess sustainable development at universities? Average = 4.00.

4.6. Step 6. Distribution of Weights using the AHP methodology

In this step, the requirement tree (hierarchy) was weighted, and the following results were obtained.

The first level does not require a weight because there is only one element, i.e., the USI. In the second hierarchy level (dimensions), values of 33% were obtained for each of the dimensions. To avoid errors, the remaining 1% is applied to the environmental dimension, where a considerable number of categories and indicators are available for evaluation.

Dimensional Weights:

  • • Social dimension = 33%

  • • Economic dimension = 33%

  • • Environmental dimension = 34%

In the third level (categories), the following weights were obtained:

Within the Social dimension:

  • • Category: Policies = 75%.

  • • Category: Research = 25%.

Within the Economic dimension:

  • • Category: Budget = 75%.

  • • Category: Administration = 25%.

Within the Environmental dimension:

  • • Category: Transportation = 8%.

  • • Category: Energy = 20%.

  • • Category: Water Management = 20%.

  • • Category: Waste = 52%.

Finally, in the fourth level (indicators), weights were obtained, as denoted in Table 1.

Table 1 Weighting of the indicators 

Code Indicator Name % Weight
Social Dimension
Category: Policies 75
PO-2 Comprehensive participation in activities 100
Category: Research 25
IN-2 Comprehensive participation in activities 13
IN-3 Commitment to research and artistic/cultural creation 87
Economic Dimension
Category: Budget 75
PRE-1 Total budget 50
PRE-2 University budget for sustainability 25
PRE-3 Budget projects 25
Category: Administration 25
ADM-1 Sustainability policy 75
ADM-6 Urbanism and biodiversity 25
Environmental Dimension
Category: Transportation 8
TM-3 Average number of bicycles on campus 17
TM-4 Mobility policy 83
Category: Energy 20
E-1 Electric power consumption (kWh per capita) 12
E-2 Energy use policy 52
E-3 Carbon footprint calculation 29
E-4 Total percentage of energy-saving technologies at the university 7
Category: Water Management 20
GA-1 Discharge disposal 22
GA-2 Water conservation program 59
GA-3 Total percentage of water-saving technologies at the university 9
GA-4 Consumption of drinking water per capita 9
Category: Waste 52
GR-1 Recycling program at the university 63
GR-2 Hazardous waste management 13
GR-3 Treatment of organic waste 13
GR-4 Treatment of inorganic waste 13

Source: Author’s own elaboration.

4.7. Step 7: Construction of the Value Functions

Value functions were constructed for all the indicators included in the index. However, the value functions are explained below by considering only the sixth and seventh indicators of the 22 indicators as an example.

This sixth indicator is selected for being quantitative, whereas the seventh indicator is selected for being qualitative. In addition, they have different models of curvilinear regression estimation.

4.7.1. Project Budget.

To construct the value function presented in Table 2, we considered the 2009 budget of the university under study to be the foundation. This budget established 0.25 USUD for each value, increasing the units of sustainable university development as more research budget was allocated.

Table 2 Value function of the project budget 

Budget projects
Years Budget USUD
2009 $ 330.000.000,00 0.1
2010 $ 330.000.000,00 0.1
2011 $ 500.000.000,00 0.25
2012 $ 725.000.000,00 0.5
2013 $ 825.000.000,00 0.75
2014 $ 942.647.500,00 1

Source: Author’s own elaboration.

The SPSS Statistics 19.0 software was used to verify the constructed value function. While conducting regression analysis through a curvilinear estimation, the value function was identified to be quadratic regression, as observed in Figure 4.

Figure 4 The value function associated with the project budget indicator obtained using the SPSS Statistics 19.0 Software 

The analytic expression for this value function can be obtained as:

U.S.U.D=0,152+((-7,051*10^(-10) )*U.P.P)+((1,703*10^(-18) )*〖U.P.P〗^2 )

4.7.2. Sustainability Policy.

To construct the value function presented in Table 3, we considered the requirement for a sustainability policy at a university, which focuses on environment as a point of departure. Therefore, we performed quantitative assessment of the degree of environmental policy development.

Table 3 Value function for the sustainability policy 

[1] No policy in place 0.1
[2] Preparation stage 0.25
[3] Initial implementation stage 0.5
[4] The policy has been implemented with some problems 0.75
[5] The policy is completely implemented 1

Source: Author’s own elaboration.

To obtain the value function, the values are adjusted by applying regression analysis on the basis of curvilinear estimation. This procedure was performed using the SPSS Statistics 19.0 software, which generated curvilinear estimation statistics based on regression and charts.

The model that best fits the observed series of values is cubic, as can be observed from Figure 5.

Figure 5 Value function associated with the sustainability policy indicator obtained using the SPSS Statistics 19.0 Software  

The analytical expression of the value function is represented as:

The completely constructed index was applied to a high-quality certified university in the city of Bogotá from which we were able to collect the necessary information required for applying the proposed system because of the support from different administrative areas. The results are presented in Table 4.

Table 4 Data from the indicators applied to a high-quality university 

Indicator Number Indicator Code Indicator Name Indicator Data
1 PO-2 Mission, vision, and institutional project 1
2 IN-2 Comprehensive participation in activities 46.7
3 IN-3 Commitment to research and artistic/cultural creation 0.62
4 PRE-1 Total budget $7.177.470.000,00
5 PRE-2 University budget for encouraging sustainability 0.06
6 PRE-3 Project budget $ 942.647.500,00
7 ADM-1 Sustainability policy 4
8 ADM-6 Urbanism and biodiversity 0.1
9 TM-3 Average number of bicycles on campus 350
10 TM-4 Mobility policy 1
11 E-1 Electric power consumption (kWh per capita) 17.9
12 E-2 Energy use policy 4
13 E-3 Carbon footprint calculation 335
14 E-4 Total percentage of energy-saving technologies at the university 0.4
15 GA-1 Raw water disposal 1
16 GA-2 Water conservation program 4
17 GA-3 Total percentage of water-saving technologies at the university 0.60
18 GA-4 Consumption of drinking water per capita 0.28
19 GR-1 Recycling program at the university 4
20 GR-2 Hazardous waste management 3
21 GR-3 Organic waste treatment 1
22 GR-4 Inorganic waste treatment 2

Source: Author’s own elaboration.

Table 5 denotes the development applied to the high-quality university for obtaining 0.503 USUD with respect to the USI. In the first columns, the indicator number, with its corresponding code and name, is provided. The following columns denote the value function formula for each indicator and the results obtained for each indicator. Similarly, the relative indicator value, the relative indicator value within the category and, finally, the indicator value within the system are denoted.

Table 5 Indicator system for the selected university  

Indicator number Indicator code Indicator name Function formula Result Indicator data Weight of the indicator within the category Value of the indicator within the category Category Category weight Value of the indicator within the dimension Dimensions Weight of the dimension Value of the indicator in the system
1 PO-2 Mission, vision and institutional project U.S.U.D =0,07-0,052*1)+(0,089*1^2)-(0,008*1^3) 0.7 4 1.00 0.75 Politics 0.75 0.56 Social 33% 0.185
2 IN-2 Participation in activities in an integral way U.S.U.D =93+(0,007*0,5)+(1,855*10^(-5)*0,5^2) 0.5 50.0 0.13 0.06 Investigation 0.25 0.02 0.005
3 IN-3 Commitment to research and artistic and cultural creation U.S.U.D=0,082+(0,008*62)+(0*62^2)+((1,229*10^ (-7))*62^3) 0.6 62 0.87 0.53 0.13 0.044
4 PRE-1 Total Budget U.S.U.D=-0,738 +((3,954*10^(-10))*7177470000)- ((2,138*10^(-20))*7177470000^2) 1.0 $7.177.470.000 0.50 0.50 Budget 0.75 0.37 Economy 33% 0.124
5 PRE-2 University budget for encouraging sustainability U.S.U.D=0,099+(0,008*G6)+(*G6^2)+((1,229*10^ (-7))*G6^3) 0.62 62 0.25 0.16 Administrative 0.25 0.12 0.039
6 PRE-3 Project budget U.S.U.D=-0,136+((8,867*10^(-10))*942647500) +((-9,709*10^(-19))*942647500^2)+((1,398*10^ (-27))*942647500^3) 0.2 $ 400.000.000,00 0.25 0.04 0.03 0.009
7 ADM-1 Sustainability policy U.S.U.D=0,07-(0,052*4)+(0,089*4^2)-(0,008*4^3) 0.8 4 0.75 0.58 0.15 0.048
8 ADM-6 Urbanism and biodiversity U.S.U.D = 0,093+(0,007*1)+(1,855*10^(-5)*1^2) 0.10 1.0 0.25 0.03 0.01 0.002
9 TM-3 Average number of bicycles found on campus U.S.U.D = =0,093+(0,007*88)+(1,855*10^(-5)*88^2) 0.85 88 0.17 0.14 Transportation 0.08 0.01 Enviromental 34% 0.004
10 TM-4 Mobility policy U.S.U.D = 0,07-(0,052*1)+(0,089*1^2 )-(0,008*1^3) 0.1 1 0.83 0.08 0.01 0.002
11 E-1 Electric power consumption (kWh per capita) U.S.U.D = 1,448+(-0,062*17,9) 0.3 17.9 0.12 0.04 Energy 0.20 0.01 0.003
12 E-2 Energy use policy U.S.U.D = 0,07-(0,052*4)+(0,089*4^2)-(0,008*4^3) 0.8 4 0.52 0.40 0.08 0.027
13 E-3 Carbon footprint calculation U.S.U.D =3,829+(-0,011*335) 0.1 335 0.29 0.04 0.01 0.003
14 E-4 Total percentage of energy-saving technologies in the university U.S.U.D =0,093+(0,007*4)+(1,855*10^(-5)*4^2) 0.12 4.0 0.07 0.01 0.00 0.001
15 GA-1 Raw water disposal U.S.U.D = -0,25+(0,31*4) 0.1 1 0.22 0.01 Water management 0.20 0.00 0.001
16 GA-2 Water conservation program U.S.U.D = 0,07-(0,052*4)+(0,089*4^2 )-(0,008*4^3) 0.8 4 0.59 0.46 0.09 0.031
17 GA-3 Total percentage of water-saving technologies in the University U.S.U.D = 0,093+(0,007*60)+(1,855*10^(-5)*60^2) 0.6 60.0 0.09 0.05 0.01 0.004
18 GA-4 Per capita water consumption U.S.U.D = =2,516-(14,687*0,28)+(33,761*0,28^2 )-(29,947*0,28^3) 0.4 0.28 0.09 0.04 0.01 0.002
19 GR-1 Recycling program at the university U.S.U.D =0,07-(0,052*4)+(0,089*4^2 )-(0,008*4^3) 0.8 4 0.63 0.49 Waste management 0.52 0.25 0.086
20 GR-2 Hazardous waste management U.S.U.D = 0,07-(0,052*3)+(0,089*3^2 )-(0,008*3^3) 1.0 3 0.13 0.13 0.07 0.023
21 GR-3 Organic waste treatment U.S.U.D = -0,25+(0,31*1) 0.1 1 0.13 0.01 0.00 0.001
22 GR-4 Inorganic waste treatment U.S.U.D = -0,25+(0,31*2) 0.4 2 0.13 0.05 0.03 0.009

Source: Author’s own elaboration.

5. Analysis Results

According to the obtained results, the following may be concluded:

the total sustainability of the university under study is 0.503 USUD or 50.3%.

the PO-2 indicator must be considerably improved because it only achieves 10% of its total value.

as depicted in Figure 6, the university under study does well with respect to its economic dimension.

Figure 6 USUD value per dimension 

the environmental dimension is regular even though several initiatives have been developed as mentioned in the introduction section.

considerable investment is required for developing social dimension programs because social dimension only satisfies 22% or 0.22 USUD of its total value.

6. Conclusions

Expert validation reduces the subjectivity in these tools, which are intended to be universally applicable.

The usage of multicriteria decision in the formulation of indicators allows the consideration of the multiple relations observed in the pillars of sustainable development, facilitating the decision-making of senior management.

The units that comprise the indicator system (index) must be transformed into a heterogeneous unit of measurement, which may be achieved through the usage of value functions.

The proposed indicator system (index) allows comparisons between different pillars of sustainable development and an assessment of the best practices used to achieve acceptable indicator values at low cost and considerable speed at the university under study.

The proposed model also organizes the values of each indicator to allow managers to make strategic decisions with respect to resource investment.

The proposed index can be applied to any university. The only requirement is that sufficient information should be available for obtaining results from the 22 indicators provided herein.

7. References

Agustini Paredes, L. R. (2018). Optimización de la selección de proveedores mediante el proceso jerárquico analítico en una universidad estatal. (Tesis de Maestría). Universidad Nacional Mayor de San Marcos, Lima, Perú. [ Links ]

Alarcón, D. (2005). Modelo integrado de valores para estructuras sostenibles. (Tesis Doctoral). Universitat Politècnica de Catalunya, Barcelona, España. [ Links ]

Berumen, S., y Llamazares Redondo, F. (2007). La utilidad de los métodos de decisión multicriterio (como el AHP) en un entorno de competitividad creciente. Cuadernos de Administración, 20 (34), 65-87. [ Links ]

Cloquell Ballester, V. A., Cloquel Ballester, V, Monterde-Diaz, R., & Santamarina-Siurana, M. C. (2006). Indicators validation for the improvement of environmental and social impact quantitative assessment. Environmental Impact Assessment Review, 26(1), 79-105. [ Links ]

Cruz, M., y Martínez, M. (2012). Perfeccionamiento de un instrumento para la selección de expertos en las investigaciones educativas. REDIE, 14(2), 167-179. [ Links ]

Cortés, E. (2016). Sistema de indicadores como herramienta administrativa para la evaluación del desarrollo sostenible de las universidades de Bogotá; caso Universidad El Bosque. (Tesis de Maestría). Bogotá, Colombia: Universidad El Bosque. [ Links ]

Cyert, R. M., y March, J. (1965) Teoría de las decisiones económicas de la empresa. Ciudad de México, México: Ed. Herrero. [ Links ]

Díaz, P. A. B., Ballester, V. A. C., Alcaraz, J. L. G., y Iniesta, A. A. (2012). El Proceso Jerárquico Analítico y Lógica Difusa: Sus Aplicaciones. Academia Journals, 4(3), 249-254. [ Links ]

Figge, F., Hahn, T., Schaltegger , S., & Wagner, M. (2002). The Sustainability Balanced Scorecard -Theory and Application of a Tool for Value-Based Sustainability Management. Paper presented at the Greening of Industry Network Conference, Center for Sustainability Management. Gothenburg, Sweden, [ Links ]

Grant, J. S., & Davis, L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269-274 [ Links ]

Germain, M. L. (2006). Stages of Psychometric Measure Development: The Example of the Generalized Expertise Measure (GEM). Retrieved from ]

Gutiérrez, F. (2011). Diseño y validación de un sistema de indicadores de sostenibilidad para la evaluación de áreas naturales con uso turístico y su aplicación al territorio Colombiano. (Tesis doctoral). Universitat Politècnica de Valencia, Valencia, España. [ Links ]

Gutiérrez-Fernández, F., Cloquell Ballester, V. A y Cloquel Ballester, V. (2012). Propuesta De Un Sistema De Indicadores De Sostenibilidad Para Áreas Naturales Con Uso Turístico. Anuario Turismo y Sociedad, 13, 55 -83. [ Links ]

Gutiérrez, L. F. (2013). Funciones de valor para construir un índice de sostenibilidad para la evaluación de áreas naturales con uso turístico. Journal of technology, 12(1), 110-117. [ Links ]

Kaplan, R. S., Norton, D. P., & Santapau, A. (1997). El cuadro de mando integral. Barcelona, España: Gestión 2000. [ Links ]

Kaplan, R. S., & Norton, D. P. (1998). Putting the balanced scorecard to work. The economic impact of knowledge, 71(5), 315-24. [ Links ]

Landeta, J. (1999). El Método Delphi: Una Técnica de Previsión para la Incertidumbre. Barcelona, España: Ariel. [ Links ]

Li, W. (2004). Envorionmental management indicators for ecoturism in China´s nature reserves: A case study in Tianmushan Nature Reserve. Tourism Management, 25, 559-564. [ Links ]

Mardle, S., Pascoe, S., & Herrero, I. (2004). Management Objective Importance in Fisheries: An Evaluation Using the Analytic Hierarchy Process (AHP). Environmental Management. Volume 33, (1), 1-11. [ Links ]

Martínez, E. (2007) Aplicación del proceso jerárquico de análisis en la selección de la localización de una PYME. Anuaria Jurídico y Económico Escurialense, 40, 523 -542. [ Links ]

Martínez, R. E., Gómez, J. C. O., Ibarra, D. E., y Moncada, C. A. L. (2018). Selección de una infraestructura de medición inteligente de energía usando una técnica de decisión multicriterio. Scientia et Technica, 23(2), 136-142. [ Links ]

Melle, M. (2007). La responsabilidad social dentro del sector público. Ekonomiaz, 65, 84-107. [ Links ]

Minguet, P. A., Ull, M. A., Piñero, A., y Martínez-Agut, M. P. (2013). La sostenibilidad en la formación universitaria: Desafíos y oportunidades. Educación xx1, 17(1), 133. [ Links ]

Pérez, E., Linares, M., Marques, D., Vento, R., y Díaz Pérez, N., (2018). Evaluación de indicadores de sostenibilidad en la comunidad “Los Jazmines”, Viñales, Pinar del Río, Cuba. Revista Brasileira de Planejamento e Desenvolvimento, 7(5), 732-754. [ Links ]

Saaty, T.L. (1980). The analytic Hierarchy Process. New York, USA: McGraw- Hill. [ Links ]

Salas, R., Rodríguez, F., Sagué, J., y Mena, A. (2010). Factores de Riesgo en el Cáncer de la Próstata. Criterios de expertos. (Método Delphi.). Ciencias Holguín, 15(4). [ Links ]

Schuschny, A., y Soto, H. (2009). Diseño de indicadores compuestos de desarrollo sostenible. Santiago de Chile, Chile: Naciones Unidas. [ Links ]

Wolfslehner, B. (2005). Application of the analytic network process in multi-criteria analysis of sustainable forest management. Forest ecology and management 207 (1-2), 157-170. [ Links ]

Annex 1. System Validation Questionnaire

Please specify your degree of agreement with the following statements.
Use the following rating scale:
(1) Strongly disagree (2) Somewhat disagree (3) Indifferent (4) Somewhat agree (5) Strongly agree
Definition Value
1. The proposed indicator system is consistent with the conceptual evaluation (measuring the sustainable development at universities).  
If you (1) strongly disagree or (2) somewhat disagree, briefly describe the reason for your response.
2. The proposed indicator system can be practically applied for the pursued objective.  
If you (1) strongly disagree or (2) somewhat disagree, briefly describe the reason for your response.
3. There is no duplication or overlap between the indicators proposed by the system.  
If you (1) strongly disagree or (2) somewhat disagree, briefly describe the reason for your response.
4. All the proposed indicators are relevant.  
If you (1) strongly disagree or (2) somewhat disagree, briefly describe the reason for your response.
5. The proposed indicators are sufficient to assess sustainable development at universities.  
If you (1) strongly disagree or (2) somewhat disagree, A) briefly describe which indicators are missing and B) briefly describe which indicators may be overlapping.

Received: August 29, 2018; Revised: November 26, 2018; Accepted: March 28, 2019

Corresponding autor: Fernando Gutiérrez-Fernández. e-mail:

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License