DOI: http://dx.doi.org/10.18684/BSAA(15)121-133

¹ Universidad del Cauca, Facultad de Ciencias Contables, Económicas y Administrativas, Grupo de Investigación en Logística y Producción Universidad del Valle. Magíster en Ingeniería Industrial. Popayán, Cauca.

² Universidad del Valle, Escuela de Ingeniería Industrial, Grupo de Investigación en Logística y Producción Universidad del Valle. Ph.D. Ingeniería Industrial. Cali, Colombia.

³ Universidad del Cauca, Facultad de Ingeniería Electrónica y Telecomunicaciones, Grupo de Investigación en Automática Industrial Unicauca, Ph.D Doctor en Robótica. Popayán, Colombia.

⁴ Universidad del Cauca. Facultad de Ingeniería Electrónica y Telecomunicaciones, Grupo de Investigación en Ingeniería Telemática (GIT). Master en Sistemas y Redes de Comunicaciones. Popayán, Colombia.

Correspondencia:carolinadelgado@gmail.com

Recibido para evaluación: 10 de Febrero de 2016. Aprobado para publicación: 14de Mayo de 2017.

ABSTRACT

Research capacities are developed scientific skills that enable universities to accomplish the dissemination of high-quality scientific knowledge. Nowadays, the modeling of their dynamics is one of the most important concerns for the stakeholders related to the scientific activity, including university managers, private sector and government. In this context, the present article aims to approach the issue of modeling the capacities of the Universities’ research systems, presenting Systems Dynamics as an effective methodological tool for the treatment of data contained in intellectual capital indicators, allowing to estimate parameters, conditions and scenarios. The main contribution lays on the modeling and simulations accomplished for several scenarios, which display the critical variables and the more sensitive ones when building or strengthening research capacities. The establishment of parameters through regression techniques allowed to more accurately model the dynamics of the variables. This is an interesting contribution in terms of the accuracy of the simulations that later might be used to propose and carry out changes related to the management of the universities research. Future research with alternative modeling for social systems will allow to broaden the scope of the study.

KEYWORDS: Systems dynamics, Estimation of parameters, Simulation, Research systems, Intellectual Capital.

RESUMEN

Las capacidades de investigación son habilidades que empoderan a las universidades para de diseminar conocimiento científico de alta calidad. Actualmente, el modelado de sus dinámicas es una de las principales preocupaciones de los grupos de interés relacionados a la actividad científica, incluida la administración de las universidades, el sector privado y el gobierno. En este contexto, el presente artículo busca abordar el problema de modelar capacidades de sistemas de investigación, presentando la Dinámica de Sistemas como una herramienta metodológica efectiva para el tratamiento de datos contenidos en indicadores de capital intelectual y permitiendo la estimación de parámetros, condiciones y escenarios. La principal contribución se centra en el modelado y las simulaciones logradas para varios escenarios, las cuales despliegan las variables críticas y más sensibles para la construcción y/ o fortalecimiento de capacidades de investigación. El establecimiento de parámetros con técnicas de regresión, permiten modelar de manera más precisa la dinámica de las variables. Este es un aporte interesante en términos de la confiabilidad de simulaciones que más adelante pueden emplearse para realizar cambios administrativos para la gestión de la investigación universitaria. Trabajos futuros con técnicas alternativas de modelado para sistemas sociales permitirán ampliar el alcance de este tipo de estudios.

PALABRAS CLAVE: Dinámica de Sistemas, Simulación, Estimación de parámetros, Sistemas de Investigación Universitarios, Capital Intelectual.

RESUMO

Capacidades de investigaç ão s ão habilidades que capacitam as universidades para disseminar o conhecimento científico de alta qualidade. Atualmente, sua dinâmica de modelagem é uma das principais preocupações dos grupos de interesse relacionados com a actividade científica, incluindo a administraç ão das universidades, o setor privado e o governo. Neste contexto, este artigo pretende abordar o problema da modelagem de recursos de sistemas de pesquisa, apresentando a Dinâmica de Sistemas como ferramenta metodológica eficaz para o tratamento de dados através de indicadores de capital intelectual e permitindo a estimativa dos parâmetros, condições e cenários. A principal contribuiç ão incide sobre a modelagem e simulações realizadas para vários cenários, que implantou as variáveis críticas e mais sensíveis para a construç ão e/ou reforço das capacidades de investigação. O estabelecimento de parâmetros com técnicas de regress ão, permitem modelar com mais precis ão a dinâmica das variáveis. Esta é uma interessante contribuiç ão em termos de fiabilidade das simulações que posteriormente pode ser usado para fazer alterações administrativas para a gest ão da pesquisa universitária. Trabalho futuro com técnicas alternativas de modelagem para sistemas sociais permitir ão alargar o âmbito de tais estudos.

PALAVRAS-CHAVE: Sistemas dinâmicos, simulação, estimação de parâmetros, sistemas de pesquisa universitária, Capital intelectual.

INTRODUCTION

Research capacities, belong to a large field of study, called Scientometrics, which is the science that analyses and measures the scientific activity (Bellis, 2009). For its purposes, it uses bibliometric methods for the measurement and evaluation of variables associated with research and development such as 'The Impact Factor –JIF': (Garfield, 2005), the SCI mago Journal Rank –SJR: (SCImago Journal and Country Rank), the H Index (Hirsch, 2005), the Source Normalized Impact per Paper SNIP (CWTS, 2016), the Altmetrics (Altmetric) and the G Index (Egghe, 2006) among others.

In Scientometrics, the Research capacities are understood as the organizational and technical skills necessary to address and undertake and disseminate high-quality scientific research effectively and efficiently [8,9]. The concept belongs to the theory of resources and capabilities (Leiblein, 2012). The resource can be exchanged and it is not specific to the organization, while the capacity is owned, developed, and specific, and acts as a catalyst for resources. Under the premise that capacities impact productivity (Osa Igbaekemen, 2014), it is necessary to address them from a systemic perspective (Capacity Development in Higher Education Institutions in developing countries, 2013).

These effects have been widely studied [13,14,15]. However, there is not a clear consensus about the exact relation between capacities and productivity and most of the research assessment techniques use expenditures and citation indicators ( Moed & Gali , 2014).

Despite the wide range of methods (Crespi, Maffiolli, Pierre and Vázquez, 2011) to perform social systems analysis, such us the agent simulation, system dynamics, neural networks, time series and forecasting, data envelopment analysis and bayesian networks [18,19] among others, in Colombia, the majority of the cited studies, have addressed the issue of capacities from the intellectual capital perspective, due to its strength and relevance compared to similar mechanisms used to treat issues related to science and scientific production. This strength has been demonstrated in the writings of several authors citing the cases and models of Skandia Celemi, Intellect and Nonaka and Takeuchi (Ramírez, 2007). This methodological approaches used in Colombia so far, are mainly about System Dynamics (Herrera, Molano and Sandoval, 2014).

Accordingly, this paper is developed in this methodological framework, relying on the assumptions already mentioned by authors like Leiblein [10] and in particular by the World Federation of engineering organizations [18], which express that in order to achieve relevance, and meet the demands of society as well as maintain a sustainable competitive advantage, organizations as universities require resources, capital and skills. The presented model shows the dynamics of the production of papers involving some of the capacities variables designed within the context of the intellectual capital.

This paper aims to answer the research question of: ¿How to model the dynamics between the investment of public resources and the building and strengthening of research capacities displayed in scientific production? The main contribution and relevance of this kind of research also relays on the fact that policymakers and stakeholders are demanding the construction of metrics that address research units as social and interacting systems (Göran Et al, 2015).

The methodology begins with a systematic mapping of the literature. Subsequently, it references two experiences of measurement of research capacities in Colombia. The University of Cauca was used as a statistical unit to conduct the modeling process. According to the System Dynamics methodology, the authors built a simulation model with statistical data and computational techniques for the determination of parameters (Izquierdo, Galán, Santos and Del Olmo, 2008 ). Later, the paper displays the results of the simulations; considering validation, it discusses about the dynamics of a system of research and the most relevant variables for the creation and strengthening of research capacities.

METHODS

The research is quantitative and descriptive. The design included: Finding the gaps in the available literature, defining a problem, establishing causal links to build a dynamic model, and proposing some possible scenarios and simulations.

The subject is a public university's research system. The data were historic values related to the scientific production and investments, collected in the reports made by the Presidency of the pilot university. The data analysis was made though statistical procedures, not requiring transformation.

Review of Experiences Related to Research Capabilities Indicators

There have been significant contributions to approach this kind of study. It is worth mentioning the main Guide for the measurement of quality, trends and research capacities (Council of Canadian Academies, 2012), The guide of evaluation of Social impact projects of R&D of the Valladolid University (Mendizabal, Gómez and Moñux, 2003), and EUROSTAT: Statistics of R&D (EUROSTAT, 2016).

The Systematic Mapping (Petersen, Vakkalanka and Kuzniazz, 2015) of the literature also provided a reference of the two main experiences approached in Colombia. The main reason to review them is that these universities used research capacity indicators and data that are very useful to the modeling and simulation process: It began in year 2009, at the Research unit of the National University of Colombia, the main reference used was the Intellectual Capital Model [28,29]. Subsequently, in year 2009 this University replicated the model in the University of Cauca. This was accomplished in three (3) projects of collaboration between the two public universities (Plazas Tenorio, 2010).

In addition to these models, there are significant contributions to the modeling and simulation of scientific capacities in Colombia, like the one provided by Ruiz, et al (Ruiz, Bonilla, Chavarro, Orozco, Zarama and Polanco, 2010), which proposes a framework that recognizes the underlying relationships between scientific production variables and the efficiency by discipline, through the use of Bayesian Network (BN) analysis.

It is also of remarkable importance the contributions of Cortés (Cortés Sánchez, 2016) and the ones provided by Ahrweiler, Pyka and Gilbert (Ahrweiler, Pyka and Nigel , 2004) about the Efficiency in the use of digital data bases for the scientific production in universities in Colombia and Simulating knowledge dynamics in innovation networks (SKIN), respectively.

Description of a University Research System

As it has been said, the assessment method proposed by the National University of Colombia, replicated to other universities of the country, identifies research capacities using variables created within the framework of Intellectual Capital; one hand the human capital, relating to knowledge, skills, values and attitudes of people, the structural capital, related to formal and informal organizational structure, methods and procedures of work, specialized software, products of the R&D, management, and culture systems and relational capital variables, or those that have to do with the set of relations that the institution has with its environment see Figure 1

Figure 1

To establish an institutional capacity profile, the figures 2 to 5 show the performance in the time range of some relevant variables:

Figure 2

Figure 3

Figure 4

Figure 5

Description of the problem issued

Many countries have not been aware of the complexity required and the large amount of resources needed to manage a university research system. Nevertheless, understanding its dynamics has become a priority for stakeholders (WORLD, 2013).

Regarding the scientific outputs, according to the SCImago ranking indicators (Scimago, 2015), Colombia, in certain time series, manages to keep in a good position among the Latin American countries, with the largest number of scientific publications after Brazil, Mexico, Chile and Argentina. Despite the consensus that in order to provide sustainable scientific outputs that keep or improve these rankings it is required to strengthen the role of universities (OCED, 2014), there are no national studies about the need of consolidation of research groups (UDEA, 2012).

Since this impacts the research capacities, it is reviewed that Colombia is investing only 0,16 percent of its budget in research. This is consistent with the decreasing budget of COLCIENCIAS, which spent 420.000 million pesos in 2012 and ended with 337.000 million pesos for 2015 and probably will spend 270.000 million pesos for year 2016 (Universidad Nacional de Colombia). This picture gives an account of how difficult it is to maintain the scientific production to competitive levels.

Problem Mapping

The problem map, based on the intellectual capital variables considered in Table 1 is illustrated in Figure 6

Figure 6

Figure 7 identifies the causal relationships among the intellectual capital variables considered; it is possible to observe a reinforcement loop for the investment in projects, with a limit in the number of research groups due to administrative decisions.

Figure 7

The research capacity composed by the number of projects per year and the annual investment also contains a reinforcement loop corresponding to the amount of the investment. It is to be expected to find cycles of compensation. However, given the relative maturity of the unit of analysis and the limited period of time, it was not possible to observe this phenomenon yet. Future research can show the occurrence of them.

Forrester Diagram

The Forrester diagram made is shown in figure 8.

Formal Definition of the Model

Stock Variables. They include the Total or cumulative number of research articles, the Accumulated investment or the aggregated value of annual investment in research projects and the stock of Total of projects.

Flow Variables. Expression 1 (Eq. 1) shows the Annual production of articles. It adds the amount of articles published in the three preceding years.

Eq. 1

Expression 2 (Eq. 2) shows the annual investment in research projects.

Eq. 2

Expression 3 (Eq. 3) shows the projects per year:

Eq. 3

Auxiliary Variables

Average annual wage of a Phd Professor. Expression 4 (Eq. 4) shows the Downloading cost per project by year:

Eq. 4

Maximum of research projects per Phd professor, corresponding to the value resulting of the internal policy of allocation of work to a teacher with a Phd degree, where there is a maximum number of research projects in which the teacher can participate Number of Phd Professors working in research projects in one year.

Maximum cost of annual dedication to the research, shown in Expression 5 (Eq. 5), according to the number of Phd professors, their rate of dedication to research and their average salary.

Eq. 5

Expression 6 (Eq. 6) Shows the cost of the annual dedication to research, which corresponds to an assumed value, based on the number of projects per year and the downloading cost.

Eq. 6

Eq. 7

Average amount of an external project, which corresponds to the budget allocation to an external project.

Articles produced last year, two, three, four and five years before: These variables scientific articles during the last 5 years, taking into account that this is the time frame considered for the classification of the research groups conducted by Colciencias.

Ranked research groups: estimated value of the number of groups ranked by Colciencias.

Internal Budget: Represents the value of the internal investment by project.

Research groups raked currently: Number of groups classified in the present year.

Production: Production of articles during the past five years.

Departments: Current number of academic units that manage a certain discipline.

Number of groups by Department.

Estimation of parameters

The parameters are: the rate of dedication of a teacher to a project per year and the denoted a1, a2 and b1. Respectively corresponding to the Relationship between the number of products in a year and the immediately preceding year, the Relationship between the number of products in a given year and two years before and the Relationship between the products in a given year and the investment in the previous year.

Parameters a1, a2 and b1 were estimated based on two series of time. The first, denoted x(k), represents a total investment in year k. The second, denoted y(k), represents the total number of scientific articles generated by the research system in year k. For the study it was considered an auto regressive model with external input presented in equation 1 (Eq. 1).

In this paper the variable will represent the number of predicted research products. In order to determine the parameters of the model, the minimum square adjustment method was used. This is a method designed to find the parameters that minimize the squares of the differences between the number products of the time series y(k) and the number of predicted products y ^(k) by the model. This principle can be mathematically expressed by the following equation (Eq. 2):

When deriving index J with respect to the parameters (a1, a2 y b1) and equal to zero, there is obtained a linear system with 3 equations and 3 unknown variables from which the parameters can be determined. The values for the parameters a1, a2 and b1 are 0.9335, -0.1712 and 0.009101.

RESULTADOS

The most relevant results by running the model are shown in (Table 1). The behavior of the variables is very similar to reality, which allows validating the proposed model.

Table 1

Scenarios for the building or strengthening of research capabilities. In order to observe sensitivity on the main variables, four scenarios were proposed:

Scenario 1: 'Base Line' Current status. Internal resources: $300,000,000. Rate of dedication to research per project: 0.25%. Research Projects per Phd professor: 1.

Scenario 2: 'Commitment to dedication of Phd professors to research' Rate of dedication of a Phd professor to research increases to 50%. Internal resources and research projects held by a Phd professor remain constant.

Scenario 3: 'Commitment to projects'. Projects per Phd professor increased to 2. Internal resources and the rate of dedications per project by year remain constant.

Scenario 4: 'Increase in investments for research'. Internal investment increased in 50%. The dedication rate per project by year and research per Phd professor remain constant.

Scenarios Evaluation. In order to evaluate the proposed changes, and observe the sensitivity of variables when attempting to improve a capacity, in table 2 the simulations of the four scenarios are presented.

Table 2

Figure 9 illustrates the behavior of the variable annual production of articles and Figure 10 corresponds to the ranked research groups simulations. For the annual production of articles, this research capacity indicator shows an improvement over the base line for the 3 remaining scenarios.>

Figure 9

Figure 10

The option of greater effectiveness for the strengthening of this capacity is to increase dedication of the Phd professors to research.

Figure 10 shows that the number of ranked groups improves at the second year, being also more effective to increase the dedication of Phd professors to research. Figure 11 shows the behavior of the variable projects per year in four stages. It is evident a growth of this variable mainly after the second year. The best scenario for strengthening the capacity is scenario two.

Figure 11

The most representative performance variable is the annual production of articles; with a sustained growth since the fifth year, from which it begins to stabilize.

After analyzing the simulations and evaluate policies considered in each scenario, it is relevant to support the alternative of stage two, i.e., that situation where the University decides to increase the rate of dedication of In addition to these models, there are significant contributions to the modeling and simulation of scientific capacities in Colombia, like the one provided by Ruiz, et al (Ruiz, Bonilla, Chavarro, Orozco, Zarama, & Polanco, 2010), which proposes a framework that recognizes the underlying relationships between scientific production variables and the efficiency by discipline, through the use of Bayesian Network (BN) analysis.

It is also of remarkable importance the contributions of Cortés (Cortés Sánchez, 2016) and the ones provided by Ahrweiler, Pyka and Gilbert (Ahrweiler, Pyka, & Nigel , 2004) about the Efficiency in the use of digital data bases for the scientific production in universities in Colombia and Simulating knowledge dynamics in innovation networks (SKIN), respectively. The Phd professors to research by 25%. In this scenario, the behavior of the production of scientific articles is encouraging, and the dynamics of growth of the ranked groups and projects is representative.

DISCUSSION

With the use of system dynamics it was achieved a conceptual integration which led to mathematical language expressions. In the simulations, the relations quantitatively explained certain dynamics that determine the behavior and the operation of a research system, showing some of the mechanisms that stimulate or inhibit capacity building.

Accordingly, in order to set a limit of scope, the modeling aimed to look for a first approximation, simple and manageable. In these terms, the main achievement has been the facilitation of the understanding of some mechanisms of operation and the presentation of predictions for variables of interest. However, this should be considered only as a first step in this task, being completely conscious that in order to fit the model to reality, it should deepen in the study of the variables in subsystems, which might lead to the development of better fitted equations.

It is important to clarify that the modeling achieved so far takes into a count the number of ranked groups, but not the ranking of the groups. If it was desired to model the dynamics taking into a count the groups ranking, it would represent additional challenges that rest on the mechanism used in Colombia for the measurement and evaluation of the research results (COLCIENCIAS, 2015); the rankings obtained by the groups depend on performance variables according to the national quartile to which the score corresponds. This fact would make it particularly difficult to associate the behavior of indicators with the dynamics of the system, and in fact, it is not possible to directly correlate the results of these variables to changes associated with institutional dynamics. Consequently, the dynamics of the ranking of the groups must be considered as an exogenous variable.

It is then discussed by the authors, whether future research will require alternate mechanisms such as those developed by the Agent Simulation Models, which has been used as a better alternative for adaptive systems (Ling Loo, Y.C. Tang and Ahmad, 2015). For now, it is clear that the present research is the first approach to the pilot University research system modeling, and that future efforts are required to determine the effects of changes in the policies considered, including scenarios about the dynamics of other research products.

CONCLUSIONS

The present research modeled a system of research at a public University, simulating it for 5 years, to observe the creation or strengthening of research capacities in relation to investments in internal resources, the number of projects held by Phd professors and the rate of dedication to the projects per year; with the scientific production represented in articles, as well as the ranked groups and the number of Projects.

The simulations for four (4) scenarios showed that there are critical variables that affect notoriously the development or inhibition of research capacities. For example, in the scientific production, there is a direct effect caused by the investment on internal and external resources, as well as the change of the policy for time allocation of Phd professors. Some variables act as catalysts or inhibitors of the development of capacities. For example, the simulation showed that in the production of research articles, the best decision in terms of strengthening is increasing the dedication of Phd professors to research (See figure 9: Simulations of the variable annual production of articles). The simulations showed that when increasing the rate of dedication of a Phd professor to research in to a 50%, maintaining the Internal resources and research projects held by a Phd professor constant, the results of the production of papers improve considerably.

Similarly, in the case of the development of capacities in ranked research groups and projects, capacity can be strengthened by changing policies to increase (50%) the dedication of Phd professors to research.

The establishment of parameters through regression techniques allowed modeling the dynamics of the variables, based on historical data. This is an interesting contribution in terms of the accuracy of the simulations that later have to be used to propose and public policy changes.

ACKNOWLEDGMENTS

The authors wish to express their gratitude to: University of Cauca, University of Valle, and the INNOVACCIÓN Project.

REFERENCES

[1] BELLIS, N.D. Bibliometrics and Citation Analysis From the Science Citation Index to Cybermetrics. 1 ed. Lanham, Maryland (Canada): The Scarecrow Press, Inc, 2009, p. 451.         [ Links ]

[2] GARFIELD, E. The Agony and the Ecstasy—The History and Meaning of the Journal Impact Factor. Philadelphia (USA): Thompson Reuteurs, 2005, p. 1-22.         [ Links ]

[3] SCIMAGO JOURNAL AND COUNTRY RANK. SCImago Journal and Country Rank, SCImago [online]. 2016. Available: http://www.scimagojr.com/ [Último acceso: 13 Abril 2016]         [ Links ].

[4] HIRSCH, J. An index to quantify an individual’s scientific research output. H Index. PNAS, 102(46), 2005, p. 1-14.         [ Links ]

[5] CERTIFIED WIRELESS TECHNOLOGY SPECIALISTS (CWTS). Journal Indicators, 25 01 2016 [online]. 2016. Available: http://www.journalindicators.com/ [Último acceso: 25 01 2016]         [ Links ].

[6] ALTMETRIC. Altametric [online]. 2014. Available: https://www.altmetric.com/ [Último acceso: 13 Abril 2016]         [ Links ].

[7] EGGHE, L. Theory and practice of the G index. Scientometrics, 69(1), 2006.         [ Links ]

[8] DEPARTMENT FOR NTERNATIONAL DEVELOPMENT (DFID). Working Paper Series: Capacity Building in Research [online]. 2010. Avalilable at:https://assets.publishing.service.gov.uk/media/57a08b8ee5274a31e0000c0a/ResearchStrategyWorkingPaperfinal_capacity_P1.pdf [cited at the 01 of February of 2016]         [ Links ].

[9] ESSENCE. Seven Principles for Strengthening Research Capacity In Low- And Middle-Income Countries. 1St ed. London (UK): ESSENSE, 2014, 36 p.         [ Links ]

[10] LEIBLEIN, M. What do Resource and Capability Based Theories Propose. Journal of Management, 37(4), 2012, p. 909-932.         [ Links ]

[11] OSA, G.I. Capacity Building. A Tool for Increase Productivity In Nigeria Public Sector Organizations. Global Journal of Human Resource Management, 2(3), 2014, p. 45-58.         [ Links ]

[12] VAN, D.R. Capacity Development in Higher Education Institutions in developing countries [online]. 2010. Avalilable at: https://www.msm.nl/resources/uploads/2014/02/MSM-WP2013-30.pdf [cited at the 01 of February of 2016]         [ Links ].

[13] PROTOGEROU, A., CALOGHIROU, Y. and SPYROS, L. Dynamic Capabilities and Their Indirect Impact on Firm Performace [online]. 2012. Avalilable at: http://webdoc.sub.gwdg.de/ebook/serien/lm/DRUIDwp/08-11.pdf.[cited at the 01 of February of 2016]         [ Links ].

[14] DAVIS, M., BEVC, C. and SCHENCK, A. Effects of Performance Improvement Programs on Preparedness Capacities. Public Health Reports, 129(1), 2014, p. 19-27.         [ Links ]

[15] TIPPER, A. and WARMKE, N. Adjusting productivity statistics for variable capacity utilization: Working harder or hardly working?: Statistics, New Zealand Working Papers [online]. 2010.         [ Links ]

[16] MOED, H. and GALI, H. Research assessment: Review of methodologies and approaches. Research Trends, 36(1), 2014, p. 3-6.         [ Links ]

[17] CRESPI, G., MAFFIOLLI, A., PIERRE, M. and VASQUEZ, G. Evaluating the Impact of Science, Technology and Innovation Programs: a Methodological Toolkit. 1st ed. New York (USA): Interamerican Developing Bank, 2011, 92 p.         [ Links ]

[18] RUIZ, C., BONILLA, D., CHAVARRO, L., OROZCO, A., ZARAMA, R. and POLANCO, X. Efficiency measurement of research groups using Data Envelopment Analysis and Bayesian networks. Scientometrics, 83(1), 2010, p. 711-721.         [ Links ]

[19] ADHIKARI, R. and AGRAWAL, R. An Introductory Study on Time Series Modeling and Forecasting [online]. 2017. Avalilable at: https://gejza.nipax.cz/_media/stochasticke_procesy:1302.6613.pdf [cited at the 01 of February of 2016]         [ Links ].

[20] RAMÍREZ, D. Capital intelectual. Algunas Reflexiones sobre su importancia en las Organizaciones. Pensamiento y Gestión, 23(1), 2007, p. 130-152.         [ Links ]

[21] HERRERA, M., MOLANO, J. y SANDOVAL, H. Diseño de Estrategias y Políticas de Investigación en la Educación bajo dinámica de Sistemas. Inventium, 17(1), 2014, p. 23-41.         [ Links ]

[22] GORAN, M. et al. Towards a future proof system for higher education and research in Finland, Ministry of Education and Culture. 1st ed. Helsinki (Findland): Department for Higher Education and Science Policy, 2015, 190 p.         [ Links ]

[23] IZQUIERDO, L. et al. Modelado de sistemas complejos mediante simulación basada en agentes y mediante dinámica de sistemas. EMPIRIA, Revista de Metodología de Ciencias Sociales, 16(1), 2008, p. 85-112.         [ Links ]

[24] COUNCIL OF CANADIAN ACADEMIES. Informing Research Choices: Indicators and Judment. The Expert Panel on Science Performance and Research Funding. 1st ed. Ottawa (USA): Council of Canadian Academies, 2012, 162 p.         [ Links ]

[25] MENDIZABAL, G., GÓMEZ, F. y MOÑUX, D. Desarrollo de una guia de evaluación de Impacto Social para Proyectos de I+D+I. Revista Iberoamericana de Ciencia, Tecnología, Sociedad e Innovación, 5(1), 2003, p. 64-72.         [ Links ]

[26] EUROSTAT. Your Key to European Statistics [online]. 2016. Available at: http://ec.europa.eu/eurostat [cited at the 01 of February of 2016]         [ Links ].

[27] PETERSEN, K., VAKKALANKA, S. and KUZNIAZZ, L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64(1), 2015, p. 1-18.         [ Links ]

[28] GONZALES, J. y RODRIGUEZ, M. Modelos de Capital Intelectual y sus indicadores en la universidad pública. Cuadernos de Administración Universidad del Valle, 26(43), 2010, p.100-113.         [ Links ]

[29] COLOMBIA. VICERRECTORÍA DE INVESTIGACIÓN UNIVERSIDAD NACIONAL DE COLOMBIA. Capacidades de investigación en la Universidad Nacional de Colombia 2000-2008 : Una aproximación desde el capital intelectual. 1st ed. Bogotá (Colombia): Universidad Nacional de Colombia, 2009, 368 p.         [ Links ]

[30] PLAZAS, A. Capacidades de investigación de la Universidad del Cauca desde la perspectiva de la gestión del conocimiento. 1st ed. Popayá (Colombia): Boletín VRI Universidad del Cauca, Vicerrectoría de Investigaciones, 2012, 28 p.         [ Links ]

[31] CORTES, J. Eficiencia en el Uso de Bases de Datos Digitales para la Producción Científica en Universidades de Colombia. Revista Española de Documentación Científica, 39(2), 2016, p. 1-15.         [ Links ]

[32] AHRWEILER, P., PYKA, A. and NIGEL, G. Simulating knowledge dynamics in Innovation Networks (SKIN): Institut Fur Volkswirtschaftslehre, 267(1), 2004, p. 1-10.         [ Links ]

[33] U. WORLD. The role of research universities in developing countries [online]. Available at: http://www.universityworldnews.com/article.php?story=20130811091502202. [cited at the 01 of February of 2016]         [ Links ].

[34] Scimago. SIR Iber Colombia 2015 [online]. 2015. Available at: http://www.scimagoir.com/pdf/iber_new/SIR%20Iber%20COL%202015%20HE.pdf [cited at the 01 of February of 2016]         [ Links ].

[35] OCED. OECD Reviews of Innovation Policy: Colombia 2014. 1st ed. Paris (France): OCED Publications, 2014, 244 p.         [ Links ]

[36] UNIVERSIDAD DE ANTIOQUIA. Investigación científica en retazos [online]. 2012. Available at: http://noticias.universia.net.co/ciencia-nn-tt/ noticia/2012/11/08/980707/problemas-atraviesa-investigacion-cientifica-colombia.html[cited at the 01 of February of 2016]         [ Links ].

[37] UNIVERSIDAD NACIONAL DE COLOMBIA. Recorte en Presupuesto de Colciencias Afecta la Generación de Conocimiento [online]. 2016. Available at: http://agenciadenoticias.unal.edu.co/detalle/article/recorte-en-presupuesto-de-colciencias-afecta-la-generacion-de-conocimiento.html [cited at the 01 of February of 2016]         [ Links ].

[38] COLCIENCIAS. Modelo de Medición de Grupos de Investigación, Desarrollo Tecnológico o de Innovación y de Reconocimiento de Investigadores del Sistema Nacional de Ciencia, Tecnología e Innovación. Año 2015. 1st ed. Santafé de Bogotá (Colombia): COLCIENCIAS, 2015, 200 p.         [ Links ]

[39] LING, Y., TANG, A. and AHMAD, A. The Gap of Current Agent Based Simulation Modeling Practices and Feasibility of a Generic Agent Based Simulation Model. International Journal of Advanced Computer Research, 5(19), 2015, p. 115-123.         [ Links ]