METHODOLOGY PROPOSAL TO DETERMINE PROJECT MANAGEMENT MATURITY LEVEL IN ENGINEERING COMPANIES
PROPUESTA METODOLÓGICA PARA MEDIR EL NIVEL DE MADUREZ DE LA GESTIÓN DE PROYECTOS EN EMPRESAS DE INGENIERÍA
PROPOSTA METODOLÓGICA PARA MEDIR A MATURIDADE DA GERÊNCIA DE PROJETOS EM EMPRESAS DE ENGENHARIA
Álvaro Julio Cuadros López1, Jessica Andrea Morales Viveros2, Ángela Beatriz Rojas Meléndez3
1 Ingeniero Industrial. Especialización en Gestión de la Innovación Tecnológica, Maestría en Administración, Doctorado en Ingeniería (en curso), Universidad del Valle. Cali, Valle, Colombia. Calle 53 norte No. 3A-02. Apto 102D. Unidad Ancora Norte, Cali, Valle del Cauca, Colombia. Tel.: 3206313195. Correo electrónico: alvaro.cuadros@correounivalle.edu.co.
2 Ingeniero Industrial. DMV Ingenieria. Palmira, Valle, Colombia.
3 Ingeniero Industrial. Provider Ltda. Palmira, Valle, Colombia.
Artículo recibido: 13-IV-2016 / Aprobado: 11-IV-2017 ]]>
Disponible online: 30 de agosto de 2017ABSTRACT
The purpose of this paper is to present a methodological proposal for small engineering companies to evaluate its project management maturity level. The proposal is based on a generic standard which was adjusted by consulting internal and external experts, to establish particular evaluation criteria and qualification scale to perform best practices evaluation. The proposal considers not only general maturity level qualification but also, by multivariate statistics, qualification by processes groups and experts groups. The proposal was validated through a case study, when applied in an engineering company, resulting on that external experts tend to qualify the maturity of management of projects higher than internal experts, the company's employees.
KEY WORDS: Project management maturity models, P3M3, Multivariate analysis, Engineering companies.
RESUMEN
El objetivo de este artículo es presentar una propuesta metodológica para que pequeñas empresas de ingeniería puedan evaluar su nivel de madurez en la gestión de proyectos. La propuesta está basada en un estándar genérico el cual fue ajustado mediante consulta a expertos internos y externos, para establecer un criterio de evaluación particular y una escala de calificación con el fin de realizar la evaluación de las mejores prácticas. La propuesta considera no sólo la calificación general de madurez sino también, mediante estadística multivariada, la medición de la madurez por grupos de procesos y por grupos de expertos. La propuesta fue validada mediante un caso de estudio, aplicada a una empresa de ingeniería, resultando en que los expertos externos tienden a calificar la madurez de la gestión de proyectos en un nivel mayor que los expertos internos, es decir los propios empleados de la organización.
PALABRAS CLAVE: Modelos de madurez de la gestión de proyectos, P3M3, análisis multivariado, empresas de ingeniería.
RESUMO
O objetivo deste trabalho é apresentar uma proposta para que as pequenas empresas de engenharia possam saber o status de gerenciamento de projetos em sua organização. A proposta está baseada num regular genérico que foi ajustado mediante consulta a experientes internos e externos, para ter um conjunto de critérios de avaliação e uma escala próprios para fazer a qualificação das melhores práticas. A proposta considera não só a classificação geral de maturidade, mas também por estatística multivariada, a medida da maturidade por grupos de processos e grupos de peritos. A proposta foi validada através de um estudo de caso, ao ser aplicada em uma empresa de engenharia, resultando em que os experientes externos tendem a qualificar num nível maior a maturidade que os experientes internos, os próprios funcionários da organização.
]]> PALAVRAS-CHAVE: modelos de maturidade de gerenciamento de projetos, P3M3, análise multivariada, empresas de engenharia.1. INTRODUCTION
Today organizations must manage not only isolated projects, but also internal and external portfolio project where the relationship between the project and the organization takes much importance. Projects are not oriented towards the solution of isolated technical problems anymore; they now extend their reach into the management of the company and change. The projects are understood as a base to carry out the strategic management and competitive advantage; these have become one of the best ways to align the Organization's resources that are always scarce and solving large corporate (Kerzner 2009).
On the same basis, organizations are searching the best practices for management of projects, evolving the discipline to treat not only the management of a project, but to find a way how projects allow to reach the goals of organizations (Ahlemann et al. 2009), (Andersen & Jessen 2003), (Solarte-Pazos & Sánchez-Arias 2014).
The concept of maturity in organizations is understood as the level in which an organization is in a perfect condition to meet their goals and objectives (Andersen & Jessen 2003). According to this definition, it has no sense to speak of a fully mature organization, it makes sense talking about levels of maturity which should be measured or characterized to stay on the path of improvement (Kerzner 2009), (Andersen & Jessen 2003), (Jugdev & Thomas 2002), (Gray & Larson 2009).
The maturity search enables organizations to reduce the inherent variability of processes and improve their average performance (Cooke-Davies & Arzymanow 2003)using pre-determined scales, and qualitative comments on the practices based on the experiences of the interviewee. Differences between companies and industries were found to exist in each domain. The most highly developed project management models (which might be said to equate to measure of project management maturity, (Brookes & Clark 2009). Thus emerge the so-called Project Management Maturity Models (P3M) which allow companies to diagnose the organizational capacity to manage projects, and to establish mechanisms to improve organizational capabilities, rather than the individual skills of project managers (Backlund et al. 2014).
Projects organizational capacity results by benchmarking the current state with the ideal condition to obtain objectives (Andersen & Jessen 2003), in other words maturity models allow companies to identify the level of development in which they are and which are the requirements to ensure projects success.
Maturity models have their origin in the field of total quality management (TQM) (Cooke-Davies et al. 2001), which handle a strategic link with continuous improvement, by analyzing the current situation of the Organization and what it aims to be in the future. In last twenty years, it have released a large number of maturity models, however is possible to identify some that are the more applied, among which are mentioned the Capability Maturity Model (CMM), the Organizational Project Management Maturity Model (OPM3), and the Programme, and Project Management Maturity Model (P3M3). These models have similar structures in terms of the factors that analyze, which usually include their own proposals for management of projects, and in addition a scale of measurement of different levels.
One of the first models of maturity appeared when, in 1991, the Software Engineering Institute (SEI) at Carnegie-Mellon University designed, in 2001, a model of maturity of capabilities, the CMM, to measure the processes in software development organizations (Mutafelija & Stromberg 2003). The model proposes a structure of five levels of maturity for analyzing some areas of key processes, common characteristics and key practices (Von Wangenheim et al. 2010), (Mutafelija & Stromberg 2003).
Another model is the OPM3, proposed by the Project Management Institute (PMI), Professional Association of management of projects that started at the end of the 1960's in the United States that currently brings together professionals from around the world (Guido & Clements 2007), (Project Management Institute-PMI 2013). This model is structured around a structure of four levels of maturity, good practices, capabilities to carry them out, the observable results, and the stages of the process of improvement(Project Management Institute-PMI 2013), (Project Management Institute-PMI 2008). The P3M3 model, proposed by the Office of Government Commerce (OGC) in the United Kingdom, was based in its origins in the CMMI, but has been particularly modified. This model is structured around five levels of maturity, a few areas of development and a group of processes (Office of Government Commerce 2006), (Office of Government Commerce 2013b), (Office of Government Commerce 2013a).
]]> This paper presents a methodology to help small engineering companies to identify the level of maturity of project management. Proposal has considered the design of the hierarchical structure of criteria to be measured, the design of the scale of maturity, and the strategy to have reliable results.2. METHODS
2.1 P3M selection
The first step of the Project was selecting a project management maturity model consistent with the kind of organization evaluated. In this case, the literature research did not highlighted a maturity model for engineering companies, but it permited to find three of the most published: Capability Maturity Model Integration (CMMI), Portfolio, Programme and Project Management Maturity Model (P3M3), and the Organization Project Management Maturity (OPM3).
The review of those models in terms of factors of analysis, evaluation scale and the volume of applications in the literature, allowed to discard the CMMI since it is more oriented to software development projects while the others are generic and may apply to the organization (Selleri Silva et al. 2015)synthesize, and present results on the use of the Capability Maturity Model Integration (CMMI, (Von Wangenheim et al. 2010), (Jiang et al. 2004).
By the other hand, PricewaterhouseCoopers and KPMG have researched for the most used project management standards. PricewaterhouseCoopers found Project Management Base of Knowledge (PMBOK), the complement of the OPM3 as the leader, been used by 27% of organizations (PricewaterhouseCoopers 2012). In similar researches, KPMG have found in different studies, the PMBOK standard as leader in applications (KPMG International 2015a), (KPMG International 2015b), (KPMG International 2013), (KPMG International 2005).
Another research carried out in Russia over the status of project management in different sectors including construction and engineering found that leading PMBOK standard is applied to 40% of the sample (Polkovnikov & Ilina 2014). In addition, OPM3 has been used as the standard to evaluate project management maturity in different industrial sectors in Portugal (Silva et al. 2014).
As result of this review, it was decided to use the generic standard OPM3 which evaluates factors promoted by PMBOK.
2.2. Experts identification
In order to perform precise analysis and make adjustments to the general structure of the model, a group of experts was identified. This people should have project management experience and should know internal management in the organization.
]]> The group of experts was attended by nine engineers, the Technical Director of the company which controls projects in the company, five project managers, a consultant PMP of a big client company and two project managers in client companies. Project experience of every expert is shown in Table 1.2.3. Domain selection
Being a general standard, next step in the process is making some adjustments to the selected maturity model to the particular case. The OPM3 considers three domains in which evaluates groups of processes for the projects, programs and portfolio management offering flexibility to the application according to specific application case.
Project and program management identifies groups of processes for initiation, planning, execution, monitoring and control and closure. Project management is evaluated asking for the existence of best practices identified as standardize, measure, control and continuously improve (SMCI).
Portfolio management identifies two groups of processes, which are used to facilitate decisionmaking and balancing portfolio: alignment processes determine what components should be categorized, evaluated and selected in portfolio management. And the monitoring and control processes review performance indicators to be aligned with the strategic objectives and verify the benefits to the organization.
For the study, it was decided to apply only the dimension of project management because the particular situation of the organization not allowed to apply the other dimensions. For the application case, the organization is a six year old company which develops engineering projects awarded by bids. As the company performs projects for third parties, is not involved in programs or portfolio environment, yet. Those projects, the company executes are not aligned with strategic objectives.
2.4. Best Practices selection
The OPM3 proposes that maturity should be measured in accordance with the existence of a set of best practices. Best practices are related to the optimal methods to achieve strategic objectives, and are shown with capabilities and successful results. Best practices are defined in two categories, on the one hand are the SMCI which refers to the cycle of breeding capabilities need to know standardization, measurement, Control and improvement. On the other hand are the organizational enablers (OE), which are structural, cultural, technological and human resources practices to support the implementation of best practices in projects.
Having decided to evaluate only the dimension of project management, the number of practices to evaluate dropped from 488 to 244. However, it was considered that the list was still long to evaluate, so a delphi exercise was done with experts to reduce it. According to his experience, his knowledge of the company, the type of projects that it develops and procedures that usually the customers demand, experts reduced the list of practices to 32 which cover different process groups from PMBOK.
]]> 2.5. Evaluation scale designLiterature reviewed showed that maturity models are usually focused in asking if best practices are applied or not. However, in order to offer a more detailed view of the maturity level, a scale with three score was designed, see Table 2.
In addition, to determine the maturity level it was settled a few ranges of score, determining three levels of maturity (maximum level, middle level and a level for ignorance). Superior score was built, performing the sum of the total number of best practices by establishing the highest score that could be obtained according to the scale developed and the total number of respondents. Finally, this maximum rating was divided into three classes (maximum, middle and ignorance levels) as seen in Table 3.
By this way, the maximum score to obtain is 1,440 points as the maximum qualification given for every SMCI and the general maturity level is 5760 points which is the sum of the fourth SMCI score. The range for each level was decided by the authors. It was stablished a minimum score of 80% to be qualified a practice in the maximum maturity level. The minimum score for middle maturity level was established in 50%.
2.6. Test design and surveys
Last part of the process was the design of the questions and collecting information. In first place, some meetings were done to inform experts about the research, later, another meeting to explain the instrument and finally another to develop the survey.
3. RESULTS
3.1. Reliability
]]> Prior to establishing the level of maturity for the organization, an analysis of reliability of the scale was done to check the representativeness of the data. This analysis was developed with the index Cronbach's alpha which is based on the internal consistency through the calculus of the correlation between the items of the scale, and it is one of the most used to stablish the reliability of scales (Vinacua & Cañas 2003). This index handles a scale from 0 to 1, being 0 a very low, contaminated error reliability and 1 a very high reliability without any error.However, there is no consensus knowing from which value can be considered acceptable a scale. Literature review identified authors as Nunnally (Nunnally & Bernstein 1994) who proposed as a minimum recommended 0.70, and Malhotra (Malhotra & Peterson 2001) who accepts as valid a value greater than 0.60.
For the research the validation was performed at processes level and at overall level by using SPSS 16. As can be seen in Table 4, the index is high enough to be considered reliable. Even the lowest, improvement practices with 0,842 are higher than those limits proposed by Nunnaly and Malhotra.
3.2. Overall maturity
To determine the general maturity level, qualifications of experts for all practices evaluated were added and compared with the total score possible and accommodated in the intervals set before. As can be seen in Table 5, with a score of 2,430 points from possible 5,760, organization is located on a basic level of ignorance which corresponds with 42% of maturity reached.
3.3. Practices maturity
In a similar way, to determine the practice maturity level, qualifications of experts were added and compared with the total score possible and accommodated in the intervals set before. As can be seen in Table 6, there are three group of practices located at ignorance level while one group is located in mid-level near to inferior limit. With 42%, 40% and 36% of maturity, the processes of measurement, control and improvement are at ignorance level while standardization practices with 51% are at the middle level.
]]> In addition to the maturity level found, a decreasing trend was found in the maturity of practices. When seen the total score possible and comparing it with the current maturity levels, can be seen that standardized practices account higher maturity than the next practices group and so on.3.4. Cluster maturity
In order to know if maturity assessment followed a pattern according to the type of expert surveyed, a cluster analysis was performed. This analysis assumes heterogeneity among elements but try to identify groups in which each items belongs to one and only one group. Groups are heterogeneous and that each group is internally homogeneous.
3.4.1. Standardization
Average score given by experts for this group was 82 points which places it in the category of medium level. However with a standard deviation of 23, cluster analysis was useful to understand behavior of experts. By this way, groups with lower standard deviation could be identified.
To identify clusters was done a principal components analysis in search of identify the dispersion of scores through the created components. As can be seen in Figure 1, two or three groups could be identified. Could be located experts identified as 1 and 3 in one group and the rest of experts in another group. However, the second group could even represents two groups, experts 2, 6 and 7 in one group and 4, 5, 8 and 9 in another group.
In order to decide how many groups declare, it were done dendogram analysis for three and four clusters as seen in Figure 2 and 3. As can be seen, for three clusters, it only isolates expert 2 locating experts 6 and 7 in second group. For the research, it was considered relevant to use four clusters letting element 7 out of the group because it adds dispersion to the data.
By this way, four groups are structured, group 1 with experts 1 and 3 (clients representatives), group 2 with expert 2 (external consultant PMP), group 3 with expert 7 (Technical Director), and group 4 with experts 4, 5, 6, 8 and 9 (project managers). With this clustering analysis data inside each category is consistent. Analyzing maturity qualification by cluster, more detailed scoring could be identified. As seen in Table 7, external experts (clients project managers and external consultant) give highest score followed by Technical Director. In this case, project managers give the lowest maturity score.
]]>
3.4.2. Measurement
In this case, average score given by experts was lower than standardization, 67 points, but also the dispersion was lower, 19, which shows more agreement in the low maturity level qualification. As can be seen in Figure 4, two groups could be identified, one including elements 1, 2 and 3, and other group including the rest of elements.
In order to have a clear idea of how to cluster the elements, dendogram analysis was done. As can be seen in Figure 5 and 6, three and four clusters were identified, but it was decided to use the four clustering structure because it offers lower differences among elements. In this case elements 1 and 3 form a first group, element 2 form a second group, elements 4, 5, 6, 8 and 9 form a third group and element 7 form a last group.
By this way, same clusters to standardized processes were identified, group 1 with experts 1 and 3 (clients representatives), group 2 with expert 2 (external consultant PMP), group 3 with expert 7 (Technical Director), and group 4 with experts 4, 5, 6, 8 and 9 (project managers). With this clustering analysis data inside each category is consistent. Analyzing maturity qualification by cluster, more detailed scoring could be identified.
As seen in Table 8, similar behavior to standardization was found. External experts (clients project managers and external consultant) give highest score followed by Technical Director. In this case too, project managers give the lowest maturity score. However it can be seen that in this case, two groups qualify the maturity in the lowest level of ignorance.
3.4.3. Control and improvement
]]> Following the same procedure, the average score and standard deviation were calculated for every group of processes, and clustering analysis was done to know qualification by group.Qualifications of experts for each process continue the decreasing trend identified previously. The control process was qualified with 64 and a deviation of 13 while improvement was qualified with 58 and a deviation of 9. Not only decreases the level of maturity but increases agreement over qualification.
As can be seen in Table 9, decreasing trend identified previously, continues until all groups qualifies maturity in the lowest level of ignorance. In the same way, group of project managers keep the lowest qualifications among groups.
4. CONCLUSIONS
Project management maturity models consist of a series of levels with which organizations evaluate their performance, they can have four or five levels. However for all models these levels start with activities related with standardization or common language and finish with continuous improvement.
Shortcomings in fulfilment objectives related to scope, time, budget, and documentation of projects, in the case study, could be explained due to the lack of a stage of monitoring and control within the development and maintenance of each of their projects.
It could be understood that general standards such as OPM3 and PMBOK can be applied in a useful way been adjusted according to specific industrial sector. The OPM3 is a complex and robust maturity model and the analyzed organization was a small business that did not meet all the criteria proposed by the model, the reasons why it was necessary to adjust the model to what you actually need the company, thus achieving optimum maturity required by the organization.
Currently evaluating what knows and carries out the company on the basis of the adjusted OPM3 model determined that the company is at a level of general maturity of 42%, which means a level of ignorance of most of the good practices. On the other hand for each of the processes was obtained percentages of 51% in the case of standardization, 42% for measurement, 40% for control and 36% for improvement that mean a mid-level in the case standardization and ignorance for remaining processes.
Best practices processes in project management are ordered and sequential which means that advancing each process and achieve a good performance should follow the order proposed by the model, i.e., first it must standardize, if practices are standardized means that they are known by employees, if they are known measurement parameters can be set, if a measurement process is done, can be controlled, and that process leads to the improvement of the organization.
]]> Principal component analysis allows identifying groups of practices which could show general maturity level however it resulted more useful to identify clusters of experts which evaluate the same way the maturity level. Helped with dendogram analysis, external experts evaluated the maturity in a higher level than internal experts. Future research could try to understand why this kind of behavior can be done.REFERENCES
Ahlemann, F., Teuteberg, F. & Vogelsang, K., 2009. Project management standards - Diffusion and application in Germany and Switzerland. International Journal of Project Management, 27(3), pp. 292-303. Available at: http://dx.doi.org/10.1016/j.ijproman.2008.01.009. [ Links ]
Andersen, E.S. & Jessen, S.A., 2003. Project maturity in organisations. International Journal of Project Management, 21(6), pp. 457-461. [ Links ]
Backlund, F., Chronéer, D. & Sundqvist, E., 2014. Project Management Maturity Models - A Critical Review. Procedia - Social and Behavioral Sciences, 119, pp. 837-846. Available at: http://www.sciencedirect.com/science/article/pii/S1877042814021855. [ Links ]
Brookes, N. & Clark, R., 2009. Using Maturity Models to Improve Project Management Practice. In POMS 20th Annual Conference. p. 11. [ Links ]
]]>Cooke-Davies, T.J. & Arzymanow, A., 2003. The maturity of project management in different industries: An investigation into variations between project management models. International Journal of Project Management, 21(6), pp. 471-478. [ Links ]
Cooke-Davies, T.J., Schlichter, J. & Bredillet, C., 2001. Beyond the PMBOK guide. In Proceedings of the 32nd Annual Project Management Institute 2001 Seminars and Symposium, Nashville, TN. [ Links ]
Gray, C. & Larson, E., 2009. Administración de proyectos 4th ed. McGrawHill, ed. [ Links ]
Guido, J. & Clements, J., 2007. Administración exitosa de proyectos 3rd ed., Thomson Editores. [ Links ]
Jiang, J.J. et al., 2004. An exploration of the relationship between software development process maturity and project performance. Information & Management, 41(3), pp. 279-288. [ Links ]
]]>Jugdev, K. & Thomas, J., 2002. Project Management Maturity Models: The Silver Bullets of Competitive Advantage? Project Management Journal, 33(4), p. 4. [ Links ]
Kerzner, H., 2009. Project management: a systems approach to planning, scheduling, and controlling. [ Links ]
KPMG International, 2015a. Climbing the curve. Global construction survey 2015. [ Links ]
KPMG International, 2005. Global IT Project Management Survey. [ Links ]
KPMG International, 2015b. Project and programme management survey 2015. [ Links ]
]]>KPMG International, 2013. Project Management Survey Report 2013. [ Links ]
Malhotra, N.K. & Peterson, M., 2001. Marketing research in the new millennium: emerging issues and trends. Marketing Intelligence & Planning, 19(4), pp. 216-232. [ Links ]
Mutafelija, B. & Stromberg, H., 2003. Systematic process improvement using ISO 9001:2000 and CMMI. [ Links ]
Nunnally, J.C. & Bernstein, I., 1994. Psychometric Theory. [ Links ]
Office of Government Commerce, 2013a. P3M3 - Project Model. Project Management Maturity Model. The Office of Government and Commerce, pp. 1-23. Available at: http://tinyurl.com/2vwcp7b. [ Links ]
]]>Office of Government Commerce, 2013b. Portfolio, Programme and Project Management Maturity Model (P3M3) Introduction and Guide to P3M3. [ Links ]
Office of Government Commerce, 2006. Portfolio, programme & project management maturity model (P3M3). [ Links ]
Polkovnikov, A. V. & Ilina, O.N., 2014. The Reality of Project Management Practice in Russia: Study Results. Procedia - Social and Behavioral Sciences, 119, pp. 805-810. Available at: http://www.sciencedirect.com/science/article/pii/S1877042814021818 [Accessed October 10, 2014] [ Links ].
Pricewaterhouse Coopers, 2012. Insights and Trends: Current Portfolio, Programme, and Project Management Practices., pp. 1-40. Available at: http://www.pwc.com/en_US/us/public-sector/assets/pwc-global-project-management-report-2012.pdf. [ Links ]
Project Management Institute-PMI, 2013. Organizational Project Management Maturity Model OPM3 Knowledge Foundation. [ Links ]
]]>Project Management Institute-PMI, 2008. Organizational Project Management Maturity Model Opm3. Project management, Standards, Mathematical models. [ Links ]
Selleri Silva, F. et al., 2015. Using CMMI together with agile software development: A systematic review. Information and Software Technology, 58, pp. 20-43. [ Links ]
Silva, D. et al., 2014. OPM3 ® Portugal Project: Analysis of Preliminary Results. Procedia Technology, 16, pp. 1027-1036. [ Links ]
Solarte-Pazos, L. & Sánchez-Arias, L.F., 2014. Gerencia de proyectos y estrategia organizacional: El modelo de madurez en gestión de proyectos CP3M© V5.0. Innovar, 24(52), pp.5-18. [ Links ]
Vinacua, B.V. & Cañas, J.C.M. i, 2003. Análisis estadístico con SPSS 11.0 para Windows 2nd ed. McGraw-Hill, ed. [ Links ]
]]>Von Wangenheim, C.G. et al., 2010. Best practice fusion of CMMI-DEV v1.2 (PP, PMC, SAM) and PMBOK 2008. Information and Software Technology, 52(7), pp. 749-757. [ Links ]
]]>
