1.1 Evidence of content and internal structure of an instrument to measure work engagement in workers belonging to Colombian labor organizations

The definition of work engagement has been much discussed in the field of positive organizational psychology (^{Bakker et al., 2008}). Several researchers highlight inconsistencies both in the different definitions of the concept and in the instruments developed for its measurement (^{Hirschfeld & Thomas, 2008}; ^{Jeung, 2011}; ^{Macey & Schneider, 2008}; ^{Nienaber & Martins, 2014}).

The construct of work engagement is gaining relevance in today’s organizations, precisely thanks to the positive psychology perspective that focuses on encouraging the well-being and motivation of workers (^{Seligman et al., 2005}, p. 411).

This theoretical framework takes into account the model of work demands and resources (^{Bakker & Demerouti, 2013}, p. 109), which proposes an interaction between these two variables and generates two processes: one of them, the deterioration of health associated with burnout, and a process related to job satisfaction and commitment to the organization, associated with engagement (^{Schaufeli et al., 2006}, p. 702).

Thus, work engagement becomes relevant in the organizational environment as it is associated with outcome variables, and there is research that relates it to both high rates of job satisfaction and job performance. In this sense, it has been found that it is strongly associated with job satisfaction, showing that the psychological state of presence in a rol facilitates the feeling of well-being at work (^{Karatepe & Karadas, 2015}).

The instrument is designed taking into account that, for the psychology of organizations and work and for management, the observation of the effects produced by the characteristics and conditions of work on people has currently gained relevance (^{Paskvan & Kubicek, 2017}). Thus, in order to understand, explain, and even predict the levels of well-being, motivation, performance, and work engagement of an organization’s employees, it is necessary to have valid and reliable instruments to measure such constructs.

The construct to be measured is work engagement. In order to operationalize the construct, it is necessary to define engagement, which was proposed by ^{Kahn (1990)} as “the use that members of an organization make of themselves in their work roles, self-employment and physical, cognitive and emotional self-expression” (p. 693).

According to ^{Meyer and Allen (1991)} engagement has three components: “emotional, which refers to emotional attachment, permanence, which refers to commitment, and finally normative commitment” (p. 62). ^{Robbins (1996)} defines engagement as “an employee’s personal identification with a particular organization, its goals and desires” (p. 93). ^{Jericó (2001}) defines this construct as “the motivation to remain and contribute to an organization” (p. 104). ^{Colquitt et al. (2007)} define it as “the employee’s desire to continue working in the organization” (p. 207).

In this order of ideas, it is necessary to highlight the definition employee engagement and work engagement, given that these concepts have been commonly used as equivalent (^{Guest, 2014}, p. 143). However, work engagement refers to “the relationship of the employee with his or her work at the individual level and employee engagement refers to the relationship of the employee with his or her organization” (^{Salanova & Schaufeli, 2009}, p. 60; ^{Tisu et al., 2020}, p. 170).

In this order of ideas, it should be clarified that the present instrument responds to the measurement of work engagement.

When tracing antecedents that refer to instruments constructed for the Colombian working population, there are few publications to highlight. During the last ten years, the only instrument that was referenced in a relevant way was the Work Engagement Scale (UWES; ^{Schaufeli & Bakker, 2003}), which has been validated multiculturally in several countries of the world. However, in Latin America there are no reports of total validations of the three initial factors proposed by its authors. This is evidenced, for example, in the research called “Validation of the UWES-9 scale in health professionals in Mexico”, conducted by ^{Hernández et al. (2016)}, who performed the validation reporting only Cronbach’s alpha (p. 89), which although it adjusts (.45 and .53), it is not so close to 1. They also do not take into account the analysis of the omega coefficient, which is an excellent alternative when it comes to testing the reliability of a psychometric test, since it is well known that the decrease in the response alternatives decreases the variability of the scale, affecting the alpha coefficient, which is an index used to measure the internal consistency reliability of a scale, that is, it evaluates the magnitude to which the items of an instrument are correlated (Cronbach, 1943, p. 487). A new scale to measure work engagement was also found, which consists of 10 items and shows an essentially unidimensional structure. Reliability was excellent (a = .92; α = .92), and evidence of validity was obtained in relation to Organizational Climate (r = .540), Entrepreneurial Personality (r = .701), Happiness (r = .674), Emotional Repair (r = .470), and Emotional Stability (r = .440). (^{Prieto et al., 2021}, p. 135).

Although this study refers to adequate reliability and validity indexes, the validation in the Colombian population had a significantly lower percentage (5.5%), compared to the percentage of the Spanish population participating (76%). Thus, the aim is to develop a scale with psychometric properties that considers the idiosyncrasies of the context, given that the conceptions of work and work commitment may be affected by it. In this order of ideas, the present research reviews these two scales and constitutes them as research antecedents.

1.2 Definition of The Measurable Variable

The variables to be measured are those that make up the engagement construct.

Authors such as ^{Meyer and Allen (1991)} state that the behavior of being committed to the organization can be explained through three components: “the emotional, the permanence and the normative” (p. 64). Considering this approach, and in order to carry out the construction of the instrument in a coherent manner, the postulates of cognitive psychology are taken up again, and this is established as the theoretical framework that will support the instrument, since this theoretical framework postulates that people generate knowledge and ascribe meaning to the world that surrounds them through schemas. These are described as information structures relatively stable in time, containing information about the world, others and oneself, and their content is generated through the processing of the information accessed by the cognizing subject. After that, behavior appears, which would then be the result of such processing and which is usually explained by the pre-established schemas.

These are classified as cognitive (interpreting the facts), emotional (in charge of the emotional response), and instrumental (preparing for action) (^{Clark, 2004}, p. 350).

The instrument is constructed by operationalizing work engagement in schemes that are susceptible to measurement and that will be grouped into three factors: cognitive, referring to those thoughts, ideas or images that are related to the meaning that work has for the subject; emotional, refers to the emotions that are predominant in the subject with respect to the work performed (i.e., happiness, tranquility, anguish); finally, behavioral is related to what the subject does (i.e., attitudes and aptitudes towards his/her work). The present study aims to collect evidence of content and internal structure of an instrument to measure work engagement in the Colombian working population.

2.1 Participants

The sample consisted of 460 workers of both sexes, 206 men (44.8%) and 254 women (55.2%), aged between 18 and 65 years (mean: 33; deviation: 11.25), from various departments of Colombia. The participants were selected by means of an accidental non-probabilistic sampling, workers of labor organizations of different economic sectors: commerce and services, 179 (38.9%); industry, 153 (33.3%); health, 91 (19.8%); and education, 37 (8%). They had an average time of work of three years (deviation: 1.59) and a link to the labor organization mostly indefinite of 41% and, less frequently, a temporary link with 2.4%.

As eligibility criteria, participants were only required to be workers, of legal age and with more than three months of work in a given organization. The data collection period was eight (8) consecutive months.

2.2 Instrument

The instrument was designed by a group of professionally trained researchers, psychologists and doctors in psychology, specialized in organizational and work psychology. Likewise, specialists and masters in training participated in its development with the objective of reviewing the state of the art on measurement instruments for the aforementioned construct. In addition, experts were involved in the development of the scale, who guided, through their judgment, the items that finally made up the initial scale.

For the development of the items, several theoretical positions were reviewed, among which the cognitive paradigm was found, as mentioned above. Likewise, the contributions of positive psychology are reviewed, which focuses on encouraging the well-being and motivation of people (^{Seligman et al., 2005}, p. 412), and also considers approaches of the psychology of work about the demands and labor resources (DLR; ^{Bakker & Demerouti, 2013}, p. 110), from which an interaction between these two variables is reported (DLR) and argues that depending on this interaction, two processes are generated: one of them, the deterioration of health associated with burnout, and another process that would be job satisfaction and commitment to the organization, associated with engagement (^{Schaufeli et al., 2006}, p. 702).

Thus, taking into account each of these contributions, the work engagement variable is operationalized in three factors: cognitive, behavioral, and emotional. As a criterion for establishing the number of items, representativeness was taken into account, that is, the number of items should show the representative behaviors of each dimension, in addition to an overestimation of the number of items by considerign their loss during the psychometric analyses. In this regard, 20 items per dimension were developed and 15 were overestimated for each one, for a total of 45 items.

The instrument was structured as follows. The behavioral dimension takes up the definition of behavior given by Watson, which is: “what the organism does or says”. He goes on to clarify that “to speak is to do, that is, to behave” (^{Watson, 1961}, p. 23). In this order of ideas, this dimension consists of statements that refer to actions and/or are related to what to do at work with items 1, 4, 10, 13, 16, 19, 22, 25, 28, 31, 34, 37, 40, 43 and 7. An example of an item of this dimension is: “I make an effort to do my job well”.

The emotional dimension, in which emotion is understood as a multidimensional experience characterized by three response systems: cognitive/subjective; behavioral/expressive, and physiological/adaptive (^{Lang, 2000}). The items that make up the dimension are oriented to identify the emotions generated in individuals with respect to their work, which are: 2, 5, 8, 11, 14, 17, 20, 23, 26, 29, 32, 35, 38, 41, and 44. An example of this item is: “My work makes me happy”.

Finally, the cognitive dimension, which understands cognition as the process of knowing, thinking or mentally processing information, such as images, concepts, words, rules, and symbols (^{Coon & Mitterer, 2010}). The items that make up this dimension are 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45. An example of this item is: “I work extra time without realizing it”. Response options were defined on a Likert scale, in which Never =0, Almost never = 1, Sometimes = 2, Almost always = 3, and Always = 4.

2.3 Procedure

The process of planning, construction, and validation of the instrument began by defining the purpose, use, justification for its design, and the theoretical perspective from which the items would be made (^{Muñiz & Fonseca, 2019}, p. 15).

When the theoretical framework was defined and elaborated, the items were designed following the technical guidelines for their construction (^{Lancheros et al., 2007}). Once the instrument was developed, the first evidence related to the content was collected through the judgment of seven experts, whose profiles were characterized by being human management managers, and their professional training had to be psychologists. This process was carried out over a period of 6 months.

The methodology used for the content-based evidence was the one proposed by ^{Lawshe (1975)} and modified by ^{Tristán (2008)}. This methodology allow to establish the index of agreement by considering the evaluation that the seven experts made of each item of the three dimensions that make up the test. The experts classified each item as essential, useful but not essential, and not necessary. After this classification, the content validity ratio (CVR) was calculated, which according to Tristán is considered acceptable if its value is higher than .58. Likewise, it was considered that an index lower than .8 and higher than .58 indicates the revision of the item. Finally, the content validity index (CVI) per dimension was calculated, taking into account the validity ratio of the items for revision or approval; an index equal to or higher than .8 is considered adequate. This evidence made it possible to improve the content of the items.

A sample of 460 participants was collected, following the criteria reported by ^{Hogan (2004)}, regarding having at least ten participants per item (p. 176). Having 460 participants reduces the probability of obtaining unstable factors and misleading results (^{Pérez & Medrano, 2010}, p. 60). The time it took to collect the information was eight months.

2.4 Data Analysis

Once the responses had been applied and collected, the corresponding analyses were carried out. Initially, descriptive analyses were performed to determine the sociodemographic and labor characteristics of the sample. Then, the item analysis was performed in order to know the response behavior of each item within the dimension; for this purpose. The discrimination was calculated using the corrected item-test correlation coefficient, seeking with this analysis to increase the internal consistency of the dimension by eliminating those items whose indicator was below .25 (^{Nunnally & Bernstein, 1995}).

Subsequently, sample adequacy analyses were done to identify the relevance of the data to perform the exploratory factor analysis (EFA). In this regard, the Kaiser Meyer Olkin index (KMO), which had a correlation greater than .7 and Bartlett’s test with a p value less than or equal to .05, was verified (^{Ferrando & Lorenzo, 2014}).

After completing the previous step, we proceeded to the EFA, using the unweighted least squares (ULS) factor extraction method, with direct Oblimin rotation, identifying the number of factors through Horn’s parallel method. For reliability, the internal consistency method was used, including the ordinal Cronbach’s alpha, considering that the scale is ordinal, as well as working under polychoric correlations (^{Ferrando & Lorenzo, 2014}; Ferrando et al., 2022, p. 10; ^{Freiberg et al., 2013}; ^{Lloret et al., 2014}).

For the residual analysis, descriptive analyses were performed to observe the frequency of the error, expecting a symmetrical distribution. To verify this indicator, the residual root mean square root (RMCR) was calculated. According to ^{Harman (1976)}, the value must be equal to or less than .05 to consider that the model fit is acceptable. This indicator was also confirmed using the analysis proposed by ^{Kelley (1935)}, whose criterion must be higher than the RMCR value. Finally, for model fit, the Gamma Index (GFI) was used to interpret the proportion of covariation between the variables explained by the proposed model; a value equal to or greater than .95 is expected (^{Ferrando & Lorenzo, 2014}; ^{Ferrando et al.,2022}, p. 10).

For descriptive analyses we worked with SPSS version 25 and for psychometric analyses we worked with FACTOR version 10.10.01 (October, 2019) (^{Ferrando & Lorenzo, 2017}).

2.5 Ethical Considerations

The application was carried out individually. Before answering the test, each participant had to sign the informed consent form, in which he/she agreed to be part of the research. It is important to note that participation was voluntary and the workers were assured of the confidentiality of the data and the anonymity of their answers.

2.6 Results

The results of the study are presented below: the first section describes the item construction process; the second section presents the results of the content evidence through expert judgment, showing the agreement indexes obtained from the experts for each of the items; the third section presents the descriptive analyses and the item-test correlation of each of the items, taking into account their dimension; the fourth section describes the second evidence of validity related to the internal structure, where the results of sample adequacy and the internal structure proposed by the exploratory factorial model with its respective reliability are presented.

2.7 Item Construction

The construction of the items was based on the approaches of the cognitive psychology paradigm, thus operationalizing the construct of engagement in three dimensions composed of the cognitive factor, the emotional factor and the behavioral factor. Those items were reviewed considering the following aspects: representativeness, relevance, diversity, clarity, simplicity, and comprehensibility (^{Muñiz & Fonseca, 2019}, p. 17). Subsequently, these items were judged by seven experts in the area of organizational psychology, all of whom work as human resources managers in various labor organizations in Colombia and had knowledge of the work engagement construct. In this process, three items were eliminated because of their wording and similarity with the others. Thus, the initial instrument was structured by 45 items, 15 of the behavioral dimensions, 16 of the affective dimensions, and 14 of the dimensions, given that it was intended that each of them be operationalized accurately and that the instrument meet reliability, validity, and objectivity criteria. After the expert review, the items were refined and the instrument was made up of 42 items (see Table 1).

2.8 Descriptive analysis and evaluation of statistical assumptions of the instrument

Table 2 shows that most of the items presented a mean around 3, the deviation in most of the items was close to 1, and the skewness and kurtosis indicated a nonnormal distribution of the data. The item-test correlation was higher than .25, indicating correlation of the items within the factor, with the exception of item 3 of the cognitive factor that does not meet this criterion. Therefore, it was eliminated to continue with the EFA.

For the exploratory factor analysis (EFA), the sample adequacy of the items was initially reviewed through the Kaiser Meyer Olkin index (KMO), which showed a value of .95 and Bartlett’s Test of Sphericity, which rejects the null hypothesis, indicating a strong relationship between the items. Therefore, the criteria for continuing the EFA were met. Table 3 presents the factor analysis; the parallel method suggests three significant factors with an explained variance of 63%. Each factor obtained a reliability of .95, .97, and .87, respectively, which indicates precision in the measurement and consistency of the factor items to measure the dimension. It is important to highlight that item 36, corresponding to the affective dimension, was grouped with the items of the behavioral factor. Likewise, items 6, 17, 26, 32, 37, 39, and 42, corresponding to the cognitive dimension, were grouped with the items of the behavioral factor. Item 1 of the behavioral dimension was grouped with the items of the affective dimension. Items 4, 9, 12, 15, 21, 35 of the behavioral dimension, and items 3, 8, and 11 of the cognitive dimensions were eliminated because they presented high residuals (standardized values higher than -2 or 2) and ambiguity (i.e., they loaded on more than one factor).

With respect to the fit indices, the GFI 0.99 and CFI 1 indicate the fit of the proposed factorial structure, and the error indicators RMSR and Kelley, with .03 and .04, indicate low residuals.

Finally, the correlation between factors is established, in which it is observed that the behavioral dimension has a relationship of .77 with the emotional dimension and .42 with the cognitive dimension; the emotional dimension has a relationship of 4.63 with the cognitive dimension.

3. Discussion

This manuscript shows the results of a research process that aimed to build and validate through evidence, such as content and internal structure, an instrument to assess work engagement in the Colombian working population, belonging to organizations from different economicsectors. After the evidence of internal structure through exploratory factor analysis, this tool was structured in three factors and 32 items as follows. The behavioral dimension contains items 18, 24, 27, 30, 33, 38, 40, which are part of the initial theoretical proposal; it should be noted that item 36, which belonged to the emotional dimension, and items 6, 17, 26, 32, 37, 39, and 42, which belonged to the cognitive dimension, were also grouped in this factor. An example of an item of the behavioral dimension is: “In my work I persist until things go well for me”. The affective dimension was made up of items 2, 5, 7, 10, 13, 19, 22, 25, 28, 31, 34, and 41, which coincided with the theoretical proposal and item 1 of the behavioral dimension was added. An example of an item of this dimension is the following: “I feel proud of my work”. And the cognitive dimension was organized with items 14, 20, 23, and 29. An example of an item of this dimension is: “I get carried away with my work to the point that I don’t feel that time is passing”.

With respect to the grouping of the items, it is recognized that the cognitive dimension, which at a theoretical level is related to the schematic structures that refer to the “meaning” that “organizational commitment” has for each employee, was the dimension that underwent the most important changes. Most of its items were grouped in the behavioral dimension, which has a theoretical relationship supported by the cognitive psychological approach, which argues that thoughts are translated into actions, that is, into behaviors (^{Delval, 1977}, p. 25). More explicitly, a thought generally translates into a behavior, into an action. This shows that the items elaborated for the cognitive dimension were finally translating elaborated behaviors related to work engagement. It is also important to note that the four items that remained in the cognitive dimension (14, 20, 23 and, 30) are translating the meaning of the individual’s commitment to his or her work, which characterizes the employee. It is important to note that, although the three established dimensions do not have the same number of items, this does not affect the quality of their measurement, since the construct of work engagement after the EFA evidenced the structure of the three components: behavioral, emotional, and cognitive, and its items are sufficient to represent the content suggested by the theoretical perspective.

Although, as mentioned in the manuscript, there are other scales to measure the construct, among which the Work Engagement Scale (UWES; ^{Schaufeli & Bakker, 2003}) and the new scale to measure Work Engagement (^{Prieto at al., 2021}, p. 135), both validated for the Spanish-speaking population, only one of them, particularly the latter, has been validated in the Colombian population. However, the percentage of the sample of Colombian workers is significantly lower with respect to the sample of Spanish workers (5% to 76%, respectively). The first scale has shown validations in the Spanish-speaking population, but the validation of its dimensions has been unstable: in some populations the reliability and validity of the three-factor structure for measuring engagement (vigor, absorption and dedication) is observed, but in other populations this structure is not proven. Based on this, such is the case of Puerto Rico, Argentina, Finland, Spain, Italy, Germany, and Norway, where the applications carried out determined that the three-factor structure is the one that best fits the model (^{Balducci et al., 2010}, p. 143; ^{Nerstad et al., 2010}, p. 327; Schaufeli et al., 2002, p. 465; ^{Seppälä et al., 2009}, p. 459; ^{Spontón et al., 2012}, p. 147). In some cases, the three dimensions share high correlations (α = .93, α = .92, and α = .93 for each factor). However, and contrary to this, in Countries such as South Africa, Brazil, China, and Japan, the unifactorial structure resulted more adequate: α = .83 (Chun-tat & Ng, 2011, p. 7; ^{Shimazu et al., 2008}, p. 510; ^{Souza et al., 2015}, p. 207; ^{Storm & Rothmann, 2003}, p. 62).

Regarding the scale constructed to measure the construct in the Colombian population, the data found in the exploratory factor analysis performed suggested (α = .95, α = .96, and α = .87 for each factor) that at a theoretical level work engagement can undoubtedly be sustained from the three proposed factors: cognitive, emotional, and behavioral. Additionally, it could even have greater context in terms of the characteristics of Colombian labor organizations, and, thus, this scale could be tested with larger samples in Latin America, given that the idiosyncrasies regarding labor organizations and work are shared.

^{Martínez et al. (2006)} state that although the instrumentconstructed by Schaufeli found empirical support for this three-factor structure in Spain among workers and students, the data they published did not support this conclusion, since the goodness-of-fit indicators obtained for the three-factor structure, although better than those found for the one and two-factor structures, yielded a GFI (Goodness of Fit Index) of .91 and a CFI (Comparative Fit Index) of .90. On the other hand, the RMSEA (Root Mean Square Error of Approximation) of the three-factor model was .06. The three statistics show that the threefactor structure would not present a good fit, but would be barely acceptable. As can be seen, these data contradict what was found in the present exploratory analysis, and for this reason it would be worthwhile to carry out a confirmatory study, as mentioned above, increasing the sample in the Latin American population.

There are also other studies conducted in Latin America that have not been able to replicate the trifactorial structure proposed by the UWES-9 (^{Parra & Perez, 2010}, p. 130), and instead accept the two-factor structure in a study conducted with psychology students.

In another study, these same authors identified a similar bifactorial structure in a sample of university students from different careers applying the 17-item version (^{Parra & Pérez, 2010}, p. 128), but it is important to clarify that in both studies academic engagement is evaluated, with no previous studies in Chilean workers.

Likewise, and taking up again the publication of the new scale of engagement at work (^{Prieto et al., 2021}, p. 135), it is evident that the measurement of work engagement continues to be of interest in recent years, in this case the scale shows reliability and validity indexes. The researchers selected a sample of 599 active workers, 51% of whom were classified as entrepreneurs. Fifty-three percent were men and the mean age was 44.41 years (SD= 8.78). The newly developed scale consists of 10 items and shows an essentially unidimensional structure. Reliability was excellent (a = .92; ω = .92), concluding that the scale developed for the assessment of work engagement shows good psychometric properties. However, it is emphasized and insisted that the Colombian population with which the scale was validated was significantly lower, as mentioned in previous paragraphs (^{Prieto et al., 2021}, p. 135).

This gap continues to make viable the need for instruments to measure the construct that consider the idiosyncrasies of the population.

Considering the psychometric analysis of the scale to measure work engagement in the Colombian working population, it can be affirmed that the test presents quality criteria that allows it to be reliable, since it confirms the internal consistency and precision in the measurement of the construct. In addition, the evidence of content and internal structure allows the interpretation of the scores under the theoretical proposal that sustains this instrument.

Therefore, and considering the theoretical criteria from which the dimensions and items were constructed and defined, it is possible to obtain an instrument that can be granted validity, reliability, and objectivity criteria to perform objective measurements of work engagement in the Colombian working population. This benefits not only professionals working in the area of work psychology in their processes of evaluation, selection, and management of human talent, but also in academic research processes.

Finally, it should be noted that the instrument considers the idiosyncrasies of the Colombian working population, understanding that this determines the forms of adaptation to the environment in general.

The instrument is comprised of 32 items, structured as follows in Table 4.