Validation of an instrument to measure tutor performance in promoting self-directed learning by using confirmatory factor analysis

Validación de un instrumento para medir el desempeño de los tutores en la promoción del aprendizaje autodirigido usando análisis factorial confirmatorio

Validação de um instrumento para medir o desempenho dos tutores na promoção da aprendizagem autodirigido usando análise fatorial confirmatório

Genoveva Amador Fierros¹;Osval Antonio Montesinos-López ²; Noemí Alcaráz Moreno³

¹Nurse, Master's. Professor, Universidad de Colima, México. email: genoveva@ucol.mx.

²Engineer, PhD. Professor, Universidad de Colima, México. email: oamontesl@hotmail.com.

³Nurse, PhD. Professor, Universidad de Colima, México. email: almoreno@ucol.mx.

Receipt date: January 15, 2015. Approval date:December 4, 2015

Article linked to research: The role of tutors in the self-managment of self-directed learning of nursing students.

Subventions: Universidad de Colima, Faculty of Nursing.

Conflicts of interest: none.

ABSTRACT

Objective.This work sought to validate and propose an instrument to measure the performance of tutors in promoting self-directed learning in students involved in processes of problem-based learning. Methods. Confirmatory factor analysis (CFA) was applied to validate the instrument composed of 60 items and six factors (self-assessment of learning gaps within the United Nations specific context: self-assessment, reflexion, critical thinking, administration of information, group skills), using a sample of 207 students from a total of 279, which comprise the student population of the Faculty of Nursing at Universidad de Colima in Mexico. (2007). Results. The CFA results demonstrated that the instrument is acceptable to measure performance of tutors in promoting self-directed learning, given that all the indicators, variances, covariances, and thresholds are statistically significant. Conclusion. The instrument permits obtaining students' opinions on how much professors contribute for them to develop each of the 60 skills described in the scale. Lastly, the results could report if professors are placing more emphasis in some areas than in other areas they should address during the problem-based learning (PBL) process, or if definitely their actions are removed from the premises of PBL, information that will be useful for school management in decision making on the direction of teaching as a whole.

Key words: learning; validation studies; preceptorship; students, nursing.

RESUMEN

Objetivo.Validar y proponer un instrumento que mide el desempeño de los tutores en la promoción del aprendizaje autodirigido en los estudiantes involucrados en procesos de ABP. Métodos. Se aplicó el Análisis Factorial Confirmatorio (AFC) para validar el instrumento compuesto por 60 ítems y seis factores (Autoevaluación de las Brechas de Aprendizaje dentro de las Naciones Unidas Contexto Especifico: Autoevaluación, Reflexión, Pensamiento Crítico, Administración de la Información, Habilidades de Grupo), utilizando una muestra de 207 estudiantes de un total de 279, que conforman la población estudiantil de la Facultad de Enfermería de la Universidad de Colima, México. (2007). Resultados. Los resultados del AFC demostraron que el instrumento es aceptable para medir el desempeño de los tutores en la promoción del aprendizaje autodirigido ya que todos los indicadores, las varianzas, covarianzas y thresholds son estadísticamente significativos. Conclusión. El instrumento permite obtener la opinión de los estudiantes sobre cuánto el profesor contribuye para que ellos desarrollen cada una de las 60 habilidades descritas en la escala. Al final, los resultados podrían informar si el profesor está haciendo más énfasis en una área que debe atender durante el proceso del ABP o en otra, o si definitivamente su actuación se aleja de las premisas del ABP, información que será útil para la administración escolar en la toma de decisiones sobre el rumbo de la docencia en su conjunto

Palabras clave: aprendizaje; estudios de validación; tutoría; estudiantes de enfermería.

RESUMO

Objetivo.Validar e propor um instrumento que meça o desempenho dos tutores na promoção da aprendizagem autodirigido nos estudantes envolvidos em processos de ABP. Métodos. Se aplicou uma Análise Fatorial Confirmatório (AFC) para validar o instrumento composto por 60 itens e seis fatores (Auto-avaliação das Brechas de Aprendizagem dentro das Nações Unidas Contexto Especifico: Auto-avaliação, Reflexão, Pensamento Crítico, Administração da Informação, habilidades de Grupo), utilizando uma amostra de 207 estudantes de um total de 279, que conformam a população estudantil da Faculdade de Enfermagem da Universidad de Colima, México. (2007). Resultados. Os resultados do AFC demostraram que o instrumento é aceitável para medir o desempenho dos tutores na promoção da aprendizagem autodirigido já que todos os indicadores, as variâncias, covariâncias e thresholds são estatisticamente significativos. Conclusão. O instrumento permite obter a opinião dos estudantes sobre quanto o professor contribui para que eles desenvolvam cada uma das 60 habilidades descritas na escala. Ao final, os resultados poderiam informar se o professor está fazendo mais ênfase em uma do que em outras das áreas que deve atender durante o processo do ABP, ou se definitivamente sua atuação se afasta e as premissas do ABP, informação que será útil para a administração escolar na toma de decisões sobre o rumo da docência em seu conjunto.

Palavras chave:aprendizagem; estudos de validação; preceptoria; estudantes de enfermagem.

INTRODUCTION

The role of professors in student learning is key, especially when they are inserted in educational methods aimed at promoting greater autonomy in students' learning processes. Such is the case of problem-based learning (PBL) in which the professor's role is that of a tutor, who is expected to guide and facilitate the work of students leading to their achieving certain skills that, as a whole, help to form self-directed learners. The constructivist theory explains this process by specifying that action is amid the contributions from the object and from the subject. If action does not occur, progress will not exist in the development of knowledge. In this case, it refers to the action of individuals who learn, if they do not act with the information, data, processes, etc., then progress does not take place in learning, no matter how much the professor is intent on teaching.¹ Due to this, in PBL, the tutor's role consists in guaranteeing the conditions in which the students' learning process can take place. Wolff and Rideout² explain that the tutor's role within a PBL process, to be effective, must start from didactics and diverse approaches that convert tutors into facilitators of learning and, in their action, emphasize on the dissemination of information, on questioning logic, values and beliefs of students, on helping them clarify their learning needs and select the resources to learn, as well as to facilitate for them the task of discussion and evaluation. All these actions describe the concept of self-directed learning, described by Malcolm Knowles, since 1975, to allude to the path students must follow to become autonomous and have control of their own learning.³

In PBL, evaluations play a predominant role, not only to monitor and support the results of learning, but also to know if professors/tutors are really performing the role expected of them in this method. In fact, this method, unlike others, promotes evaluation in all its forms: self-assessment, evaluation among peers, and student evaluation of the tutor, besides the professor/tutor evaluation of the students. Thus, the concept of evaluation in PBL changes the idea of judging for the idea of helping to improve. All the players in PBL know that the evaluation results serve to make decisions that permit improving the individual or group learning process.

Students' opinions and not only the results of their learning matter when interest centers on knowing what and how much was achieved, as well as on how such achievements were obtained. This is why it is interesting to assess the work of professors and in this task, the best sensors, besides the self-assessment, will always be their students. With that purpose an instrument was designed, Likert-type scale, composed of 60 items. It was constructed from the skills and expected disposition in students as a way to develop the skill of being self-directed students. The underlying idea in the design of the instrument is that the work of the tutor must focus on guaranteeing the conditions that permit students to develop skills to direct their own learning. The skills and dispositions upon which the instrument is based were described by Crooks, Lunky-Child, Patterson and LeGris in their chapter "Facilitating Self-Directed Learning",² taking from there the idea of the structure of the instrument that measures six factors (self-assessment of learning gaps, self-assessment, reflexion, critical thinking, administration of information, and group skills) and for each factor ten items were used. The ten items related to the group process were taken in totality from Figure 4.3 titled: Criteria for Evaluation of Individual Performance in Tutorial-Group Process, with rights belonging to the School of Nursing at the University of McMaster.²

The purpose of this article was to introduce evidence of the validation of the instrument to measure tutor performance in promoting self-directed learning in students involved in PBL processes. It is worth mentioning as an antecedent that in 2007 Amador et al., published an article titled "The Role of Tutors in Self-management of Self-directed Learning of Nursing Students", where they used an instrument that measured four skills integrated into three reactive areas: reflexion and critical thinking; administration of information, and the group process.⁴ The instrument did not take into consideration two skills linked to self-assessment, which are part of the skill for the self-direction of learning, hence, the authors continued working until finding a way to evaluate said skill in the most comprehensive manner. For this reason, according to the review of the literature cited, an instrument with six factors was proposed (F1 = self-assessment of learning gaps within a specific context, F2 = self-assessment, F3 = reflexion, F4 = critical thinking, F5 = administration of information, F6 = group skills) with 10 indicators per factor as acceptable to measure this phenomenon, through the opinions from students of the Faculty of Nursing at Universidad de Colima.

METHODOLOGY

Instrument and data collection

This research was conducted in the Faculty of Nursing at Universidad de Colima, México. At the moment of collecting the data (2012), the student population in the Faculty of Nursing was of 279 students. The sample used in this study was of 207 students selected under simple random sampling from the student population in the Faculty of Nursing, representing 74.1% of the population. The questionnaire was applied through a survey of the students selected with two basic criteria; in the first place the students had to be in their second year of studies or above, given that they would already be familiarized with the learning method; the second selection criterion was that they could not be carrying out their social service, but still in the school phase through tutoring. Sixty indicators and six latent factors have been proposed for the instrument. All the indicators were measured with a five-point Likert scale (always, almost always, sometimes, rarely, and never). Hence, the analysis was performed through ordinal approach.

According to the theory reviewed, empirical research in this area of study, and a combination of both, 60 items are proposed linked to six underlying latent factors (F1 to F6) [F1I1 = guides students into exploring their current state of knowledge in relation to the problem studied; F1I2 = asks students to express what they do know about the problem of study; F1I3 = asks students to express the knowledge they ignore about the problem of study; F1I4 = asks students to express what they need to know to understand the problem; F1I5 = encourages students to define what they want to learn; F1I6 = asks students to state how much they want to learn about the problem; F1I7 = guides students to define what they will need to do to achieve the knowledge and learning they wish to obtain; F1I8 = helps students identify the limitations to obtain the knowledge desired; F1I9 = provides confidence for students to define what they are realistically capable of doing; F1I10 = encourages students to discern between what they should do and what they will really do to obtain the learning desired; F2I1 = gets students to become familiar with evaluation standards and criteria; F2I2 = invites students to gather and interpret data on their own performance during the evaluation process; F2I3 = provides feedback that helps students to construct their learning plans in areas of their concern; F2I4 = guides for students to center their self-assessment less on themselves and more on knowledge, skills, abilities, as well as on professional, technical, and ethical conducts appropriate to the experience; F2I5 = asks students to examine how comfortable or uncomfortable they feel with the idea of directing their own learning and with the self-assessment; F2I6 = invites students to externalize their anxieties with respect to the different approaches presented; F2I7 = invites students to explore their own learning method; F2I8 = asks students to communicate anxieties with respect to their strengths and weaknesses; F2I9 = helps students see the evaluation as an opportunity for students to identify their own strengths and weaknesses; F2I10 = motivates students to set challenges and reach them, thereby, increasing their confidence in themselves; F3I1 = promotes introspection (internal observation of thoughts, feelings, or acts) in the individual experience; F3I2 = asks students to remember and describe an experience in retrospective; F3I3 = asks to identify possible feelings toward the experience that can act as facilitators were limitations of the learning; F3I4 = requests validating prior learning and identifying new learning; F3I5 = invites to become aware that reduction needs to be exquisite moment of action or experience or after such; F3I6 = asks to describe the feelings and determine the source (that originates them) and the reason for those feelings; F3I7 = invites to examine situations objectively, which makes it more apt for students to identify bias; F3I8 = propitiates for students to make a conscious and periodic examination of their own learning process; F3I9 = helps create awareness that beliefs and values play an important role in the way situations are reflected upon; F3I10 = helps to understand that the attitude of students and their success may be influenced or determined by beliefs and values; F4I1 = makes sure that students are aware that critical thinkers suspend their judgment until having tested all the evidence; F4I2 = helps to clearly establish the purpose of learning, in clear, significant, and realistic terms; F4I3 = makes students consider each angle of the problem and that reasoning strategies being openly explored; F4I4 = promotes four points of view to be examined openly against other alternative points of view of the group to identify weaknesses and strengths from those points of view against the problem in question; F4I5 = motivates for the assumptions for arguments proposed on the problem to be identified, testified, or discarded by the group members; F4I6 = asks students or group members for data to be sufficient to support their discussions and to consider other alternative data that may even contradict them; F4I7 = makes sure the key concepts are identified, explained, explored, and used with precision; F4I8 = constantly makes sure that inferences or conclusions are derived from consistent evidence and data; F4I9 = promotes for the reasoning process to be examined by each group member to determine how it leads to implications or consequences; F4I10 = promotes attitudes of responsibility, discipline, perseverance, integrity, and humility as part of the formation of a critical thinker; F5I1 = makes sure the problem or question that will guide the learning process has been established by the group; F5I2 = promotes the exchange of ideas on search alternatives for information, collection and personal filing of relevant sources, including computer searches; F5I3 = makes sure the group members know how to and where to look for relevant information for the program in which they are working; F5I4 = promotes persistence and attention to detail during the information search process; F5I5 = makes sure the information delivered by students is used to debate ideas and reach conclusions; F5I6 = asks to read and critique research in nursing, in health sciences, and related disciplines; F5I7 = requests from students to explain how they would apply research findings from nursing, health sciences, and from other disciplines in their nursing practice; F5I8 = makes sure students and group members organize and select information in the way that best helps to understand what to address the problem; F5I9 = asks the group to expose the sources of information consulted and two issue its own evaluation on their reliability and quality; F5I10 = provides feedback on the evaluation of the sources of information and the means used to investigate and be informed; F6I1 = contributes to the development of the group's objectives; F6I2 = helps maintain the orientation of the group toward the task; F6I3 = makes sure tasks have been defined according to the group's negotiation; F6I4 = effectively communicates ideas and information. F6I5 = listens and responds to each student; F6I6 = motivates participation from group members; F6I7 = helps group members in their learning; F6I8 = respects the rights of group members to express their values and opinions; F6I9 = provides constructive feedback; F6I10 = takes constructive action will approach conflict within the group.

Also, to estimate the hypothetical model the first coefficient (load) of each factor has been set at 1 to guarantee a well-identified model (with more observations than parameters to estimate). This model also allows correlated factors. Fifteen correlations or covariances were established because there were six factors. This model is over identified (with more observations than parameters to estimate), given that the number of observations is greater than the number of parameters. Sixty indicators yield 60 ((60 + 1))/2 = 1830 as the number of observations. There were 308 parameters (54 load factor variations, 6 factor variances, 233 thresholds, and 15 covariances among factors) to estimate. Hence, there were 1522 degrees of freedom to test this model. When variables are categorical, thresholds are required to obtain the estimations of each of the response categories, which is why five thresholds were used for each categorical variable.

The hypothetical model sought to be confirmed to measure tutor performance in promoting self-directed learning is composed of six factors: F1 = self-assessment of the learning gaps within the United Nations specific context; F2 = self-assessment; F3 = reflexion; F4 = critical thinking, F5 = administration of information; F6 = group skills. It is assumed that F1 is composed of items F1I1, F1I22, F1I3, F1I4, F1I5, F1I6, F1I7, F1I8, F1I9, and F1I10, shown in Table 1. Factor F2 is conformed of items F2I1, F2I2, F2I3, F2I4, F2I5, F2I6, F2I7, F2I8, F2I9, and F2I10 (Table 1). Factor F3 is made up of items F3I1, F3I2, F3I3, F3I4, F3I5, F3I6, F3I7, F3I8, F3I9, and F3I10 (Table 1). Factor F4 is made up of F4I1, F4I2, F4I3, F4I4, F4I5, F4I6, F4I7, F4I8, F4I9, and F4I10 (Table 1). Factor F5 is comprised of F5I1, F5I2, F5I3, F5I4, F5I5, F5I6, F5I7, F5I8, F5I9, and F5I10 (Table 1); factor F6 is composed of items F6I1, F6I2, F6I3, F6I4, F6I5, F6I6, F6I7, F6I8, F6I9, and F6I10 (Table 1). It is assumed that the factors are correlated (relationship exists among them) and that the items are conditionally independent within each factor. The following tests the credibility of the hypothetical model based on data from the sample previously described by using CFA. This implies that the hypothetical model we seek to corroborate includes 60 indicators observed and six correlated factors.

Validity of construct and analysis

Validity refers to the degree in which an instrument measures that which it intends to measure. Construct validation establishes that an appropriate measurement operationalizes its underlying construct. Exploratory factor analysis (EFA) and CFA are commonly used to investigate the validity of construct.⁵ The EFA is usually used to research the validity of construct in cases in which the relationships among variables are unknown or ambiguous. While it is suggested to use CFA when the researcher has more knowledge of the theory, the empirical research, or both, and postulates relationships among the measurements observed and the underlying factors a priori and then seeks to statistically test this hypothetical structure.

Both EFA and CFA are tools researchers normally employ to reduce data dimension (variables observed) in latent factors, which facilitate data interpretation. The CFA differs from EFA in that it helps reduce measurement error and permits comparison of alternative models, fixed a priori at latent factor level.⁶ The CFA may also be used to statistically compare the structure of latent factors in two or more groups (for example, different disease conditions, different areas of knowledge). Using CFA to investigate the validity of construct of measurement instruments based on hypothesis adds a higher level of statistical precision and may aid in developing a less extensive and parsimonious instrument or in the confirmation of its possible subdomains.⁷

For this reason, to evaluate the validity of construct of the hypothetical model CFA⁸ was used, seeking to determine if the number of factors and the load of (indicators) the variables measured fit what is expected in the hypothetical model described previously, on the basis of the theory established. It is assumed that the latent factors or constructs are linked to the indicators observed from each factor. Evaluation of goodness of fit, of the model proposed with CFA, was carried out through the chi-square test. A non-significant value of the chi-square test indicates a good fit. However, the chi-square test is more sensitive with bigger sample sizes. Hence, Hu and Bentler recommend reporting two or three additional indexes.⁹ Hence, the comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square error (SRMR) will also be reported. The CFI evaluates the fit of the model in relation to another base model. Better fits are indicated with values close to 1, thereby, acceptable fits are considered with values > 0.96 for CFI and the Tucker-Lewis Index (TLI).¹⁰ The RMSEA shows lack of fit in a model compared to a perfect model. A value below 0.06 for RMSEA offers an acceptable model.¹¹ The WRMR evaluates average weighted residuals, which range from 0 to 1, suggesting good fit for values close to 1.0. The CFA analysis was performed with the MPLUS statistical package, version 6.11.¹²

Given that the responses to the items are ordinal, it is not appropriate to use the sampling Pearson correlation matrix to apply the CFA because the Pearson correlation coefficients are higher when they are calculated between two continuous variables than when calculated on a binary or ordinal scale. Due to this, in this work the analysis is based on the polychromic correlation matrix and using the estimation method of adjusted weighted least squares means and variance (WLSMV), along with the probit function to obtain the estimations of the appropriate parameters of the ordinal variables. The polychoric correlation matrix is the correct correlation matrix when variables are ordinal.

RESULTS

Data distribution by categories is quite biased with high proportions of categories 2 and 3, which means "sometimes" and "almost always", and low proportions in categories 0 and 5, meaning "no answer" and "never". Additionally, a low proportion of responses is noted for category "always", considered the highest positive value to score the tutor's role as enabler of the development of skills for self-directed learning.

>Confirmatory factor analysis

The hypothetical model described previously was fit with a CFA, given that the latent factors to measure tutor performance in promoting self-directed learning are continuous variables, while the indicators are ordinal responses. Therefore, the CFA performed is also known as confirmatory analysis under the item response theory (IRT). Data with ordinal responses (five categories) were fit by using a probit model through the WLSMV estimation method with probit function. The global fit of the model is reasonable, given that χ_ ((1695)) ^2 = 2628.924, p < 0.0001; the RMSEA was 0.052 < 0.06, with 90% confidence interval between 0.048 and 0.055; the CFI was 0.968 > 0.96; the TLI was 0.967 > 0.96; and the WRMR was 1.194, close to 1. Hence, with the support from these criteria, we have elements accept the hypothetical model presented previously to measure tutor performance in promoting self-directed learning. This indicates the confirmation, based on the data analyzed, that the hypothetical model given is an acceptable representation to measure tutor performance in promoting self-directed learning.

All the indicators shown in (Table 1) were statistically significant (p-value <0.001), as well as variances, covariances, and thresholds; although these last results are not shown. In addition, it is important to indicate that all the correlations among the latent factors are greater than 0.64, which shows that the factors are highly correlated (F2-F1 correlation equal to 0.755; F3-F1 correlation equal to 0.643; F3-F2 correlation equal to 0.828; F4-F1 correlation equal to 0.715; F4-F2 correlation equal to 0.813; and F4-F3 correlation equal to 0.771).

Table 1.

Given the satisfactory fit of the hypothetical model proposed, the instrument proposed can help to measure six areas that lead to the development of certain skills that, as a whole, Dauna Crooks, Ola Lunky-Child, Chris Patterson and Jeanette LeGris have suggested help students to be formed as self-directed learners². Hence, the tutor promoting development of these skills in consistent and harmonic manner, will be facilitating the path traveled by students to self-direct their learning. In PBL, it is crucial to constantly and progressively monitor how the tutors commit to the development of said skills.

DISCUSSION

Evaluation of tutor performance in promoting self-directed learning is of great importance for higher education institutions because it helps to make decisions on how the faculty staff must perform their role as enablers in the formation of students who seek greater autonomy and control in their learning processes. However, finding the adequate method to guarantee that the evaluation serves to make decisions and, thus, maintain, improve, or definitely change what we do or how we do it is not simple. Until now, results of learning are the most effective way to assess the professor's work. However, in PBL it is important to know how we manage to obtain learning and, herein, students' opinions are fundamental on how professors influence in the development of certain skills that, as a whole, lead to developing the skill of self-directed learning. It is so because forming self-directed learners is the biggest and best contribution of PBL and not merely the amount of skills developed within the framework of the discipline in question.

Due to this, this work has introduced a measurement instrument that according to CFA is acceptable to measure tutor performance in promoting self-directed learning. This instrument is composed of six factors (F1 = self-assessment of learning gaps within the United Nations specific context, F2 = self-assessment, F3 = reflexion, F4 = critical thinking, F5 = administration of information, F6 = group skills). The validated instrument has 10 indicators per factor and each indicator and the response options of each indicator are in a five-point Likert scale. The instrument permits obtaining the opinion of students on how much the professor contributes for them to develop each of the 60 skills described in the scale. Lastly, the results could report if professors are placing more emphasis in some areas than in other areas they should address during the PBL process, or if definitely their actions are removed from the premises of PBL, information that will be useful for school management in decision making on the direction of teaching as a whole. However, it is worth mentioning that the results found here are only valid for the students from the Faculty of Nursing at Universidad de Colima in Mexico, given that the sample only represented this faculty. The estimation method used was WLSMV because the instrument proposed contains indicators with ordinal scale. The function used was the probit function rather than the logit function, although both functions produce quite similar results. The correlations found among the factors indicated the existence of a strong relationship among the factors.

Lastly, it is important to indicate that further research is required to generalize these results to all the students at Universidad de Colima and to Mexican universities. For this reason, using the instrument proposed must be done with caution. Particularly, additional proof is needed to replicate the findings in other places. It would also be quite interesting to verify the invariance of measurement among genders and faculties at Universidad de Colima.

REFERENCES

1. Piaget J, García R. Psicogénesis e historia de la Ciencia. Undécima edición. México: Siglo XXI Editores; 2008.         [ Links ]

2. Rideout E. Transforming nursing education through problem-based learning. Hamilton, Ontario, Canada: Jones and Bartlett Publishers, 2001; pp. 345.         [ Links ]

3. Knowles M. Self-directed learning. A guide for learners and teachers. New York: Cambridge Adult Education, Globe Fearon; 1975.         [ Links ]

4. Amador G, Chávez AM, Alcaraz N, Moy NA, Guzmán J, Tene C.E. El papel de los tutores en la autodirección del aprendizaje de los estudiantes de enfermería. Rev. Investigación y Educación de Enfermería. 2007; 15 (3): 52-59.         [ Links ]

5. Boelen PA, van den Hout MA, van den Bout J. The factor structure of posttraumatic stress disorder symptoms among bereaved individuals: a confirmatory factor analysis study. J Anxiety Disord. 2008; 22:1377-1383.         [ Links ]

6. McArdle JJ. Current directions in structural factor analysis. Curr Dir Psychol Sci. 1996; 5:11-18.         [ Links ]

7. Atkinson TM, Rosenfeld BD, Sit L, Mendoza TR, Fruscione M, Lavene D, & Basch E. Using confirmatory factor analysis to evaluate construct validity of the Brief Pain Inventory (BPI). Journal of pain and symptom management. 2011; 41(3): 558-565.         [ Links ]

8. Kline RB. Principles and practice structural equation modeling. New York, NY: The Guilford Press; 2011.         [ Links ]

9. Hu & Bentler. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives, Structural Equation Modeling. 1999; 6(1): 1-55.         [ Links ]

10. Yu CY, Muthen B. Evaluation of model fit indices for latent variable models with categorical and continuous outcomes (Technical report). Los Angeles, CA: University of California at Los Angeles, Graduate School of Education & Information Studies; 2002.         [ Links ]

11. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen K A, Long JS. (Eds.) Testing Structural Equation Models. Beverly Hills, CA: Sage; 1993.         [ Links ] 12. Muthén LK, Muthén BO. Mplus User's Guide. Sixth Edition. Los Angeles, CA: Muthén & Muthén; 2007.         [ Links ]