SciELO - Scientific Electronic Library Online

 
vol.21 número2Ethnocentrism of the Colombian and Brazilian market and its relationship with the attitudes towards intention to purchase foreign and domestic productsPercepción de conflicto intergrupal y su relación con la adhesión a creencias esencialistas índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google

Compartir


Suma Psicológica

versión impresa ISSN 0121-4381

Suma Psicol. vol.21 no.2 Bogotá jul./dic. 2014

 

http://dx.doi.org/10.1016/S0121-4381(14)70013-X?

Structural Equations Model (SEM) of a questionnaire on the evaluation of intercultural secondary education classrooms

Modelo de ecuaciones estructurales (SEM) de un cuestionario sobre la evaluación de clases interculturales de educación secundaria

Eva María Olmedo Moreno*, Emilio Berrocal de Luna, María del Carmen Olmos Gómez and Jorge Expósito López

University of Granada, Spain

*Corresponding author. E-mail address: emolmedo@ugr.es (E.M. Olmedo Moreno).

Historia del artículo:
Recibido el 7 de marzo de 2014 Aceptado el 2 de julio de 2014


Abstract

This research includes the design of a questionnaire for evaluating cultural coexistence in secondary education classrooms (Berrocal, Olmedo & Olmos, 2014; Olmedo et al., 2014), as well as the comparison of its psychometric properties in a multicultural population of schools in southern Spain. An attempt is made to create a valid, reliable and useful tool for teachers to measure conflict situations in the classroom, as well as understanding the nature of the conflict from the point of view of all those involved. The metric aspects show a maximized content and construct validity (Muñiz, 2010) using a Structural Equation Model (SEM) and Confirmatory Factor Analysis (CFA) analysis, checking and modifying its model by Wald and Lagrange indicators (Bentler, 2007), to obtain the most adjusted model to the theoretical and goodness criteria.

Keywords: Structural, Equation, Model, Confirmatory factor analysis, School.


Resumen

Esta investigación incluye el diseño de un cuestionario para la evaluación de la coexistencia en las clases de educación secundaria (Berrocal, Olmedo & Olmos, 2014; Olmedo et al., 2014) y la comparación de sus propiedades psicométricas en una población multicultural de centros educativos en el sur de España. Se busca crear un instrumento válido, fiable como instrumento, útil al profesorado para evaluar las situaciones de conflicto en el aula, pero entendiendo la naturaleza del conflicto desde el punto de vista de aquellos que están involucrados en él. Los aspectos métricos muestran una alta validez de contenido y constructo (Muñiz, 2010) empleando un modelo de ecuaciones estructurales (SEM) y un análisis factorial confirmatorio (CFA), para comprobar y modificar su modelo de acuerdo con los indicadores de Wald y Lagrange (Bentler, 2007), para obtener el modelo más ajustado a la teoría y los criterios de bondad.

Palabras clave: Modelos de ecuaciones estructurales, Análisis factorial confirmatorio, Escuela.


It is very common to find conflicts in the current school classrooms; it is actually quite normal for schools' staff to have difficulties in its proper management. These situations lead to behaviours and attitudes which are negative for intercultural coexistence. School life is made up of experiences, values and relations and could be understood as a collective and dynamic construction of human interactions in school communities (Kriesberg, 1998; Bakerman & Maoz, 2005). Therefore, these experiences and values should emerge from an agreement about minimum principles such as respect, understanding, justice or freedom, and made all them from diversity.

This coexistence perspective focused on schools with a wide and diverse cultural contexts, usually determined by geography locations next to border zones, should be understood as "the conditions that serve as the fundamental prerequisites for the evolvement of advanced harmonious intergroup relations […], the very recognition in the right of the other group to exist peacefully with its differences, and to the acceptance of the other group as a legitimate and an equal partner with whom disagreements have to be resolved in non-violent ways" (Bar-Tal, 2004; p. 256).

Strategies for intercultural adaptation have been revelled as the best ways to work conflicts out and improve the schools coexistence (Crawshaw, 2002; Jain, 2012; Lima & Castro, 2005; Tomé et al., 2010). However, a new approach is required, based on teachers and students' cultural models and demands, to supply intercultural adaptation strategies and empower those which contribute to a positive life improvement (Olmedo, Berrocal & Olmos, 2014).

It is interesting to know the actions which are most widely taken to improve coexistence at school in a specific geographical area; likewise, it is important to analyse which of them have an influence on relevant variables to perform and develop students' motivation, group cohesion level or frequency of violent incidents, among others. It is not enough to study only the actions written on school planifications or work plans in order to analyse these aspects. It is necessary to have adequate incorporation of information about the educational community' opinion in relation to what is being actually done inside the classroom, especially from the point of view of students as real witnesses of these incidents.

This paper presents outcomes for validation of this questionnaire about assessment of coexistence in Secondary classrooms. It have been developed and supported on a National Research Project (I+D+I: Assessment and Development of Intercultural Adaptation Strategies to Improve Coexistence in Secondary Education Schools/EDU2010-22130) funded by the Ministry of Education and Science of the Spanish Government.

Starting the making process from a wide literature review of studies focused on methodologies or instrument to collect data about coexistence, such as references by Gutiérrez (2002, 2007 & 2008) focused on The Third Space development (Expósito, Olmedo, Pegalajar & Tomé, 2014) on educational interventions to resolve classroom conflicts (Moreno Olmedilla, 1998), about social behaviour (Armas, 2007), proposals and strategies to prevent bad behaviours (Ortega, 1999), who offers a vision of what school coexistence is and how it might be addressed.

This revision shows questionnaires as the most widely used instrument to collect data about coexistence opinions or information at mandatory secondary schools (Berrocal, Olmedo & Olmos, 2014; Díaz-Aguado, 2010; Olmedo et al., 2014), based on its efficiency and validity. Also, it is an inexpensive instrument, immediately applicable and that requires a few resources and easily procedures in comparison with other techniques. However, test or questionnaire scores may contain mistakes by multiple reasons that have to be minimized through adequate procedures (Muñiz, 2010). The underlying theoretical background for the process of elaboration of this questionnaire process takes the best theoretical and metric principles of reliability and validity in order to maximize inferences reached from its use.

The main aim of this paper is to build a questionnaire and calculate its reliability and validity through an analysis based on Structural Equation Modelling (SEM) within a Confirmatory Factor Analysis (CFA). The afore mentioned elements enable to contrast the model constructed a priori, establishing the whole set of relations between elements that form it, to give greater flexibility to the researcher in establishing a hypothesis about the construct structure (Bentler, 2007). Therefore, it is a kind of inferential analysis that uses multivariate regression to relate patterns of responses to a set of latent factors not directly observed, but according to the Substantive Theory (Rizopoulos, 2006) that exist in continuous dimensions of the people evaluated. This Structural Equation Model provides some technical procedures and criteria for validation of measurement models under two conditions (González-Montesinos & Backhoff, 2010):

  • Conditional independence, understood as a set of latent factors. For instance, skills, attitudes or perceptions, that influences a group of observed variables measured by the questions that make up a scale. The answers are mutually independent, but conditioned by the latent variable which determines them.
  • Latent factors can be quantified by a dimensional structure based on a Substantive Theory that postulates the existence of psychological constructs that perform a causal influence on people's responses to a reactive group.

Method

Participants

This study included a sample of 20 selected multicultural schools in the south of Spain, with at least 15% of immigrant students. It is one of the areas with greatest flow of immigration, given the 450 km border between two continents with different ethnicities; Africa and Europe. The procedure included a sample selection by cluster process to select groups/classes with a minimum of 15% of immigrant students. We collected data from the whole amount of 767 students. Of the students, 12.31% were immigrants and 87.68% were autochthonous. These students were 393 females (51.2%) and 374 males (48.8%), and all of them were aged between 11 and 16 years.

Instruments

The instrument for collecting data is Questionnaire for Assessment Coexistence shared experiences in Intercultural Secondary Classrooms (QACISC) for students (Olmedo et al., 2014; Berrocal, Olmedo & Olmos, 2013). It consists on 48 items grouped in six blocks. The first 10-item set is intended for social and demographic identification and the rest of them is a Lickert scale. Scores are encrypted from 1 to 4 (1, "nothing important"; 2, "not important"; 3, "important", and 4, "very important") to prevent intermediate values being positive or negative scores on each item.

Results

Items analysis

Table 1 shows all response categories chosen by percentage of students and their standard deviations, all them higher than .87 as a sign of discrimination items. The corrected item-total correlation (ri-t) is positive on all items, with values between .340 and .724. This indicates that all contribute to gauge what the test measures and further in the same direction.

The scores on skewness/kurtosis indices and the Kolmogorov-Smirnov, of the sample evaluated in each of the statements in the Questionnaire, does not fit a normal distribution (table 2). All items, except 1, 2, 7, 9, 10, 11, 12, 21, 22 and 23, show negative skewness, which means that students tend to score higher values on the scale. Regarding kurtosis indices are negative for all items except: the 14, 25, 26, 27, 30, 31, 32 and 38, indicating that scores cluster are below the normal distribution curve.

Exploratory factorial analysis

The exploratory factorial analysis is made by an extraction method of calculating the Kaiser-Meyer-Olkin index and Bartlett's test of sphericity (KMO=0.959; x2=703 for 18 578.691 d.f.; P=.000). The result of this initial analysis suggested the existence of five factors explaining 63.283% of total variance; the first one explains the majority of the whole variability.

The factorial construct validity of the questionnaire, among the various methods of orthogonal (varimax, quartimax and equamax) and oblique rotation, is verified through an equamax method that simultaneously simplifies and changes factors. The criterion proposed to determine the required correlation with the variable factor is Comrey (1985) 0.3. It indicates that any variables below this value are less than 10% of their variance in common with the factor, and those which are above are considered to have a weight factor in the component. All this information is shown on table 3 with communalities of all the items above .30, grouped in five factors.

The fourth factor (6/837% of the total variance) refers to behaviour towards classwork, and includes items such as p16/ Forgetting homework, p17/ Avoiding work in class, p18/ Not paying attention to the explanations of the teacher, p19/ Talking while the teacher explains, p20/ Forgetting materials, notebooks, p21/ Putting things away too early, p22/ Making insulting comments about the task, p23/ Getting up without permission.

The fifth factor (4.809% of the total variance) refers to behaviour related to school absenteeism and dropout, including items such us p1/ Being late to class, at the entrance, between classes or after breaks, p2/ Ask to be let go to the toilet continuously, or p12/ Leaving the classroom on class changes.

The first factor (25.274% of the total variance) refers to general, psychological and physical violence, and includes (see table 4) items such as p13/ Poor hygiene, p14/ Skipping classes, p15/Frequent truancy, p24/ Systematically disobeying rules and instructions, p25/ Disrespect to the teacher, p26/ Confronting teacher when he/she reproaches, p27/ Threatening the teacher, or p31/ Fighting with a partner (pushing, hitting, threats, assaults).

TABLE 4

The second factor (16.007% of the total variance) refers to structural violence and refers to items such as p3/ Noises and shouts in class, p4/ Damage of equipment, p5/ Throwing things in the classroom, p6/ Damage of school supplies, p7/ Eating candy or chew gum, p8/ Listening to music through headphones or mobile devices, p9/ Disorganizing furniture, p10/ Playing in the classroom, or p11/ Lack of order when entering or exiting the class.

The third factor (11.466% of the total variance) refers to problems with insubordination and vandalism towards the teacher and classmates, grouping items related to p28/ Interrupting the teacher with inappropriate questions, jokes and insulting comments; p29/ Ignoring the teacher when he/she is speaking; p30/ Insulting teachers; p32/ Disrespecting classmates; p33/ Taking away, hiding, spoiling, playing with classmates' materials; p34/ Arguing with peers in class; p35/ Showing sexist behavior; p36/ Hampering peers' work; p37/ Talking without being given the floor, and p38/ Xenophobic behavior.

These five factors are in accordance with the dimensions defined by Torrego et al. (2006), describing different behaviours of students in the classroom that can generate conflict. The internal consistency, reliability and homogeneity, denotes acceptable outcomes. For instance, the overall Cronbach's alpha coefficient is .717 for the whole sample, obtaining higher scores on some factors (F1, α=.934; F2, α=.895; F3, α=.927; F4, α=.877) and lower in F5 (α=.5221).

Confirmatory factor analysis to verify models by structural equation models

The confirmatory factor analysis aims to improve the questionnaire through its three structural model configutarion, based on a criterion of maximum likelihood according to the multivariate normality criteria of items. Each of these models will be judged overall by a set of indices for adjustment (Jöreskog & Sörbom, 1984). Modifications of the theoretical model a(M1) are generated from analysis of variation of parameter estimates, residual values and modification tests.

  • Model (M1): This is a model which originates in exploratory factor analysis as a theoretical model. Parsimonious adjustment PNFI is close to 1 (0.731), and comparative setting indexes CFI, TLI and NFI are 0.809, 0.796 and 0.783, respectively. Although these are good scores, it is necessary to study some other indexes. The most interesting one is the root mean square error of approximation (RMSEA), which is slightly above the critical limit: 0.085. Therefore, the approach has been fitted to the adjusted model.
  • Model (M2): This second model arises from the modification of the first model (figure 1) removing inappropriate items or items pruning (Batista-Foguetet et al., 2004). It preserves 25 items and four of five factors from M1: general, psychological, physical, and structural violence; problems with insubordination and vandalism towards the teacher and classmates, and behaviour towards classwork. It is interesting to realize the level of RMSEA and setting indexes CFI, TLI, NFI as well as that of PNFI.
  • Model (M3): The lack of setting verified by x2, requires the use of Wald and Lagrange modification indexes (Bentler, 2007) to implement the contrast of the Lagrange multipliers on parameters absent on the model, with an approximate value of their estimates: Lagrange test suggests the introduction of new relations in the model from a new second order factor called coexistence factor, which subsumes the fourth factor. The Wald test in turn suggests eliminating four items and applying relationships between certain latent errors. All that provides a setting model to theory, goodness fitted and set of values CFI RMSEA of 0.940 and 0.073 (table 5 and figure 2).

FIGURE 1

FIGURE 2

TABLE 5

Reliability

The reliability of the questionnaire was determined by verifying the internal consistency by α=.954. This index increases with the removal of any item.

Discussion

The theoretical background of this research work by Olmedilla (1998), Ortega (1999), Torrego (2000), Torrego and Moreno (2003), Gutiérrez (2007, 2008), Santos Guerra (2003), or Weapons (2007), provide a model about what coexistence should be like in schools and how it could be addressed. The questionnaire, which was designed with capabilities for its contextual adaptation, facilitates information and helps to improve coexistence and reduce the frequency of violent incidents. But a questionnaire might only be an important tool for scientific research in the field of education, if it is tested on its psychometrical characteristics in order to optimize the extent of interest variables.

The analysis of the items shows a positive total item right correlation (ri-t) in all items, with values between .340 and .724. The exploratory factor is performed by the extraction method, having previously used the fit of goodness and AFE indicators. There are five factors which explain 63.283% of the total variance, with an Equamax orthogonal rotation and a limit on correlation degree between the variable and the factor proposed by Comrey (1985) of 0.3.

The confirmatory factorial analysis (CFA) is performed through a progressive verification of the three structural equation models, globally judged by a set of fit indexes (Jöreskog & Sörbom, 1984). M1 and M2 have a parsimonious index with PNFI close to 1. M3, which is more complex and hierarchical, is considered a more appropriate setting to the theory on good scores of goodness RMSEA=0.073 and CFI=0.940 (Byrne, 2006; Arbuckle, 1997; Saris & Stronkhorst, 1984; Long, 1990). These structural equations allow assessment of each item and establish workable modifications, sequentially, until arriving at M3, in which there is a second order factor called the coexistence factor, and four factors of the first order: general, psychological, physical and structural violence; problems with insubordination and vandalism to teacher and classmates; bad behaviours while in class; absenteeism and dropouts. These data arise from a sample of students from schools with high levels of immigration, calculating their correlations and specifying a model with exploratory factorial analysis (EFA), thus having been thoroughly demonstrated as to ensure its validity.

The internal consistency, estimated by the Cronbach's alpha index, is considered adequate for each one of factors. However, it should be taken in to account that model modification has been developed on data only from a specific sample. Therefore, capitalization on chance (McCallum et al., 1992) by cross-validation should be studied to extend outcomes beyond the population.


References

Arbuckle, J.L. (1997). Amos users' guide. (Version 3.6.). Chicago, IL: SmallWaters Corporation.         [ Links ]

Armas Castro, M.(2007). Prevención e integración ante problemas de conducta. Estrategias para centros y familias. Madrid, España: Gráfias Muriel.         [ Links ]

Bar-Tal, D. (2004). Nature, rationale and effectiveness of education for coexistence. Journal of Social Issues, 60, 253-271.         [ Links ]

Batista Foguet, J.M., Coenders, G., & Alonso, J. (2004). Análisis factorial confirmatorio. Su utilidad en la validación de cuestionarios relacionados con la salud. Medicina Clínica, 122, 21-7.         [ Links ]

Bekerman, Z., & Maoz, I. (2005).Troubles with identity: obstacles to coexistence education in conflict ridden societies. Identity: An International Journal of Theory and Research, 5, 341-357.         [ Links ]

Bentler, P.M. (2007). Can scientifically useful hypotheses be tested with correlations? American Psychologist, 62, 772-782. doi: 10.1037/0003-066X.62.8.772.         [ Links ]

Berrocal, E., Olmedo, E., & Olmos, M.C. (2014). Validation of an evaluation tool for shared experience in intercultural secondary classrooms through a Structural Equation Model. Procedia - Social and Behavioral Sciences, 114, 244-256. doi: 10.1016/j.sbspro.2013.12.693.         [ Links ]

Byrne, B.M. (2006). Structural Equation Modeling with EQS: Basic concepts, applications, and programming (Multivariate Applications). Beverly Hills, CA: Sage Publications.         [ Links ]

Comrey, A. (1985). Manual de análisis factorial. Madrid, España: Cátedra.         [ Links ]

Expósito, J., Olmedo, E., Pegalajar, M., & Tomé, M. (2014). Pseudoconflicts in intercultural ESO (Mandatory Secondary Education) classroom from the point of view of students and teachers. Procedia - Social and Behavioural Sciences, 132, 100-106. doi: 10.1016/j. sbspro.2014.04.284.         [ Links ]

Ferrer, B.M., Ochoa, G.M., Muñoz, L.V.A., & Gimeno, M.C.M. (2012). Estatus sociométrico y violencia escolar en adolescentes: implicaciones de la autoestima, la familia y la escuela. Revista Latinoameriacana de Psicología, 44, 55-66. doi: 10.14349/rlp.v44i2.1031.         [ Links ]

González-Montesinos, M.J., & Backhoff, E. (2010). Validación de un cuestionario de contexto para evaluar sistemas educativos con Modelos de Ecuaciones Estructurales. Revista Electrónica de Investigación y Evaluación Educativa, 16, 1-17.         [ Links ]

Gutiérrez, K., & Larson, J. (2007). Discussing expanded spaces for learning. Language Arts, 85, 69-77.         [ Links ]

Gutiérrez, K. (2008). Developing a sociocritical literacy in the third space. Reading Research Quarterly, 43, 148-164. doi: 10.1598/RRQ.43.2.3.         [ Links ]

Jain, U. (2012). Cultural construction of environmental problems. Procedia - Social and Behavioural Sciences, 68, 6-15. doi: 10.1016/j. sbspro.2012.12.202.         [ Links ]

Jöreskog, K.G., & Sörbom, D. (1984). LISREL VI. Analysis of linear structural relationships by maximum likelihood, instrumental variables, and least squares methods. Mooresville, Indiana: Scientific Software.         [ Links ]

Lima, M.L., & Castro, P. (2005). Cultural theory meets the community: Worldviews and local issues. Journal of Environmental Psychology, 25, 23-25. doi: 10.1016/j.jenvp.2004.11.004.         [ Links ]

Long, J.S. (1990). Covariance Structure Models: An Introduction to LISREL. Sage University Paper Series on Quantitative Applications in the Social Sciences, 007-034. Newbury Park, CA: Sage.         [ Links ]

Mc Callum, R.C., Roznowski, M., & Necowitz, L.B. (1992). Model modification in covariance structure analysis: the problem of capitalization on chance. Psychological Bulletin, 111, 490-504. doi: 10.1037/0033-2909.111.3.490.         [ Links ]

Moreno Olmedilla, J.M. (1998). Comportamiento antisocial en los centros escolares: una visión desde Europa. Revista Iberoamericana de Educación, 18, 189-206.         [ Links ]

Muñiz, J. (2010). Las teorías de los tests: Teoría Clásica y Teoría de Respuesta a los Ítems. Papeles del Psicólogo, 31, 57-66.         [ Links ]

Olmedo, E., Aguaded, E., Berrocal, E., Buendía, L., Expósito, J., Sánchez, Ch., & Carmona, M. (2014). Constructing an instrument for evaluating group relations in intercultural secondary school classes. The International Journal of Assessment and Evaluation, 21, 11-21.         [ Links ]

Ortega, R. (1999). La convivencia escolar: qué es y cómo abordarla. Programa educativo de prevención del maltrato entre compañeros y compañeras. Sevilla, España: Consejería de Educación, Junta de Andalucía.         [ Links ]

Rizopoulos, D. (2006).An R Package for Latent Trait Modeling and Item Response Theory Analyses. Journal of Statistical Software, 17, 1-25.         [ Links ]

Santos Guerra, M.A. (2003). Aprender a vivir en la escuela. Madrid: Editorial Akal.         [ Links ]

Saris, W.E., & Stronkhorst, L.H. (1984). Causal Modelling in Non-Experimental Research. Amsterdam: Sociometric Research.         [ Links ]

Tomé, M., Berrocal, E., & Buendía, L. (2010). Contenidos fundamentales para la formación del ciudadano intercultural. Propuestas del alumnado tras el primer curso de educación para la ciudadanía: Un estudio exploratorio. Revista de Investigación Educativa, 28, 425-444.         [ Links ]

Torrego, J.C., & Moreno, J.M.(2003). Convivencia y disciplina en la escuela. El aprendizaje de la democracia. Madrid, España: Alianza Editorial.         [ Links ]

Torrego, J.C.(2000). Mediación de conflictos en instituciones educativas. Madrid, España: Narcea.         [ Links ]