SciELO - Scientific Electronic Library Online

 
vol.51 número1Validation of the Grit Scale and the Treatment Self-Regulation Questionnaire (TSRQ) in the Mexican context índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Artigo

Indicadores

Links relacionados

  • Em processo de indexaçãoCitado por Google
  • Não possue artigos similaresSimilares em SciELO
  • Em processo de indexaçãoSimilares em Google

Compartilhar


Revista Latinoamericana de Psicología

versão impressa ISSN 0120-0534

rev.latinoam.psicol. vol.51 no.1 Bogotá jan./jun. 2019

http://dx.doi.org/10.14349/rlp.2019.v51.n1.1 

Artículos

Psychometric properties of the Brief Symptom Inventory-18 in a heterogeneous sample of adult cancer patients

Propiedades psicométricas del Inventario Breve de Síntomas-18 en una muestra heterogénea de pacientes de cáncer adulto

Paula Martínez-Lópeza  *  , Andrea Conchado-Peirób  , Yolanda Andreu-Vailloc  , María José Galdón-Garridoc 

a Area of Health Sciences. Valencian International University, Valencia, Spain.

b Department of Statistics, Applied Operations Research and Quality, Polytechnic University of Valencia, Valencia, Spain.

c Department of Personality, Evaluation and Psychological Treatment. University of Valencia, Valencia, Spain.

Abstract

The Brief Symptom Inventory-18 (BSI-18) is a self-reporting screening instrument that is widely used to assess global psychological distress and three kinds of symptoms: anxiety, depression, and somatization. The present study tests the factor structure of the BSI-18 using confirmatory factor analysis (CFA), its reliability, convergent validity, and invariance for both sexes. A heterogeneous sample of 1183 cancer patients completed the BSI 18 and the NCCN Problem List. Hierarchical models of three and four subscales with GSI as a main factor provided an adequate and similar model fit. Nonetheless, the hierarchical three-factor model (the theoretical proposal) was selected for methodological and theoretical reasons. Reliability indexes (Cronbach's alpha and Composite Reliability) were satisfactory. The positive significant associations between BSI-18 (GSI and subscales) and emotional and physical categories of the Problem List showed the suitable convergent validity of the instrument. Finally, multigroup CFA revealed an essentially invariant structure of the BSI-18 for both sexes. The BSI-18 is a short instrument that can be used by researchers and health professionals to assess the psychological distress of cancer survivors.

Key words: Brief Symptom Inventory-18; cancer; emotional distress; psychometric properties; assessment

Resumen

El Inventario Breve de Síntomas-18 (BSI-18) es un instrumento de cribado ampliamente utilizado para evaluar distrés emocional (GSI) y tres tipos de síntomas: ansiedad, depresión, y somatización. Este trabajo estudia la estructura factorial del BSI-18, utilizando análisis factorial confirmatorio (AFC), su fiabilidad y validez convergente, así como su invarianza factorial a través del sexo. Una muestra de 1183 pacientes de cáncer completó el BSI 18 y la lista de problemas de la NCCN. Los modelos jerárquicos de tres y cuatro factores proporcionaron ajustes adecuados y similares. Sin embargo, el modelo de tres factores (propuesta teórica) fue seleccionado por razones metodológicas y teóricas. Los índices de fiabilidad (alfa de Cronbach y fiabilidad compuesta) fueron satisfactorios y las correlaciones positivas y significativas entre el BSI-18 (GSI y subescalas) y las categorías emocionales y físicas del listado de problemas evidenciaron la adecuada validez convergente del instrumento. Finalmente, el AFC multigrupo reveló una estructura básicamente invariante del BSI-18 a través del sexo. El BSI-18 es un instrumento breve que puede ser utilizado por investigadores y profesionales de la salud para evaluar el malestar psicológico en la población con cáncer.

Palabras-clave: Brief Symptom Inventory-18; cáncer; distrés emocional; propiedades psicométricas; evaluación

Research has shown that people with any type of cancer frequently experience levels of psychological distress during the course of their illness, diagnosis, and treatment. The prevalence of mood disorders is around 38% (the range is from 28% to 49%) (Grassi, Caruso, Mitchell, Sabato, & Nanni, 2018; Mitchell et al., 2011). In the last decade, psychological distress has become the sixth vital sign in cancer care and, consequently, numerous international organizations and professional societies consider routine screening for it to be a standard requirement, for example the Joint Action on Cancer Control (CanCON, 2017), the International Psycho-Oncology Society (IPOS) (Watson & Bultz, 2010), and the National Comprehensive Cancer Network (NCCN, 2017). In this context, selecting an adequate assessment tool is crucial for the effectiveness of screening programs. To date, many instruments have been developed to measure emotional distress in cancer patients (Mitchell, 2010; Vodermaier, Linden & Siu, 2009). The Brief Symptom Inventory-18 (BSI-18) (Derogatis, 2001) is one of the most common checklists used in this context, not only to screen for psychological distress but also as a criterion measure to validate other measures of distress. The BSI-18 is considered a "short-tool" because it consists of only 18 items (Vodermaier et al., 2009). It was designed to assess three clinically relevant psychological symptoms (anxiety, depression, and somatization), and to provide an indicator for the overall level of psychological distress, which is known as the Global Severity Index (GSI). Furthermore, the BSI-18 includes normative data of cancer populations and allows "casesness" to be identified based on a criterion proposed by Derogatis (2001).

The psychometric properties of the BSI-18 have been widely examined in the general population (e.g., Franke et al., 2017) as well as for different subgroups: Latinos (Torres, Miller, & Moore, 2013), women (Wiesner et al., 2010), home-bound older adults (Petkus et al., 2010), drug users (Wang et al., 2010), psychiatric outpatients (Andreu et al., 2008), temporomandibular disorder patients (Durá et al., 2006), and Parkinson disease patients (Abraham, Gruber-Baldini, Harrington, & Shulman, 2017), using mainly the English version of the instrument (e.g., Franke et al., 2017; Wiesner et al., 2010) followed by the Spanish version (e.g., Andreu et al., 2008; Durá et al., 2006). Several studies have been conducted in cancer settings. Zabora et al. (2001) analysed the psychometric properties of BSI-18 in a large sample of patients who were actively undergoing cancer treatment. Principal components analysis revealed a four-factor structure; the first three factors correspond to somatization, depression, and anxiety scales (Derogatis' theoretical proposal); the fourth factor had only one significant item related to thoughts of ending one's life (item 17). Based on a sample of Spanish patients with breast cancer and using confirmatory factor analyses (CFA), Galdón et al. (2008) tested the fit of several models: a single-factor model measuring a single dimension of psychological distress, a three-factor model (the theoretical proposal), and a four-factor model (the empirical structure proposed by Derogatis (2001), based on community sample: somatization, depression, general anxiety, and panic). The results demonstrated that the improved three-factor model of the Spanish version of the BSI-18 was superior. Likewise, although the correlations between three factors reflected a possible second-order factor representing overall psychological distress, this model was not tested. Recklitis et al. conducted a series of studies among adult survivors of childhood cancer (Recklitis et al. 2006; Recklitis & Rodriguez, 2007; Merport & Recklitis, 2012). The factor structure of the BSI-18 has also been studied in a large and heterogeneous sample using both exploratory and confirmatory factor analysis (Recklitis et al. 2006). Exploratory factor analysis (EFA) supported the theoretical three-factor structure of the BSI-18. CFA was used to test the model's fit from the EFA and alternative models, including a single-factor model (general psychological distress), a four-factor model (somatization, depression, general anxiety, and panic) and hierarchical three- and four-factor models. Results showed that fit statistics for the two hierarchical models were similar. However, authors considered the hierarchical three-factor model to be preferable. Moreover, analysis of the three-factor model showed excellent reliability and consistent fit among both male and female participants; this supported the invariance of factor structure for both sexes. A subsequent study added data about the adequate convergent validity of the BSI-18 as a result of its high correlations with the corresponding Symptom Check-list-90-R subscales and highlighted the excellent internal consistency of the instrument (Recklitis & Rodriguez, 2007).

To date, the validity of evidence based on the internal structure of the BSI-18 in cancer settings has been investigated with oncology subgroups: specific cancer-sites such as breast cancer (Galdón et al., 2008) as well as during certain phases of the illness, for example when receiving medical treatment (Zabora et al., 2001), or as a survivor of childhood cancer (Recklitis et al. 2006). However, evidence of psychometric properties of the Spanish version of the BSI-18 based on heterogeneous samples of adult cancer patients are still not available, and this is crucial because its study would yield representative and generalizable estimates for the overall population. On the other hand, previous studies have used different statistical methods, for example EFA (Zabora et al., 2001) and CFA (Galdón et al., 2008; Recklitis et al. 2006), which have not always tested hierarchical structures that allow us to obtain a general score of distress.

At the same time, more advanced statistics such as factorial invariance also deserve special attention in this setting. Testing for invariance is necessary to ensure that reported mean level differences are true score differences rather than confounded by errors in measurement. When such testing is not performed, the potential for comparing groups on non-equivalent measures is increased, resulting in biased interpretations of the results (Brunet, McDonough, Hadd, Crocker, & Sabiston, 2010). About this, only the study with a sample of survivors of childhood cancer has addressed this topic; consequently, further research is necessary to properly understand the possible differences by sex in an adult cancer setting (Grassi et al., 2018).

Given this context, the goal of this study was to provide data that offers an integral vision of the psychometric properties of the BSI-18 in the adult cancer population whilst overcoming prior limitations. Specifically, the current study aims to (i) examine the factor structure of the BSI-18 in a large and heterogeneous sample of adult oncology patients to test the fit of a single-factor model as well as three- and four-factor models and their hierarchical forms; (ii) provide information on the convergent validity and the internal consistency of the instrument; and (iii) determine the factorial invariance of the BSI-18 for both men and women.

Method

Participants and Procedure

This study is part of a large research project on the detection and treatment of emotional distress in adult oncology patients at the Fundación Instituto Valenciano de Oncología (FIVO). The study was approved by the institution's Ethics Committee. Between May 2015 and July 2017, consecutive patients who visited the FIVO to receive care in any of its medical departments were invited to participate. Eligible patients were required to be over 18 years of age, be diagnosed with cancer, and be able to provide informed consent. Patients received information about the study, and those who were interested provided their informed consent and complete the questionnaire package. Of 1259 patients who were approached, 76 (6%) declined to participate. The final sample consisted of 1183 oncological patients (65% female). Participants' mean age was 57.71 years (SD=13.09; Range = 19-94 years). Most participants were living with a partner (73%) and had completed primary studies (87%). The most frequent primary cancer type was breast cancer (40%), followed by gynaecologic (15%), and prostate (12%) cancers; other localized primary cancer types included respiratory (7%), digestive (7%), melanoma (6%), and miscellaneous (9%) cancers. About 50% of participants were in a non-advanced stage of disease (stages I and II), 20% were in stage III, and 22% were in stage IV. Finally, 64% of participants were receiving medical treatment at the time of the study: surgery (10%), adjuvant treatment (47%), and hormonotherapy (7%).

Materials

Sociodemographic and medical data

A data sheet was used to obtain sociodemographic data including age, sex, marital status, education level, and employment status. Medical information including cancer type, stage of disease, phase of illness, and medical treatment at the time of the study was gathered by chart review.

Brief Symptom Inventory-18

The BSI-18 is a self-reported symptom checklist (Derogatis, 2001; Spanish version, Derogatis, 2013). Respondents rate a total of 18 items on a five-point Likert scale to reflect their distress during the previous week. The instrument provides three symptom scores: somatization (items 1,4,7,10,13,16), depression (items 2,5,8,11,14,17), and anxiety (items 3,6,9,12,15,18) as well as an overall score (GSI). Moreover, the BSI-18 allows cases of clinical distress to be identified using normative data proposed by Derogatis (2001): the scores are transformed into T-scores (T ≥ 63 in the GSI or in at least two subscales). The BSI-18 has shown adequate psychometric properties in studies of cancer patient populations (Vodermaier et al., 2009).

NCCN Problem List

The Problem List (NCCN, 2017) consists of 36 problems commonly experienced by oncology patients, which are grouped into five categories: practical (5 items: e.g. insurance/financial), family (3 items: e.g., dealing with partner), emotional (6 items: e.g., nervousness), physical (20 items: e.g., fatigue), and spiritual/religious (1 item: spiritual and religious concerns). Respondents indicate whether or not they have experienced any of the problems during the previous week. For the present study, the multi-item categories on the NCCN Problem List had adequate internal consistency.

Statistical Analysis

Data analysis was conducted in two stages. First, in the total sample, we tested the factor structure of five competitive models of the BSI-18 reported in the literature. As the assumption of multivariate normality was not fulfilled (Mardia, 1970), we tested measurement invariance of the selected model of the BSI-18 for both sexes using EQS 6.0 and robust normal theory maximum likelihood (MLR) estimation methods. To assess the goodness-of-fit for the models, we considered the Satorra-Bentler chi-squared statistic (X2) as well as other goodness-fit indexes, such as Comparative Fit Index (CFI; cutoff values close to 0.95), Root Mean Square Error of Approximation (RMSEA; cutoff values lower than 0.08), Non-Normed Fit Index (NNFI; cutoff values of 0.90), and Standardized Root Mean Square of the Residuals (SRMR; cutoff values of 0.08) as described by Hu and Bentler (1999) and Browne and Cudeck (1993). However, Hu and Bentler's (1999) guidelines are regarded as a conservative approach to the performance of fit statistics, which may vary depending on the complexity of the specified model (Beauducel & Wittman, 2005).

The evidence of multigroup invariance was based on ACFI and AX2 (Cheung & Renswold, 2002; Meade, Johnson, & Braddy (2008). Besides, partial invariance was eventually tested for some particular groups of constraints (Byrne, Shavelson, & Muthén, 1989). We estimated the configural, metric, scalar, and structural invariance of the final model of the BSI-18 for both sexes (Vandenberg & Lance, 2000). Significant latent mean differences and effect sizes were assessed using Cohen's d parameter according to Cohen (1988).

Reliability of the BSI-18 was measured using Cronbach's alpha (α) together with Composite reliability (CR) corrected by error covariances according to Brown (2015). Finally, convergent validity of the BSI-18 was assessed using a series of Pearson's correlation coefficient between the BSI-18 and the NCCN Problem List. Only two missing data were found for items 14 and 15. Pairwise deletion was used to deal with these data, which was assumed to be randomly missing.

Results

Factor structure

As shown in Table 1, five competing models were compared. All items loaded significantly on its corresponding factor. As expected, the one-factor model obtained the poorest fit to the data. Model fit for the three-factor and three-factor hierarchical models was identical as is to be expected in cases where fewer than four first-order factors are associated with a particular second-order factor (Thompson, 2004). Fit statistics for the four- and four-factor hierarchical models were very similar and slightly better than the three-factor models. In order to select the most parsimonious model consistent with the theoretical basis, the hierarchical three-factor model was eventually retained and improved, with three error covariances between items (5 & 14, 11 & 17, and 9 & 18). Post hoc model specifications were selected on the basis of modification indexes and a substantive rationale since the items belonged to the same subscale. The improved hierarchical three-factor model was retained for subsequent analysis as the best-fitting factor structure. This conclusion held for the total sample as well as for the sample groups based on sex (see Table 1).

Table 1 Model fit of preliminary models estimated for the total sample and sex-subgroups. 

a robust version

Multigroup factorial invariance

Multisample CFA results are presented in Table 2. Once reasonable evidence of configural invariance (Model 1) across sex was achieved, equality constraints on all freely estimated first-order factor loadings were added. The assumption of full factor loading invariance (Model 2a) was untenable for sex, as shown through a significant worsening of model fit compared with the baseline model.

Table 2 Testing for invariance across sex 

Thus, partial factor loading invariance was tested by relaxing the factor loading invariance assumption for items 6 and 15 from the anxiety subscale (Model 2b). Sex differences in the freely unstandardized estimated factor loadings were minor (item 6: 1.022 for women, 1.021 for men; item 15: 0.734 for women and men). Model 3b, which also allowed partial invariance in items 6 and 15 for sex, fit the data better than Models 2a and 3a, which had full invariance constraints in all factor loadings.

Subsequently, the difference between the latent factor means across the subgroups were estimated (Table 3). Model 4b obtained a good fit to the data, indicating that women obtained significantly higher scores than men in all subscales, regardless of items 6 and 15 from the anxiety subscale. The highest differences were found for anxiety, as indicated by Cohen's d, followed by the depression, and somatization subscales.

Table 3 Testing for latent factor mean differences across sex 

Reliability and convergent validity

As shown in the first column of Table 4, the values of Cronbach's alpha and Composite reliability for the three sub-scales of the BSI-18 and the GSI were satisfactory. Although discriminant validity could not be verified due to high second-order factor loadings, as shown in Figure 1 (λ-SOMA = 0.83, λ-DEPR = 0.91, λ-ANX = 0.95), and low values of Average Variance Extracted (AVE-SOMA =0.40, AVE-DEPR = 0.40, AVE-ANX = 0.27), goodness of fit indexes for the one-dimensional structure did not improve the excellent fit of the hierarchical three-factor structure, as exposed in Table 1.

Table 4 Reliability and convergent validity for improved hierarchical three-factors model. 

** p ≤ 0.01; *** p ≤ 0.001

Figure 1 Structural model of the BSI-18. 

The correlations between the BSI-18 (GSI and subscales) with the 5 categories of the NCCN Problem List are shown in Table 4. GSI correlated the highest with emotional problems followed by substantial correlations with physical, family, practical, and the lowest with spiritual/religious. The Anxiety and Depression subscales correlated quite similar to the previous ones while Somatization showed a higher correlation with physical than emotional problems.

Discussion

This research tested the factor structure and the in-variance of the Brief Symptom Inventory-18 (BSI-18) in a heterogeneous sample of more than 1000 cancer patients. The BSI-18 is considered a reference instrument to assess cancer patients' emotional distress: it is a suitable option to be used in routine screening due to its length (it only has 18 items), it can feasibly be used in clinical settings (it takes only a few minutes to complete), and has norms for an oncological population (both general and separate by sex). Results from this study based on the total sample and subgroups defined by sex confirmed the BSI-18 factor structure through the satisfactory data fit of two hierarchical models: the improved three-factor model (the theoretical proposal) and the four-factor model (the empirical solution reported by Derogatis (2001). Nevertheless, although both achieved satisfactory adjustment indexes, the improved hierarchical three-factor model was considered the best option in cancer setting. This model, in addition to being the most consistent with the intended design of the BSI-18, is more parsimonious and easily interpretable. The three-factor model coincides with the four-factor model in terms of item distribution in the GSI as well as in the depression and somatization dimensions, but it diverges in the anxiety dimension as it has a more simple distribution: one factor vs. two factors: panic and general anxiety. Therefore, the three-factor model provides information about general distress and psychological symptomatology such as anxiety, depression, and somatization. Our results coincide with those obtained by previous studies (using CFA) undertaken with breast cancer patients (Galdón et al., 2008) and survivors of childhood cancer (Recklitis et al., 2006).

Data from this study also suggest that there is partial invariance across sex after relaxing the invariance assumption in only two items. The findings, based on factor loadings and subsequent types of invariance, showed that the majority of items were interpreted equivalently between men and women, but two items belonging to the anxiety sub-scale ("feeling tense or keyed up" and "feeling restless / can't sit still") were not invariant. Apart from this slight discordance, the current findings revealed minimal differences across sex. This means that psychological response is measured similarly across the subgroups that were considered when using the BSI-18 hierarchical three-factor model. Consequently, an important implication of these findings is that comparisons of means, based on the considered demographic variable, are valid and may help professionals to identify which subgroups of cancer patients experience higher levels of distress. Moreover, we observed variations in mean levels of BSI-18 across sex: women showed higher scores - especially in anxiety, although they also scored higher in depression and somatization symptomatology. These results may be due to a possible tendency that men under-report psychological symptoms and, thus, appear to apparently experience a lower prevalence of mental disorders (Zabora et al., 2001). However, given that these findings are based on the partially invariant construct of anxiety, assessments about the size of the differences between sexes should be interpreted with caution.

The reliability of the three original subscales and the total score of the BSI-18 was satisfactory. The Cronbach's alpha values (α = 0.72-0.91) were good to very good and had similar ranges to other studies that included large samples of cancer patients carried out by Derogatis (2001), Zabora et al. (2001), Recklitis et al. (2006), and Recklitis and Rodríguez (2007). Likewise, the Composite Reliability (CR = 0.69-0.90), an index not previously used, also supports the internal consistency of the three subscales and global scale. Oppositely, weak discriminant validity was identified for first-order factors. Consistently, future research could test a bifactor model: an in-depth examination of the contribution each item makes to both general and specific factors.

Using the categories belonging to the NCCN Problem List (NCCN, 2017), the results of the study support the BSI-18's convergent validity due to the highest correlation of sub-scales and GSI being with the emotional and physical categories of problems on the NCCN Problem List. Furthermore, both the total score and anxiety and depression subscales have a higher correlation with the category of emotional problems than with physical problems; the opposite occurs for the somatization subscale. These results endorse those presented by Clover et al. (2016), which showed that Problem List items associated with emotion, physical function, and support were significantly associated with moderate or severe distress.

The current research has both strengths and limitations. It contributes to gaining a better understanding of BSI-18's psychometric properties, but does not address other important issues such as the controversy of the cut-off point for identifying clinical distress. In addition, we are aware that the nature of partial invariance analysis is exploratory, which may be a concern in psychometric work. However, this application of invariance evaluation procedures was conducted in the absence of explicit hypothesis testing. Given the large number of constrained parameters and the large sample size, it is also possible that parameters from the anxiety subscale differ by chance. In future research it would be productive to examine differences between sex in terms of the definition of the anxiety construct. On the other hand, this is the first study to systematically examine the factor structure of the BSI-18 in a large and heterogeneous adult oncology sample. The remarkable strength of the present study is that we tested a theoretical model and empirical proposals that have previously been studied, and we also confirmed the invariance of the BSI-18 across groups based on sex. Moreover, we analyzed the internal consistency of the BSI-18 using a classic index (Cronbach's alpha) as well as a more recently-created index (Composite reliability) and support its convergent validity. Therefore, this work is a further step to validating and analyzing the psychometric properties of the BSI-18 in a cancer setting. The direct implication of these findings is that the BSI-18 may be confidently used as a reliable and valid measure of emotional distress in oncology patients; also, meaningful comparisons among men and women are possible.

Acknowledgement

The authors would like to thank all the study participants and staff at the FIVO for their cooperation. This research was funded by the Ministry of Economy and Competitiveness, Government of Spain (PSI2013-45905-R). The authors have declared that no conflict of interest exists.

References

Abraham, D. S., Gruber-Baldini, A. L., Harrington, D., & Shulman, L. M. (2017). The factor structure of the Brief Symptom Inventory-18 (BSI-18 in Parkinson disease patients. Journal of Psychosomatic Research, 96, 21-26. http://dx.doi.org/10.1016/j.jpsychores.2017.03.002. [ Links ]

Andreu, Y., Galdón, M. J., Durá, E., Ferrando, M., Murgui, S., García, A., & Ibáñez, E. (2008). Psychometric properties of the Brief Symptoms Inventory-18 (BSI-18) in a Spanish sample of outpatients with psychiatric disorders. Psicothema, 20, 844-850. [ Links ]

Beauducel, A., & Wittmann, W. W. (2005). Simulation study on fit indexes in CFA based on data with slightly distorted simple structure. Structural Equation Modeling, 12(1), 41-75. [ Links ]

Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research. New York: Guilford Publications. [ Links ]

Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K.A. Bollen and J.S. Long (Eds.), Testing Structural Equation Models, (pp. 136-154). Newbury Park: Sage focus editions. [ Links ]

Brunet, J., McDonough, M. H., Hadd, V., Crocker, P., & Sabiston, C. (2010). Posttraumatic Growth Inventory: An examination of the factor structure and invariance among breast cancer survivors. Psycho-Oncology, 19, 830-838. http://dx.doi.org/10.1002/pon.1640. [ Links ]

Byrne, B. M., Shavelson, R. J., & Muthen, B. (1989) Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin, 105, 456-466. http://dx.doi.org/10.1037/0033-2909.105.3.456. [ Links ]

CanCON. (2017). European Guide on Quality Improvement in Comprehensive Cancer Control. Retrieved from https://cancercontrol.eu/archived/uploads/images/Guide/pdf/CanCon_Guide_FINAL_Web.pdf. [ Links ]

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. http://dx.doi.org/10.1207/S15328007SEM0902_5. [ Links ]

Clover, K.A., Oldmeadow, C., Nelson, L., Rogers, K., Mitchell, A.J., & Carter, G. (2016). Which items on the distress thermometer problem list are the most distressing? Supportive Care in Cancer, 24, 4549-4557. http://dx.doi.org/10.1007/s00520-016-3294-z. [ Links ]

Cohen, J. (1988). The effect size index: d. In J. Cohen (Ed.). Statistical power analysis for the behavioral sciences. (pp. 473-480). Hillsdale, N.J.: Lawrence Erlbaum Associates. [ Links ]

Derogatis, L. R. (2001). Brief Symptom Inventory 18. Minneapolis: National Computer Systems Pearson, Inc. [ Links ]

Derogatis, L. R. (2013). Inventario Breve de 18 Síntomas. Manual. Barcelona: Pearson Educación. [ Links ]

Durá, E., Andreu, Y., Galdón, M. J., Ferrando, M., Murgui, S., Poveda, R., & Jiménez, Y. (2006). Psychological assessment of temporomandibular disorders patients. Confirmatory analysis of the dimensional structure of the Brief Symptom Inventory 18 (BSI-18). Journal of Psychosomatic Research, 60, 365-70. http://dx.doi.org/10.1016/j.jpsychores.2005.10.013. [ Links ]

Franke, G. H., Jaeger, S., Glaesmer, H., Barmann, C., Petrowsky, K., & Braehler, E. (2017). Psychometric analysis of the brief symptom inventory 18 (BSI-18) in a representative German sample. BMC Medical Research Methodology, 17, 14. http://dx.doi.org/10.1186/s12874-016-0283-3. [ Links ]

Galdón, M. J., Durá, E., Andreu, Y., Ferrando, M., Murgui, S., Pérez, S., & Ibáñez E. (2008). Psychometric properties of the Brief Symptom Inventory-18 in a Spanish breast cancer sample. Journal of Psychosomatic Research, 65, 533-539. http://dx.doi.org/10.1016/j.jpsychores.2008.05.009. [ Links ]

Grassi, L., Caruso, R., Mitchell, A. J., Sabato, S., & Nanni, M. G. (2018). Screening for emotional disorders in patients with cancer using the Brief Symptom Inventory (BSI) and the BSI-18 versus a standardized psychiatric interview (the World Health Organization Composite International Diagnostic Interview). Cancer, 124(11), 2415-2426. http://dx.doi.org/10.1002/cncr.31340. [ Links ]

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1), 1-55. [ Links ]

Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3), 519-530. [ Links ]

Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568-592. [ Links ]

Merport, A., & Recklitis, C. J. (2012). Does the Brief Symptom Inventory-18 case rule apply in adult survivors of childhood cancer? Comparison with the Symptom Checklist-90. Journal of Pediatric Psychology, 37(6), 650-659. http://dx.doi.org/10.1093/jpepsy/jss050. [ Links ]

Mitchell, A. J. (2010). Short screening tools for cancer-related distress: a review and diagnostic validity meta-analysis. Journal of National Comprehensive Cancer Network, 8, 487-494. http://dx.doi.org/10.6004/jnccn.2010.0035. [ Links ]

Mitchell, A. J., Chan, M., Bhatti, H., Halton, M., Grassi, L., Johansen, C., & Meader, N. (2011). Prevalence of depression, anxiety, and adjustment disorder in oncological, haematological, and palliative-care settings: a meta-analysis of 94 interview-based studies, Lancet, 12, 160-174. http://dx.doi.org/10.1016/S1470-2045(11). [ Links ]

NCCN (2017). NCCN Clinical Practice Guidelines in Oncology: Distress Management V.2.2017. Retrieved from http://www.nccn.org/professionals/phisician_gls/PDF/distress.pdf. [ Links ]

Petkus, A. J., Gum, A. M., Small, B., Malcarne, V. L., Stein, M. B., & Wetherell, J. L. (2010). Evaluation of the factor structure and psychometric properties of the brief symptom inventory-18 with homebound older adults. International Journal of Geriatric Psychiatry, 25, 578-587. http://dx.doi.org/10.1002/gps.2377. [ Links ]

Recklitis, C. J., Parsons, S. K., Shih, M., Mertens, A., Robinson, L. L., & Zeltzer, L. (2006). Factor structure of Brief Symptom Inventory-18 in adult survivors of childhood cancer: results from the childhood cancer survivor study. Psychological Assessment, 18, 22-32. http://dx.doi.org/10.1037/1040-3590.18.1.22. [ Links ]

Recklitis, C. J., & Rodríguez, P. (2007). Screening childhood cancer survivors with the brief symptom inventory-18: Classification agreement with the symptom checklist-90-revised. Psychooncology, 16(5), 429-436. http://dx.doi.org/10.1002/pon.1069. [ Links ]

Thompson, B. (2004). Exploratory and confirmatory factor analysis. Washington, DC: American Psychological Association. [ Links ]

Torres, L., Miller, M., & Moore, K. M. (2013). Factorial invariance of the Brief Symptom Inventory-18 (BSI-18) for adults of Mexican descent across nativity status, language format, and gender. Psychological Assessment, 25, 300-305. http://dx.doi.org/10.1037/a0030436. [ Links ]

Vandenberg, R.J. and Lance, C.E. (2000) A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research. Organizational Research Methods, 3, 4-70. http://dx.doi.org/10.1177/109442810031002. [ Links ]

Vodermaier, A., Linden, W., & Siu, C. (2009). Screening for emotional distress in cancer patients: a systematic review of assessment instruments. Journal of the National Cancer Institute, 101, 1464-1488. http://dx.doi.org/10.1093/jnci/djp336. [ Links ]

Wang, J., Kelly, B. C., Booth B. M., Falck R. S., Leukefeld C., & Carlson, R. G. (2010). Examining factorial structure and measurement invariance of the Brief Symptom Inventory (BSI)-18 among drug users. Addictive Behaviors, 35, 23-29. http://dx.doi.org/10.1016/j.addbeh.2009.08.003. [ Links ]

Watson, M., & Bultz, B.D. (2010). Distress, the 6th vital sign in cancer care. Caring for patients' emotional needs: what does this mean and what helps? Psycho-Oncologie, 4(3), 159-163. http://dx.doi.org/10.1007/s11839-010-0269-z. [ Links ]

Wiesner, M., Chen, V., Windle, M., Elliott, M. N., Grunbaum, J. A., Kanouse, D. E., & Schuster, M. A. (2010). Factor structure and psychometric properties of the Brief Symptom Inventory-18 in women: A MACS approach to testing for invariance across racial/ethnic groups. Psychological Assessment, 22, 912-922. http://dx.doi.org/10.1037/a0020704. [ Links ]

Zabora, J., Brintzenhofeszoc, K., Jacobsen, P., Curbow, B., Piantadosi, S., Hooker, C., Owens, A., & Derogatis, L. (2001). A new psychosocial screening instrument for use with cancer patients. Psychosomatics, 42, 241-6. http://dx.doi.org/10.1176/appi.psy.42.3.241. [ Links ]

Received: July 24, 2018; Accepted: February 12, 2019

* Corresponding author. E-mail address: paula.martinez@uv.es

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License