Scientific and Technological Research

Validation to Spanish of the Iowa Satisfaction with Anesthesia Scale (ISAS) for monitored anesthesia care in ophthalmic surgery^*

Validación al español de la escala «The Iowa satisfaction with anesthesia scale (ISAS)» para cuidado anestésico monitorizado en cirugía de oftalmología

Corresponding author. E-mail address: smartaji@hotmail.com (L.F. Jiménez García).

^*Please cite this article as: Jiménez García LF, Capera ADR. Validación al español de la escala "The Iowa Satisfaction with Anesthesia Scale (ISAS)" para cuidado anestésico monitorizado en cirugía de oftalmología. Rev Colomb Anestesiol. 2014;42:272–280.

Received 24 February 2014 - Accepted 21 July 2014

Abstract

The Iowa Satisfaction with Anesthesia Scale (ISAS) was rigorously developed and has been shown to be valid and reliable for evaluating patient satisfaction with monitored anesthesia care (MAC) in several scenarios. Such an instrument is important given the success that MAC is enjoying in the field.

Objective: Validate the ISAS in Spanish and establish indicators of validity and reliability in patients undergoing ophthalmic surgeries with MAC.

Methods: A translation and back-translation of the scale, face validity and pilot tests for adjustmentswere completed. The final instrumentwas applied to 117 subjects over18 years of age, ASA I–III, in two healthcare institutions in Villavicencio (Meta Department, Colombia) in order to measure the concurrent criterion validity between the patients and the anesthesiologist of the case. The internal consistency of the scale was established in its first application to the subjects and later applied for the second and third times to verify the test–retest reliability.

Results: A Pearson anesthesiologist/patient concurrent criterion validity 0.85 CI 95% (0.79–0.89), intra-class 0.82 CI 95% (0.77–0.88),was confirmed. Internal consistencywas measured with a Cronbach's alpha of 0.71. Test–retest reliability (40–65 min) was measured with Pearson and intra-class 0.95 CI 95% (0.93–0.96) and, (12–36 h) Pearson 0.65 CI 95% (0.52–0.75), intra-class 0.64 CI 95% (0.53–0.76).

Conclusions: The validation of the ISAS in Spanish allows for the use of a valid and reliable instrument to objectively measure the satisfaction of the patient in ophthalmic surgery under MAC.

Keywords: ISAS scale, Monitored anesthesia care, Patient satisfaction, Validation studies, Reliability and validity.

Resumen

La herramienta «The IOWA satisfaction with anesthesia scale (ISAS)» fue desarrollada rigurosamente y ha demostrado ser válida y confiable para evaluar la satisfacción del paciente con el cuidado anestésico monitorizado (CAM) en múltiples escenarios; un instrumento de estas características es importante, dado el auge que el CAM está teniendo en el medio.

Objetivo: Validar al español la escala ISAS y establecer los indicadores de validez y confiabilidad en pacientes tratados mediante cirugías de oftalmología bajo CAM.

Métodos: Se hizo traducción y retrotraducción de la escala, validación de apariencia y pruebas piloto para ajustes. Se aplicó el instrumento definitivo a 117 sujetos mayores de 18 años, ASA I-III en 2 instituciones de salud en Villavicencio (departamento del Meta, Colombia) para medir la validez de criterio concurrente entre los pacientes y el anestesiólogo del caso; se estableció la consistencia interna de la escala en su primera aplicación a los sujetos y se aplicó por segunda y tercera vez el instrumento para verificar confiabilidad test-retest.

Resultados: Se constató validez de criterio concurrente anestesiólogo-paciente, con Pearson 0,85 IC 95% (0,79-0,89), intraclase 0,82 IC 95% (0,77-0,88), se midió la consistencia interna con un alfa de Cronbach de 0,71, confiabilidad test-retest (40-65 min), Pearson e intraclase 0,95 IC 95% (0,93-0,96) (12-36 h) Pearson 0,65 IC 95% (0,52-0,75), intraclase 0,64 IC 95% (0,53-0,76).

Conclusiones: La validación de la escala ISAS al español permite usar un instrumento válido y confiable para medir objetivamente la satisfacción del paciente en cirugía de oftalmología bajo CAM.

Palabras clave: Escala EISA, Cuidado anestésico monitorizado, Satisfacción del paciente, Estudios de validación, Confiabilidad y validez.

Introduction

The incidence of “MAC” (monitored anesthesia care) - during the time it has been used in the OR in the United States shows that over one-third of ambulatory surgeries are administered using MAC and or peripheral block (P < 0.0001).¹

MAC is frequently used in ophthalmology and it is difficult to have one instrument to assess patient satisfaction with this anesthesia technique. The IOWA Satisfaction with Anesthesia Scale (ISAS) is available in English for this purpose (The IOWA Satisfaction with Anesthesia Scale (ISAS) in Spanish). Dexter, Aker, and Wright developed this scale at the University of Iowa in 1997.² It is a direct estimation scale because it allows for straightforward extraction of quantitative data; it is one-dimensional, discriminative with brief instructions (Table 1). It comprises 11 items all with equal weight (Table 2). The items are described as statements. The first statement expresses a negative feeling, the second one a positive feeling, and so on until the end of the statements. These presentation avoids the risk of acquiescence, defined as the trend of the respondent to always agree with the questions or statements in the scale, regardless of its content.³ For each item there is a choice among six numerical options, with a score from −3 to +3 (no categories) (Table 3) to enable a quantitative analysis. If the statements described with a positive feeling, for instance “I felt good” are answered with one of the “Agree” options, that implies satisfaction with the anesthesia; however, if the statements describing a negative feeling are answered with an “Agree” option, the respondent would be expressing dissatisfaction with the anesthesia.²

Some publications refer to the usefulness, validity and reliability of the EISA scale in clinical trials^4-11 including Dexter and Candioti¹⁰ who showed that the EISA scale may be used as a primary outcome measurement in multicenter clinical trials. Chanthong et al.¹² did a systematic review searching measurement instruments to assess patient satisfaction following ambulatory anesthesia and found just two questionnaires with rigorous psychometric methods; one of them was the EISA scale. Moreover, they argue that out of 11 multi-item questionnaires used to measure patient satisfaction with anesthesia, only the EISA scale has been used in other trials. Other two reviews^13,14 confirm that the process for developing the EISA scale was psychometrically correct.

It is clear that it is better to validate an already accepted instrument because it opens the possibility to undertake studies in different countries or cultures¹⁵; the objective of this study was to validate the “IOWA satisfaction with anesthesia scale (ISAS)”, establishing its validity and reliability indicators, as a tool to measure patient satisfaction with MAC in ophthalmological surgery.

Methods

A study to validate the scale carried out with the approval of the medical ethics committee of the Hospital Departamental de Villavicencio. The study took place at this institution and at the Clinica de Cirugía Ocular of that same city. The authorization of the author for validation of the Scale was obtained in addition to the patient’s informed consent. The patients selected were surgical patients from the ophthalmology unit that met the inclusion criteria: over 18 years old, choosing the MAC anesthetic technique and ASA-PS classification between 1 and 3. Patients with any mental or awareness disorders were excluded, as well as non-Spanish speaking patients. The sample selection was convenient and non probabilistic sequential until completing the desired number of patients, from June through December 2011. The anesthesiologist responsible for the case made the decision about the patient’s mode of sedation. Three phases were completed: the first phase was the translation - back translation of the scale with the participation of two specialists in the area. The direct translation (English into Spanish) was done; after one month a translation was made back into English, taking into account that the version translated was from the translation previously done by the other translator into Spanish (back-translation). The respective translations and back-translations were submitted to a review board (2 anesthesiologists, 3 ophthalmologists and a language expert), to make sure that the documents were consistent with the original scale and a Spanish version of the EISA scale was established (Tables 4-6).

A group of 12 experts on the subject, including ophthalmologists and anesthesiologists also evaluated the apparent validity, using a questionnaire to collect their opinions.

The second phase was the pilot test where the participants answered a questionnaire to check the apparent validity and assess their understanding of the instructions, statements (items) and the various optional answers. The usefulness of the scale was evaluated during this phase through a questionnaire for the staff that administered the scale and for the participating anesthesiologists.

The third and final phase used the final instrument (after making the changes resulting from the pilot tests) to evaluate validity and reliability.

The concurrent criterion validity was measured based on the consistency shown in the scores of the self-administered scale of the anesthesiologist responsible for the case and those of the patient when filling out the final scale for the first time. This was done using Pearson correlation coefficient and the intra-class correlation coefficient.

The reliability was measured based on the internal consistency estimating Cronbach's alpha in the final instrument completed by the patients during the first application.

The test–retest reliability was checked calculating Pearson correlation coefficients and intra-class. The final instrument was administered for a second time, before the patients left the post-anesthesia care unit "PACU" (40–65 min after completing the questionnaire for the first time), trying to reproduce Dexter et al., 1-h time.² There was a third round (between 12 and 36 h) when the patient came back for postsurgery control. This time interval was chosen because many patients lived in the rural areas and it was difficult to contact them again.

During the pilot tests and the application of the final scale in Spanish, the intention was to respect the methodology and the original format of the scale in the English version.²

Patient participation

After at least 15 min into Phase II of the PACU, the patients were invited to participate, explaining to them that the objective was to identify any potential improvements so that they could feel better with the anesthesia administered. If they agreed to participate, they were given the scale, a pen and an envelope. Then they were left alone and the time to read and complete the questionnaire before placing it into the envelope was recorded. In as much as possible, the tool was self-administered; however, in some cases it had to be administered as an interview (by the nursing staff) because the scale was validated in visually impaired individuals - this was different from the design in Dexter et al.² and rather followed the parameters established for the sample of patients in the study by Fung, Cohen, Stewart, and Davies⁴ where the scale was used in patients who underwent cataract surgery.

Sample size

In accordance with Norman and Streiner¹⁶ and Cortina,¹⁷ at least 111 patients were required to estimate the internal consistency (10 per item). To estimate the sample size to measure intra-class correlation coefficients, a formula based on the range of the confidence interval was used, considering a type 1 error or a 0.05 level of significance, a correlation coefficient of at least 0.8 and a level of two-tailed accuracy of 0.1 resulted in 75 patients. To estimate the sample for Pearson correlation coefficient, the hypothesis test formula was used, considering type I error probability or a 0.05 level of significance; type II error or 0.2; correlation coefficient for a population of 0.8 and two-tailed null hypothesis test value of 0.6 that resulted in an estimate of 99 patients. A larger number of patients were recruited for each sample, except for the last one because there were patient losses between the first and the third questionnaire completed. If done otherwise, the third application would not have the 99 patients sample required to calculate the test-retest reliability value between the first, the second and the third round. This had a positive effect narrowing the confidence intervals. All the estimates were based on the statistical software STATA version 11.1.

Results

Pilot Test 1: once the experts committee verified the apparent validity of the scale, the majority agreed that it did measure what was intended to measure; an attempt was made to administer the scale to 20-30 patients in a pilot test that was interrupted after 7 patients because of the completion time results: mean 16.78 min; SD 5.19 min, minimum 13 min maximum 28 min. This time was too long as compared to the time reported by² of 4.6 + 0 − 2.1 min mean of 5 min, maximum 10 min. It was felt that these results were due to the way the answer options were drafted that had to be rephrased for many patients (5 of 7) who were unable to read because of their visual impairment. A second pilot test was done for the patients to choose between two drafts for the possible answers (Table 7).

Pilot Test 2: the mean time to complete the questionnaire in 21 patients was 8.76 min, with a SD of 3.4 min, a minimum of 4 min and a maximum of 20 min. This result was due to the fact that 16 out of 21 patients (76.19%) chose the second draft for the answers, which resulted in shorter completion times - mean of 7.62 min versus a mean of 12.4 min for 5 (23.81%) out of 21 that chose the Draft 1 (used in the first pilot test). The patient that took 20 min to complete the scale was blind and chose Draft 1 that took patients longer in average to complete.

16 Patients 76.19% (n = 21) considered that the wording of the instructions was easy to comprehend (ideally 100% should say that it was easy to comprehend), and this led to the changes suggested on Table 8 - right hand side.

As far as the answer options is concerned, 13 patients, 61.9% (n = 21) thought they were absolutely easy to understand, but making changes in the answer options to reduce the completion time (a mere play on words), could totally change the original meaning based on the scale score and a methodological validation does not allow for these type of changes.

Statement 4 (Me sentí relajado) changed to “Me sentí relajado tranquilo”; statement 6 (Me sentí seguro) changed to “Me sentí seguro confiado”; and statement 8 (“Quedé satisfecho con mi cuidado anestésico”) changed to “Quedé satisfecho contento con mi cuidado anestésico” because 100% of the patients did not fully understand and felt that the language was not completely adequate for these items. Adding these terms was the result of exploring with the patients from the rural areas the best-suited terminology to explain the words relajado, seguro and satisfecho. Table 9 shows the final version of the items on the EISA scale.

In this second pilot test, the 21 patients (100%) answered «Yes» to the question “In your opinion, do you think that the scale measures what it is intended to measure (patient satisfaction with his/her anesthesia care)? - Ratifying the apparent validity in a group of patients. The anesthesiologists and the nursing staff felt that the scale was easy to administer and the anesthesiologists thought it was easy to establish the scale scores.

Phase 3: Out of 122 patients invited to complete the final version of the EISA scale in Spanish, 119 (97.54%) agreed, three refused saying that they were not interested in being part of a research trial. The 119 patients answered all the questions in all the scales administered, but only 117 were analyzed because two of the respondents were illiterate and this fact was evidenced just at the end of the test, because as mentioned before, assistance was provided to patients with visual impairment.

The CAM was administered by five anesthesiologists, two of them researchers; six surgeons operated on the patients. The sample distribution was as follows: Clnica de Cirugía Ocular ponerlo 107 (91.45%), Hospital Departamental de Villavicencio 10 (8.55%).

Tables 10-13 show the results of the patients characteristics, completion time, need for assistance to respond to the scale based on level of education and the scores obtained in the EISA scale.

Result of the concurrent criteria validation

Pearson correlation coefficient between the score one given by the patient and the score given by the case anesthesiologist (N = 117) was 0.85 (0.79-0.89) CI 95% r²= 0.72 and the score of the intra-class correlation coefficient for these same two variables was 0.82 (0.77-0.88) CI 95%.

Reliability results

For the internal consistency of the scale the Cronbach’s alpha value (N = 117) was 0.71 with a lower limit no less than 0.64 and a 95% CI. If the decision was made to remove item 1 from the scale, this value could change to a Cronbach’s of 0.71; removing item two, it could change to 0.69; removing item three to 0.72, removing four to 0.70, removing five to 0.67, removing six to 0.70, seven to 0.72, eight to 0.68, nine to 0.64, ten to 0.67 and eleven 0.69.

For test-retest reliability, see the information in Table 14.

Discussion

Being able to make changes during the pilot tests was significant in order to accomplish applicability, effectiveness and usefulness of the Spanish scale and to clarify the terms if the scale were administered to a rural population (a reality in our country). The patient’s acceptability of the scale was good and they completed it thoroughly. Trans-cultural equivalence and trans-language equivalence was identified without affecting the psychometric philosophy underlying the design of the scale.

Most patients with less five years of primary school required assistance to complete the scale (Table 12) (Dexter et al.² explains that the scale was drafted to be understood by someone with fourth level of writing in the United in 1997) because the respondents may be confused about how to choose from the various options offered in the answer; however, it is unclear whether the low level of education was due to the inability to read because of visual impairment and depend on listening to someone else reading which increases the difficulty to understand the answering mechanism or rather because of a poor understanding as a result of a low level of education (this information is critical since some data may be lost when the scale is not administered through an interview if needed). It would be interesting to determine in non-visually impaired individuals whether the level of education influences the need for help or not.

The difference in the scores of the case anesthesiologists and the first score from the patients (n = 117) was 0.18; Dexter et al.² reported (n = 32) 0.3. We found that anesthesiologists believe that the patients perceive more satisfaction that what they really feel; that is why it is important to evaluate patient satisfaction objectively when undergoing CAM, to improve the quality of anesthesia.

The values for validating the concurrent criteria of Pearson correlation coefficients and intra-class between the patient and the anesthesiologist scores were positive and high correlation factors, and much higher than those obtained by Dexter et al.² of r² = 0.23 (P < 0.01 n = 32) that also reported a positive correlation or those reported by Fung et al.⁴ where the level of correlation was insignificant - Pearson 0.09. There could have been some bias because on several occasions the case anesthesiologist was one of the two researchers and they were probably more receptive to realize how the patients felt (it must be noted however that the researchers did not help the patients in completing the scale). However, in short surgical procedures as is the case in most ophthalmological surgeries, it is easier to perceive how the patient feels than in long CAM procedures. Another reason for explaining these results could be cultural; our patients may be more spontaneous and expressive with their emotions.

An acceptable Cronbach’s alpha value of 0.71 (n = 117)^18,19 was achieved, lower than the 0.80 (n = 80) value reported by Dexter et al.,² probably because there was increasing patient heterogeneity in that trial because they underwent surgical interventions of diverse specialties. This value was similar to that reported by Fung et al.⁴ of 0.68 where the total population underwent cataract surgery. It was noted that when removing any item, the Cronbach’s alpha value did not increase much (maximum 0.72), showing that none of the items is erroneously raising the scale’s homogeneity.

Since the EISA scale has less than 20 items, respecting in Spanish the criteria used in English to avoid the acquiescence of the respondents and evaluating a less heterogeneous population, Cronbach’s alpha could be underestimated.

Pearson and intra-class correlation coefficients for test-retest reliability decrease as the time to the readministration of the scale increases; this result is probably associated to the fact that the patient becomes disconnected from the emotions experienced during the procedure and assigns less weight to these feelings as time progresses. Nonetheless, the values obtained for the test-retest reliability between the first and the second application of the scale n = 116 show a high reliability,²⁰ abov e the value reported by Dexter et al.² who administered the scale for the second time after 1 h, obtaining values of r² = 0.74 (P < 0.01) n = 9 and Fung et al.⁴ intra-class of 0.57 n = 26 (3-9 h after the first administration). The test-retest reliability values of between the first and the third administration n = 99 in this trial were lower than those reported by Dexter et al.² r² = 0.76 P < 0.001 n = 11 (4.4 days plus or minus 1.7 following the first administration); Fung et al.⁴ did not administer the scale for the third time.

Although these date are being analyzed, it is difficult to compare and interpret them and the difference identified for the test-retest in the results of these three studies may be due to: (1) different sample sizes; (2) the situation wherein the scale was administered for the second and third time: Dexter’s third application is ambulatory and forwarded to patients via mail, while in this case the administration of the scale was intra-institutionally always; (3) the time gap between the first and the third administration of the scale are different between these trial and Dexter et al.²; and (4), Dexter’s patients² did not receive any assistance to answer the questionnaire, while the patients of Fung et al.⁴ and the patients in this trial did receive assistance.

However we may claim that the test-retest reliability for the EISA scale in Spanish is high if administered for the second time with 1-h difference in between and it is acceptable between 12 to 36 h.

Conclusions

The EISA scale in Spanish was validated to assess patient satisfaction with monitored anesthesia care, when undergoing ophthalmological procedures. The scale was proven to be useful and friendly in Spanish as it is in its original English language. It would be interesting to verify whether this scale is equally valid and reliable in Spanish in other surgical scenarios apart from ophthalmology. This new tool available in Spanish encourages anesthesia research. This is a task²¹ that should involve every anesthesiologist.

Funding

Authors' resources.

Conflicts of interest

The authors have no conflicts of interest to declare.

Acknowledgements

To the team of anesthesiologists, ophthalmologists and nurses from the Hospital Departamental of Villavicencio and Eye Surgery Clinic. To Dr. Franklin Dexter and the University of Iowa Research Foundation for their authorization to validate the scale and to Dr. Guillermo Sánchez Vanegas, professor of Clínica Universidad del Bosque, from the epidemiology specialization program by his continue Guide.

References