ARTÍCULO ORIGINAL / ORIGINAL ARTICLE

La legibilidad de los materiales educativos sobre la diabetes: Implicaciones para la educación de pacientes con materiales escritos

Readability of Diabetes Education Materials: Implications for reaching patients with written materials

Carolina Aguilera¹, Miguel A. Pérez², Luz Marina Alonso Palacio³

¹ Consultora de salud pública. aguileracarolina808@gmail.com

² Professor and chair, Department of Public Health. California State University. 2345 E San Ramón Ave. MS30, Fresno, CA. (USA). mperez@csufresno.edu.

³ Profesora del departamento de Salud Pública. Universidad del Norte, Barranquilla (Colombia). lmalonso@uninorte.edu.co

Correspondencia: Universidad del Norte Km 5 Via Puerto Colombia, A.A. 1569, Barranquilla (Colombia).

Fecha de recepción: 9 de marzo de 2010
Fecha de aceptación: 15 de mayo de 2010

Resumen

Objetivo: Determinar el nivel educativo necesario para comprender los materiales educativos suministrados a pacientes diabéticos por organizaciones gubernamentales y NGOs.

Materiales y métodos: Este es un estudio descriptivo que explora la legibilidad de 81 materiales de educación disponibles a pacientes con diabetes y distribuidos por proveedores de salud. Se utilizó 2 medidas para determinar los niveles de legibilidad de los materiales informativos para diabéticos, el SMOG Readability Formula y el Fray Graph. La muestra excluyó materiales educativos que no estuvieran en inglés y aquellos con objetivos comerciales. Para el análisis se utilizó medidas descriptivas y prueba t para muestras y se interpretó el valor de p.

Resultados: Los resultados muestran que aunque los materiales provistos por organizaciones no gubernamentales son más fáciles de leer, éstos están generalmente escritos a niveles de lectura más alta de la audiencia para la cual son desarrollados.

Conclusiones: Se concluye que los materiales educativos de educación en salud para diabéticos no permiten una comprensión total de su contenido, ya que están escritos utilizando vocabulario más complejo que el que posee la población que los recibe.

Palabras clave: Diabetes Mellitus, educación en salud, alfabetismo en salud.

Abstract

Objective: To ascertain the health literacy levels of diabetes patient education materials distributed by government-funded or nonprofit organization.

Methods: This descriptive study explored readability levels of 81 written diabetes health education materials available from healthcare providers. The Simplified Measure of Gobbledygoop (SMOG) readability formula and the Fray Graph were utilized to determine readability levels of diabetes patient information materials. The sample size excluded materials not in English and those with commercial purposes. Data analysis included measures of central tendency. In adition used t test and p-value.

Results: Results from this study show that while education materials provided by nonprofit organizations are easier to read, they are still generally above the reading level of a large portion of the population they are intended to reach.

Conclusion: Results from this study suggest that the majority of diabetes patient education materials are not adequate to reach their intended population due to high readability levels.

Keywords: diabete mellitus, health education, health literacy.

INTRODUCTION

Diabetes mellitus, a metabolic disorder related to the body's inability to produce or metabolize sugars, is increasing in the United States (1-2). According to the Centers for Disease Control and Prevention, diabetes was the sixth leading cause of death among Americans in 2006 (3) .The federal government estimates that some 23.6 million Americans are living with that condition in the US (4). Additional reports have stated that 1.5 million new cases of diabetes emerge each year within the U.S. population age 20 years or older with significant disparities observed by race/ethnicity. It is also estimated that 11.2% of men, 10.2% of women, and 23% of people over 60 have diabetes (5). In addition to the increasing number of diagnosed cases, the National Diabetes Education Program estimates that some 57 million Americans aged 20 and older are in the pre-diabetes stage (4).

The American Diabetes Association (ADA) has estimated the economic burden caused by diabetes on the U.S. economy at about $336 billion per year including loss workplace productivity and some $117 billion in Medicare benefits (6). These projections take into account based on expected diabetes cases, but did not incorporate the expected impact on diabetes expenditures from expected higher obesity rates.

Noncompliance with diabetes treatment leads to chronic complications such as blindness among adults and end-stage renal disease. Similarly, 60% to 70 °% of people with diabetes have some degree of nerve damage, 60% of amputations reported in the U.S. are diabetes related, and 15% to 20% of spontaneous abortions are due to poor control of diabetes during pregnancy(7). Non-compliance with diabetes treatment appears to be related to health literacy levels in some of the cases.

HEALTH LITERACY

Health literacy is defined as having the ability to read, write, understand, and apply health related information regardless of its source. Individuals use the health related information to improve their health status and follow medical instructions given by health care providers (8-9). People with inadequate functional health literacy or whose reading skills are below the basic literacy level confront several barriers when attempting to access medical care (10).

Studies have found that poor readers cannot find their way through hospitals, complete medical information forms, follow through with instructions from prescriptions, and/ or understand the content in written health education materials, or even informed consent forms (11-14). Studies have also found that individuals with low health literacy skills are less likely to adhere to and comply with medication regimes (15-16). Patients with low health literacy skills are also more likely to make more medical errors that can reduce their quality of life and/or put their lives in danger (17-19).

Weiss and Palmer (2004) found that "persons with low-literate skills generate higher charges for health care than do persons with better reading skills". They estimated the costs of low literacy to vary from $50 billion to $73 billion annually in the form of health problems and unnecessary hospitalizations. Other estimates have placed the estimated expenditures related to the lack of adequate health literacy in the year 2001 from $32 to $58 billion excluding healthcare costs (13). For years healthcare providers have erroneously utilized the patient's highest educational level as a proxy to ascertain their reading, writing, and comprehension skills (20). The literature suggests that using the number of years of school completed does not accurately reflect an individual's actual literacy level. In fact, some studies have found that participants' literacy level was two to several years below the equivalent of their last school year completed (21-22).

Data from the National Assessment of Adult Literacy in 2003 found that some 87 million Americans had basic or below basic health literacy levels. The same study found that women had higher health literacy levels than males and that almost two thirds (60%) of the elderly had basic or below basic health literacy (23-25). Unlike the flu, skin infections, or food poisoning, whose symptoms are easy to recognize, inadequate health literacy has been very hard to detect because, many people have hidden their problem from their healthcare providers. In fact, people have managed to hide it from their spouses, children, friends, and/or coworkers (25-27).

HEALTH EDUCATION AND HEALTH LITERACY

Research has found an inconsistency between the readability of health educational materials and the reading capabilities of more than 90 million adults in the United States (25, 28, 31). One study evaluating cancer educational materials for African Americans found readability levels to be higher than the target population's health literacy skills. Similar results were obtained when Internet brochures about children and adolescent mental health were tested for readability (32-33). Another study found that 40% of brochures and pamphlets about the risk factors of cardiovascular diseases targeting the African American population and women did not match the literacy skills of those populations (34).

HEALTH LITERACY AND DIABETES

Studies have found a direct relationship between the treatment outcome of diabetes care and health literacy (35-36). Diabetes patients with inadequate health literacy were less likely to manage their sugar level, were more likely to have diabetes complications, and were more likely to report diabetic retinopathy. These findings supported the direct relationship between health literacy and health status reported by other researchers (21-35-38).

Written materials are often used to educate and to provide treatment related instructions to patients with diabetes (39-40). Despite their popularity as an education tool, written materials are seldom measured for their readability levels or to assure that the educational information matches the health literacy of their intended audience (41). This oversight may place the health status of some diabetes patients at risk and may account for a proportion of those patients who do not adhere to treatment regimes.

One way to ameliorate the consequences of low health literacy is to provide patients with written information they can read and understand. Readability formulas such as the SMOG formula, Flesch-Kincaid formula, or FOG index are commonly utilized in the literature.

Despite extensive research in the area of document readability (42-43) little has been done to evaluate recent print educational materials about diabetes; therefore, the purpose of this study was to assess the readability of selected written health information materials regarding diabetes. In this study, the Simplified Measure of Gobbledygoop (SMOG) Readability Formula and the Fry Graph method were used to determine the readability of the materials being analyzed.

METHODS

As the purpose of this investigation was to assess readability levels of printed diabetes education materials, the investigators randomly identified nonprofit healthcare organizations and federally funded medical centers which provide these materials to their patients as part of their diabetes management programs. The investigators contacted the health educators or health program managers of these medical centers and explained the purpose of this study.

Seven of the ten agencies contacted for this study agreed to provide copies of the written education materials they give their patients. Representatives from two organizations said that while they provided diabetes education they limited their education to an oral format and that in the few cases where diabetes written materials were provided the agencies they represented used the health education materials produced by the American Diabetes Association. Only one organization refused to provide sample education materials alleging that their materials were developed by the federal government and were likely already included in the sample.

Written materials from the Internet used in this study were those found in the websites for the California Diabetes Program (CDP), Diabetes Coalition of California (DCC), Food and Drug Administration (FDA), the American Diabetes Association (ADA), the California Diabetes Program (CDP), and the National Diabetes Education Program (NDEP). Some of the materials were published through the collaboration of the previously mentioned agencies and the National Institutes of Health (NIH), the Centers for Disease Control and Prevention (CDC), the National Diabetes Information Clearinghouse, and the U.S. Department of Health and Human Services (DHHS).

The criteria for inclusion as part of this readability assessment were as follows:

1. Education materials had to be written in English.

2. The content had to be relevant to diabetes.

3. Online materials had to have a publication date and/or list their publisher or sponsor.

4. Online materials had to be created by organizations or agencies that provide services in California.

5. Education materials produced by pharmaceutical companies were considered as acceptable due to their educational content except for the written materials distributed directly by pharmacies.

6. Written health education materials utilized solely for advertising a diabetes-care product or an agency, without educational content, were excluded from the sample.

MEASUREMENT METHODS

To measure readability, the researchers used the Readability Plus 2005 (version 7.0) computer program. This software contains seven of the most commonly used readability formulas: the Powers-Summer-Kearl formula, the Flesch Reading Ease formula, the Flesch Grade Level formula, FOG Index formula, SMOG, FORCAST, and the Fry Graph. These programs assesses text only, taking into account the number of words in each sentence, the number of syllables in each word, and the number of sentences in the sample entered. None of the programs have the ability to evaluate graphics or pictures which can be found in the printed materials.

ANALYSIS OF WRITTEN DIABETES EDUCATION MATERIALS

Research has suggested using a combination of different formulas to ascertain document readability increases the chance of producing valid and accurate results, therefore, the investigators utilized two formulas (40, 44).^. The two formulas utilized in this study to test the readability of the written diabetes education material sample were the SMOG readability formula and the Fry Graph. Although the purpose of this study is not to compare the effectiveness of readability formulas, the investigators compared the results from the two formulas. This gave two reading grades for each material, thereby increasing the likelihood of accurate and valid results.

Each item was tested by the SMOG formula and the Fry graph formula; these two formulas calculate the number of syllables per word and the number of words per sentence. Therefore, the first step in analyzing the written material was to select the number of words or sentences needed by each readability formula. After selecting the sentences and/ or words, the sample was entered in a Word document and saved as a text file. The text file was then analyzed with the Readability Plus software.

THE SMOG READABILITY FORMULA

To use the SMOG formula, the researcher does not have to evaluate the entire written document; a sample of 30 sentences is sufficient. The sample consists of 10 sentences from the beginning of the material, 10 from the middle, and another 10 from the end. Using this method any word with more than three syllables is circled. The circled words are tallied, and the nearest number that equals a perfect square root of this figure is found. Finally, the square root is added to a constant that is equal to three. The number that is the sum is the grade level required to understand the reading material (45). Therefore, the researchers chose 30 sentences from each of the written materials in the sample. Ten of the sentences were from the first pages, 10 sentences were from the middle of the written material (the "middle" is relative to the number of pages in each item), and 10 sentences were from the last page(s) of the diabetes education materials. When the materials contained fewer than 30 sentences, all the sentences in the document were used for readability analysis. It is also important to note that certain punctuation marks, such as hyphens and quotation marks, were omitted when the information was being inputted. Bold letters, bullets, visual aids, and other symbols that may enhance the reading ease of the materials were also omitted. Question marks and exclamation marks were retained as they serve the same purpose as periods; they separate sentences. The SMOG readability formula does not recognize different connotations or contexts given to certain words.

THE FRY GRAPH READABILITY FORMULA

The Fry Graph readability formula in the Readability Plus software requires three samples of 100 words from the written material; however, it is capable of analyzing hundreds of words effectively. In an effort to ensure an adequate comparison of materials tested by the two formulas, equal numbers of words were selected for testing. For instance, the 30 sentences selected for analysis by the SMOG formula varied in word count. One 30-sentence sample contained 480 words, another contained 340 words, and a third item contained 193. For each item, an equal number of words and sentences were selected for assessment by the Fry Graph formula.

As the data were entered in the program, symbols that may enhance readability were omitted to make data analysis easier. The symbols might be hyphens, quotation marks, bullets, asterisks, and visual aids. Hard breaks between paragraphs were also omitted to avoid program malfunctions. As with the SMOG formula, question marks, exclamation marks, and periods were retained.

RESULTS

A total of 140 written diabetes education materials were considered for inclusion in this study. Of the 140 items received 17 were rejected due their content including mostly tables, charts, and pictures; their content being written entirely in Spanish (46), and 12 were duplicates of materials that already formed part of the sample. Another five were excluded because their content was primary for commercial purposes and contained no educational content.

In the end, 90 publications met the criteria for inclusion in this research and were considered for readability analysis (see table 1). The diabetes education materials included in the sample were categorized by the type of information contained with the majority falling into the general information category (see table 2).

Information gathered from each written education material consisted of the title, publication date, publisher and/or information about who created or adapted the item, and whether the source was a government-funded or nonprofit organization or medical center. The materials were analyzed using the SMOG and Fry Graph methods to determine the reading levels assigned to each publication. The SMOG readability formula was used to assess all 90 written diabetes education materials; however, the Fry Graph was unable to calculate reading levels for nine of the materials. Therefore, the nine pairs of data were omitted from this analysis, leaving only 81 complete pairs of data (reading grades) to be tested in the data analysis. The grade levels for the materials according to the two formulas are shown in Table 3.

The highest mean reading grade, 11.8, was found for the sample of written materials produced by a collaboration of the U.S. Department of Health and Human Services, the National Institutes of Health (NIH), National Kidney Disease Education Program (NKEDP), and National Institute of Diabetes and Digestive and Kidney Disease (NIDDK). Materials created by the California Diabetes Program (CDP) and posted on its website also had a high reading grade: 11.5.

A comparison of readability levels revealed that the Fry Graph gives lower reading grade levels than the levels given by the SMOG readability formula. For instance, the Fry Graph gave a reading grade level of 6 for item 30 in Table 3, which was equivalent to the readability for materials used in 6^th grade (elementary school). The SMOG method calculated a reading level of 8.5, which equaled the readability in materials used at a middle school level. In other words, when text was assessed using the Fry Graph, the readability appeared to be higher.

A Paired-Sample T-test to 81 pairs of data (see Tables 4 and 5), revealed statistically significant differences between the two formulas. The statistical analysis also showed that the standard deviation was greater (± 2.8872) for the Fry Graph results, in comparison to the standard deviation (± 1.7968) from the SMOG readability formula results.

Table 6 shows Readability Results by Source of Materials. According to the SMOG readability formula, materials obtained from government-funded organizations were easier to read than those made available by nonprofit organizations. On the other hand, the Fry Graph analysis showed that the materials used by the nonprofit organizations scored lower reading grades than the materials obtained from organizations that used government funds.

DISCUSSIONS

Health care providers, health educators, pharmacists, and even community lay educators often use written health materials to educate, reinforce, or expand health information and patient instructions following medical visits. The premise that written health materials will assist patients better understand information and adhere to treatment regimes is predicated on the assumption that patients posses sufficiently high health literacy levels to understand the materials. Given the stigma - and limited time- associated with asking patients to demonstrate health literacy it is not surprising that many providers and disease prevention specialists rely on the self-reported maximum number of school years completed to estimate patients' literacy. Studies, however, have shown that relying on years of education as proxy for measuring health literacy provides erroneous estimates (20, 47-50) which may result in unforeseen adverse treatment outcomes. In fact, an increasing body of evidence suggests that materials should be tailored to the target population's health literacy level.

Diabetes educators and health providers have developed many types of educational materials ranging from general information to medical instructions to lifestyles information to assist their patients better understand their condition, adhere to treatment regimes, and to improve quality of life. Results from this study suggest that many of the most commonly used materials' reading level may exceed their intended audience's health literacy level. The discrepancy between their purpose and the audience's ability to understand them may in fact render some of these materials useless and provide a false sense of security to the provider who thinks that their instructions are not only understood by the patient, but are also being followed. It is therefore, recommended, that written educational materials be evaluated periodically to ascertain not only their accuracy, but reading level as well.

The national health objectives enshrined in Healthy People 2010 have identified improving health literacy as a priority in an effort to decrease problems associated with patient non-compliance. Given the increasing diabetes rates in the US population (1, 5), it is reasonable to identify the need to measure written education materials' readability level as a way to assist patients. Results from this study suggest the following recommendations with the goal of providing better quality of services to individuals who have been diagnosed with diabetes.

1. There is a need to increase awareness related to health literacy and its impact on the health status of diabetes patients.

2. Assessing patient's literacy level should be a priority to any health care provider, health educator, pharmacist, or community health educator working with diabetic patients.

3. Anyone working with diabetes patients' should know their health literacy level in addition to their overall literacy level. This information will be vital in determining the appropriate reading level for written materials provided.

4. Health care providers, health educators, pharmacists, and community health educators should periodically assess the suitability as well as readability of printed education materials. Suitability includes font size, color, graphics, position of text, and position of graphics.

CONCLUSION

The purpose of this research was to assess the readability of selected printed diabetes education materials and as such did not measure cultural appropriateness or include materials in other languages. The readability assessment and statistical analysis showed that the majority of printed education materials provided by nonprofit medical centers and government-funded health organizations require a higher educational level. People receiving these materials should have attained an educational level of approximately 10^th grade (the equivalent of bachillerato en Colombia) to read these materials. However, the National Assessment of Adult Literacy found that, in 2003, 14% of the US population read at a 5th-grade level (23) - or the equivalent of primary school in Colombia - indicating that many patients would not be able to read these materials. Although the purpose of this study was not to evaluate the effectiveness of the methods utilized to assess text readability, the investigators found that the SMOG readability formula appears to provide a more accurate measurement than the Fry Graph method.

One key implication from this study is that health care providers, health educators, and diabetes educators should be cautious about the written information they provide to their patients. The written education materials disseminated to patients (at healthcare facilities) and to the general public (at health fairs) are intended for recipients who can read English proficiently. However, this study has demonstrated that the text in written health education materials can be hard to read. Therefore, further and more comprehensive research in this area is needed.

Finally, while not a direct focus of this study, the literature suggests that health literacy should be considered along with cultural competency (51-52) in developing written diabetes education materials.

Interest's conflicts: None.

Funding: California State University (Fresno).

REFERENCES

1. American Diabetes Association. Total prevalence of diabetes & pre-diabetes: Diabetes statistics. Retrieved February 15, 2010, from http://www.diabetes.org/diabetes-basics/diabetes-statistics/        [ Links ]

2. American Diabetes Association. All about diabetes: Pre-diabetes. Retrieved February 25, 2006, from http://www.diabetes.org/pre-diabetes.jsp        [ Links ]

3. Centers for Disease Control and Prevention. Leading Causes of Death. 2009. Retrieved February 15, 2010 , from http://www.cdc.gov/nchs/FASTATS/lcod.htm.        [ Links ]

4. National Diabetes Education Program. The facts about diabetes: America's seventh leading cause of death. Retrieved January 10, 2010, from http://ndep.nih.gov/diabetes-facts/index.aspx.        [ Links ]

5. National Institute of Diabetes and Digestive and Kidney Diseases. National Diabetes Statistics fact sheet: General information and national estimates on diabetes in the United States. Bethesda, MD: US Department of Health and Human Services; 2005.        [ Links ]

6. Huang ES, Basu A, O'Grady M, Capretta. J. Projecting the future diabetes population size and related costs for the U.S. Diabetes Care 2009; 32(12): 2225-2229.        [ Links ]

7. Centers for Disease Control and Prevention. National diabetes fact sheet: General information and national estimates on diabetes in the United States, 2003. Atlanta (GA): U.S. Department of Health and Human Services.        [ Links ]

8. Brown DR, Luwing R, Buck GA, Durham D, Shurmand T, Graham S. S Health literacy: Universal precautions needed. J Allied Health 2004; 33(2): 150-155.        [ Links ]

9. Hironak LK, Paasche-Orlow MK. The implications of health literacy on patient-provider communication. Arch Dis Chid 2008; 93(5):428-432.        [ Links ]

10. McCray AT. Promoting health literacy. J Am Med Inform Asso 2005; 12: 152-163.        [ Links ]

11. Baker DW, Schillinger D, Gazmararian J. A. The health care experience of patients with low literacy. Arc Family Prac 1996; 5(6): 329-334.        [ Links ]

12. Breese P, Burman W. Readability of notice of privacy forms used by major health care institutions. JAMA 2005; 293: 1593-1594.        [ Links ]

13. Center for Health Care Strategies. Health literacy fact sheets. January 15, 2010. http://www.chcs.org/publications3960/publications_show.htm?doc_id=291711        [ Links ]

14. Christopher PP, Foti ME, Roy-Bujnowski K, Appelbaum P. S. Consent form readability and educational levels of potential participants in mental health research. Psychiatr. Serv. 2007; 58: 227-232.        [ Links ]

15. Gazmararian JA, Williams MV, Peel J, Baker D. W. Health literacy and knowledge of chronic disease. Patient Education and Counseling 2003; 51(3), 267-275.        [ Links ]

16. Hussey LC. overcoming the clinical barriers of low literacy and medication non-compliance among the elderly. J of Geron Nursing 1991; 17(3), 27-29.        [ Links ]

17. Amini H, Casamassimo PS, Lin HL, Hayes JR. Readability of the American Academy of Pediatric Dentistry patient education materials. Pediatr Dent. 2007; 29(5):431-5.        [ Links ]

18. Hanchak NA., Patel MB, Berlin J. A Strom B. L. Patient misunderstanding of dosing instructions. J Gen Internal Medicine 1996; 11: 325-328.        [ Links ]

19. Weiss BD, Palmer R. Relationship between health care costs and very low literacy skills in medically needy and indigent Medicaid population. J Am Board of Fam Prac 2004; 17(1): 44-47.        [ Links ]

20. Davis T. C., Meldrum H., Tippy P. K. P., Weiss B. D. & Williams M. V. How poor literacy leads to poor health care. Patient Care 1996; 30(16): 94-106.        [ Links ]

21. Baker DW, Parker RM, Williams MV, Clark W. S, Nurss J. The relationship of patient reading ability to self-reported health and use of health services. Am J Public Health 1997; 87(6): 1027-1030.        [ Links ]

22. Moon RY, Cheng TL, Patel KM, Baumhaft K, Scheidt PC. Parental literacy level and understanding of medical information. Pediatrics 102(2). January 10, 2010. http://www.pediatrics.org/cgi/content/full/102/2/e25        [ Links ]

23. Kutner M, Greenberg E, Baer J. A first look at the literacy of America's adults in the 21st century: National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics; 2003.        [ Links ]

24. Kutner M, Greenber E, Jin Y, Paulsen C. The health literacy of American's adults: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: US Department of Education; 2006.        [ Links ]

25. Kirsch IS, Jungleblut A., Jenkins L, Kolstad A. Adult literacy in America: A first look at the findings of the National Adult Literacy Survey. 3rd ed. Washington, DC: National Center for Education Statistics; 2002.        [ Links ]

26. Institute of Medicine. Health literacy: A prescription to end confusion. Washington, DC: Author; 2004.        [ Links ]

27. Parikh NS, Parker RN, Nurss JR, Baker D. W, Williams MV. Shame and health literacy: The unspoken connection. Patient Education and Counseling 1996; 27: 33-39.        [ Links ]

28. Adkins AD, Elkins EN, Singh NN. Readability of NIMH easy-to-read patient education materials. J Child Fam Studies 2001; 10(3):279-285.        [ Links ]

29. Davis TC, Crouch MA, Wills G, Miller S, Abdehou D. M. The gap between patient reading comprehension and the readability of patient education materials. Journal of Family Practice 1990; 31(5): 533-538.        [ Links ]

30. French KS, Larrabee JH. Relationships among educational material readability, client literacy, perceived beneficence, and perceived quality. J Nursing Care Quality 1999; 13(6), 68-82.        [ Links ]

31. Wilson F. Are Patient Information Materials Too Difficult to Read? Home Healthcare Nurse 2000; 18(2): 107-115.        [ Links ]

32. Guidry JJ, Fagan P, Walker V. Cultural sensitivity and readability of breast and prostate printed cancer education materials targeting African Americans. J National Medical Association 1998; 90(3): 165-169.        [ Links ]

33. King MM, Winton A S W, Adkins A. D. Assessing the readability of mental health Internet brochures for children and adolescents. J Child and Family Studies 2003; 12(1): 91-99.        [ Links ]

34. Sellers DB, Thompson-Robinson M, Parra-Medina D, Wilcox S, Thompson NL, Will JC. Readability of educational materials targeting CVD risk factors in African American Women. Am J Health Studies 2003; 18(4): 188-195.        [ Links ]

35. Schillinger D, Grumbach K, Piette J, Wang F, Osmond D, Daher C, Palacios J, Sullivan GD, Bindman AB. Association of health literacy with diabetes outcomes. J the Am Medical Association 2002; 288(4):475-482.        [ Links ]

36. Jeppesen KM, Coyle JD, Miser WF. Screening questions to predict limited health literacy: A cross-sectional study of patients with diabetes mellitus. Ann Fam Med 2009; 7(1):24-31.        [ Links ]

37. Nath C. Literacy and diabetes self-management. Am J Nursing 2007; 107(6): 43-49.        [ Links ]

38. Parker FM, Baker DW, Williams MV, Nurss J R. The test of functional health literacy in adults: A new instrument for measuring patient's literacy skills. J General Internal Medicine 1995; 10(10): 537-541.        [ Links ]

39. Aldridge MD. Writing and designing readable patient education materials. Nephro-logy Nursing Journal 2004: 31(4): 373-377.        [ Links ]

40. Kahn A, Pannbacker M. Readability of educational materials for clients with cleft lip/palate and their families. Am J Speech-Language Pathology 200; 9: 3-9.        [ Links ]

41. Hill-Briggs, F. Evaluation of diabetes and cardiovascular disease print patient education materials for use with low-health literate populations. Diabetes Care 2008; 31(4): 667-671.        [ Links ]

42. Gal I, Prigat A. Why organizations continue to create patient information leaflets with readability and usability problems: An exploratory study. Health Educ Research 2004; 20(4): 485-493.        [ Links ]

43. Mancuso J. Assessment and measurement of health literacy: An integrative review of the literature. Nursing & Health Sciences 2009; 11(1): 77 - 89.        [ Links ]

44. Dollahite J. Thompson C, McNew R. Readability of printed sources of diet and health information. Patient Education and Counseling 1996; 27(2): 123-134.        [ Links ]

45. McLaughin G H. SMOG grading-A new readability formula. J Reading 1969; 12: 639-646.        [ Links ]

46. Cotugna N, Vickery CE, Carpenter-Haefele K. Evaluation of literacy level of patient education pages in health-related journals. J Community Health 2005; 30(3): 213-219.        [ Links ]

47. Hosey GM, Freeman W L, Stracqualursi F, Ghodes D. Designing and evaluating diabetes education materials for American Indians. The Diabetes Educator 1990; 16: 407-414.        [ Links ]

48. Jones J. Patient illiteracy. AORN 2007; 85(5): 951-955.        [ Links ]

49. Scott T L, Gazmararian JA, Williams MV, Baker D. Health literacy and preventive health care use among Medicare enroll-ees in a managed care organization .2002; Medical Care 40(5): 395-404.        [ Links ]

50. Shaw S, Huebner C, Armin J, Orzech K, James V. The Role of Culture in Health Literacy and Chronic Disease Screening and Management. Journal of Immigrant and Minority Health 2009; 11(6): 1557-1912.        [ Links ]

51. Friedman B, Kaos E. A Comprehensive Assessment of the Difficulty Level and Cultural Sensitivity of Online Cancer Prevention Resources for Older Minority Men Preventing Chronic Diseases. Public Health Research Practice and Policy 2008; 5(1): 1-12.        [ Links ]

52. Zancheta MS, Poureslami I. Health literacy within the reality of immigrant's culture and language. Canadian Journal of Public Health 2006; 97 (Supplement 2): 26-30.        [ Links ]