Identifying Health Literacy in Kansas Using the Behavioral Risk Factor Surveillance System (BRFSS)
Amy K Chesser1*, Trisha V Melhado2, Robert B Hines2 and Nikki Keene Woods1
1Wichita State University, USA
2University of Kansas School of Medicine -Wichita, USA
*Corresponding author: Dr. Amy Chesser, Wichita State University, 1845 Fairmount St, Box 43, Wichita, KS 67260, USA, Tel: 316-978-3060, E-mail: email@example.com
J Fam Med Dis Prev, JFMDP-2-032, (Volume 2, Issue 2), Short Communication; ISSN: 2469-5793
Received: April 27, 2016 | Accepted: May 30, 2016 | Published: June 02, 2016
Citation: Chesser AK, Melhado TV, Hines RB, Woods NK (2016) Identifying Health Literacy in Kansas Using the Behavioral Risk Factor Surveillance System (BRFSS). J Fam Med Dis Prev 2:032. 10.23937/2469-5793/1510032
Copyright: © 2016 Chesser AK, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Introduction: Despite the increasing recognition of the importance of health literacy, there is a lack of evidence regarding the use of a brief health literacy assessment for population-based, state-led surveys. The objective of this study was to examine the feasibility of adding a health literacy measurement tool to the Kansas Behavioral Risk Factor Surveillance System (BRFSS).
Methods: A cross-sectional research design was used. Health literacy data were extracted from the state-specific module of the BRFSS telephone survey. Demographic and health status variables were extracted from the core BRFSS dataset. The association between demographic and health status characteristics with health literacy were obtained using weighted samples in multivariable logistic regression models.
Results: Incorporation of the tool into the BRFSS survey was deemed a viable option for measurement of health literacy in Kansas as well as other states. Most respondents had moderate health literacy (61.6%), followed by high (29.8%), and low (8.6%) health literacy. Demographic variables associated with health literacy were race, education, employment status, income, gender, and age. General health status, chronic conditions, activity limitations, and poor physical and mental health were also associated with health literacy.
Conclusions: A BRFSS-based health literacy survey can be a valuable tool for evaluating health literacy at the population level. Identifying subgroups with lower health literacy has implications for improving population health through health literacy promotion and education.
Despite the increasing recognition of the importance of health literacy, there is a lack of current knowledge on population-based assessments . Limited health literacy is associated with poor health and adverse health outcomes [2-7]. Literacy rates for the state of Kansas have been estimated only for basic prose literacy skills with approximately 8% (n = 164,000) of the population lacking these fundamental skills . Prose literacy refers to general text comprehension and does not measure the more nuanced health literacy. To date, no statewide health literacy assessment has been conducted using the Kansas Behavioral Risk Factor Surveillance System (BRFSS) survey. A population-level assessment would allow public health educators in our state and others to target segments of the population to provide support services. Additionally, health care providers and stake-holders could establish health literate organizations [9,10] to be placed across the globe in areas with a high need for health literacy interventions and programs. In order to identify segments of the population that could benefit most from interventions and services, population-based assessments have to be conducted.
Several health literacy assessment tools have proven valid and reliable and have been established within the literature . The most commonly used health literacy assessment instruments include: The Wide Range Achievement Test - Revised (WRAT-R) , the Rapid Estimate of Adult Literacy in Medicine (REALM) , the Test of Functional Health Literacy in Adults (TOFHLA) , and the Newest Vital Sign (NVS) . However, these instruments are time-consuming as they are usually given in person, and there is often an expense for required staff training to administer the test. Several researchers have tested and validated a brief screening tool to identify patients with low health literacy using an in-patient population and three health literacy questions on a 5-point Likert scale [2,16-20].
The objective of this study was to examine the feasibility of adding a health literacy screening tool to the Kansas Behavioral Risk Factor Surveillance System (BRFSS).This was a Kansas-specific study. Additionally, we were interested in examining the prevalence of health literacy and the association with demographic and health status variables.
The study population was comprised of Kansas residents who participated in the 2012 Kansas BRFSS and completed the health literacy assessment portion of the survey. The BRFSS is a randomly sampled national telephone survey conducted by the Centers for Disease Control and Prevention (CDC) which collects self-reported information about health-related risk behaviors, preventive service use, and chronic health conditions for adults aged 18 years and older. Detailed survey methodology is available from the CDC . This study was approved by the Kansas Department of Health and Environment (KDHE) Institutional Review Board (IRB) and the University of Kansas IRB (HSC#22031598).
Health literacy instrument
In preparation for the large-scale feasibility assessment, the instrument was cognitively tested (to assess general understanding of each question by telephone participants) with a convenience sample which confirmed the usability of the 3-item scale via telephone. Approval was obtained to include the adapted 3-item scale in the optional Kansas-specific module of the 2012 BRFSS. A cross-sectional research design was used. Health literacy data were collected through the supplemental, state-specific module (a.k.a. arm) of the BRFSS telephone survey.
The primary outcome of interest for this study was health literacy. The 2012 Kansas BRFSS state module included a health literacy screening developed by Chew et al. which consists of three health literacy questions [16,17,19]. Specifically, the health literacy tool assessment consisted of the following three questions and responses on a 5 point Likert scale:
1. How confident are you in filling out medical forms by yourself? For example insurance forms, questionnaires, and doctor's office forms. Would you say...
Possible responses were: 1) Not at all, 2) A little, 3) Somewhat, 4) Quite a bit, 5) Extremely, 7) Don't know/not sure, 9) Refused.
2. How often do you have problems learning about your health condition because of difficulty in understanding written information? Would you say...
Possible responses were: 1) Always, 2) Often, 3) Sometimes, 4) Rarely, 5) Never, 6) Never visited doctor's office, 7) Don't know/not sure, 9) Refused.
3. How often do you have someone help you read medical materials? For example: family member, friend, caregiver, doctor, nurse or other health professional. Would you say...
Possible responses were: 1) Always, 2) Often, 3) Sometimes, 4) Rarely, 5) Never, 7) Don't know/not sure, 9) Refused.
The independent variables included demographics and health status. The dependent variable was health literacy . Cumulative scores were calculated for responders who answered all three health literacy questions. Health literacy responses were categorized as follows: low health literacy (scores 3-8), moderate health literacy (scores 9-14) and high health literacy (score = 15). Cut-off points were determined in consultation with published literature and in consultation with the authors [16,17,19]. All analyses utilized this trichotomized health literacy variable as the outcome rather than a scaled score .
All data were analyzed using IBM SPSS Statistics version 20 for complex samples. Chi-square tests were used to evaluate the significance of bivariate associations for each demographic and health status variable with health literacy. Associations that met entry criteria (p ≤ 20) were included in the initial multivariable model for complex samples . A polytomous logistic regression model was used to obtain the adjusted relative odds for the association between the demographic variables with moderate or high health literacy compared to low health literacy. A separate polytomous regression model assessed the association between health status and health literacy.
Among the 11,801 responders of the BRFSS survey, 5,638 participants were asked the Kansas-specific supplemental questions including the health literacy questions and 5,494 responders answered all three health literacy questions (response rate = 97.5%). As shown in table 1, most residents of Kansas reported moderate health literacy (61.6%), followed by high health literacy (29.8%), and low health literacy (8.6%). The participants of this study were representative of the state of Kansas in terms of sex, age, and ethnicity as reported by the U.S. Census Bureau.
Table 1: Weighted demographic and health status characteristics of Kansas BRFSS 2012 sample in relation to their health literacy (HL) (N = 2,015,525). View Table 1
A higher proportion of racial/ethnic minority groups reported low health literacy (Black 9.7% vs. 4.0% high health literacy; Hispanic 15.7% vs. 5.8% high health literacy). Home owners represented the majority of all health literacy groups, and the relative proportion increased with increasing health literacy. Most Kansans had health care insurance coverage, and the proportion increased with increasing health literacy. Residents in rural areas comprised the majority of the low health literacy group (44.0%) whereas those living in MSA counties with large cities represented the largest proportion of those with high health literacy (38.1%). The majority of those taking the survey did so in English, and the relative proportion of those taking the survey in English increased with increasing health literacy.
Respondents ≥ 65 years of age represented the largest proportion of the low health literacy group (23.2%) whereas young adults and the middle-aged were equally represented in the high health literacy group. Most respondents had chronic conditions although the relative proportion by health literacy declined with increasing health literacy. Residents without activity limitations were the majority across the three levels of health literacy, and their relative proportion increased with increasing health literacy. A similar pattern was observed for mental health, although those who reported no poor mental health days comprised the majority of all health literacy groups.
In table 2, adjusted associations between the demographic variables and the outcomes of moderate/high health literacy vs. low health literacy are depicted. Men had a 46.0% decreased odds (OR = 0.54, 95% CI 0.37-0.78) of having high health literacy. Compared to residents ≥ 65 years, those 25 to 34 years had a 2.6-fold increased odds (OR = 2.61, 95% CI 1.24-5.47) of having high health literacy and adults 35 to 44 years had 2.1 times the odds (OR = 2.11, 95% CI 1.00-4.46). Non-Hispanic blacks had a 50% decreased odds (OR = 0.50, 95% CI 0.25-0.99) of having moderate health literacy. The non-Hispanic other race category also had a 74.0% decreased odds (OR = 0.26, 95% CI 0.11-0.62) of having moderate health literacy.
Table 2: Kansas BRFSS 2012 sample: Adjusted associations between demographic variables and health literacy (HL)*. View Table 2
For education, the adjusted odds ratios for the relative odds of having moderate and high health literacy for high school graduates, those with some college/technical school, and college/technical school graduates were all significant compared to those who did not graduate from high school. For employment status, compared to those with full-time employment, the only relationship that was significantly associated with health literacy was for those who were unable to work. These individuals had a 75-77% decreased odds of having moderate and high health literacy, respectively. Compared to residents with an annual household income ≥ $50,000, those making < $15,000 an 84% decreased odds (OR = 0.16, 95% CI 0.08-0.31) of having high health literacy.
The adjusted associations between the health status variables and health literacy are displayed in table 3. Compared to those with a general health rating of poor/fair health, those with good/better health had 2.4 times the odds (OR = 2.41, 95% CI 1.51-3.87) of having high health literacy. Those with chronic conditions 53% decreased odds (OR = 0.47, 95% CI 0.28-0.78) of having high health literacy. Compared to residents with activity limitations, those without limitations had 2.1 times the odds (OR = 2.09, 95% CI 1.39-3.14) of having high health literacy.
Table 3: Kansas BRFSS 2012 sample: Associations between health status variables and health literacy*^. View Table 3
This study was novel in its approach to use the three question health literacy screening tool as part of the statewide BRFSS survey. The addition of the screening questions as part of the BRFSS survey for the state of Kansas proved feasible. Furthermore, this study reinforces the findings that race, education, employment, income, sex, age, and health status are related to health literacy. There is a body of published literature investigating health literacy and health status for a variety of populations [24,25]. In this study, black respondents and those of other racial/ethnic categories had decreased odds of moderate and high health literacy which is consistent with the literature .
Not surprisingly, increasing educational attainment was also associated with increased odds of higher health literacy as others have reported [27-29]. Moreover, those who were unable to work and those of lower income had decreased odds of moderate/high health literacy which is consistent with other research . Men had decreased odds of having high health literacy. This is mostly likely due to the fact that women's access and utilization of health care system is higher than that of men . Younger residents, those 25-44, were found to have increased odds of having high health literacy than patients 65 years of age and older which may be due to younger residents being more accustomed to comprehending novel information and the cognitive decline associated with aging. Also, younger patients may not be in the health care system as much as older patients since younger populations tend to be healthier than older populations. Therefore, the lack of contact with the health care system among younger patients may cause them to overestimate their health literacy. Interestingly, some of the demographic variables that we hypothesized may be associated with health literacy were not in significant the final model including marital status, home ownership, rurality, and language. These demographic variables are of interest and should continue to be incorporated for longitudinal study analysis. Mediators of these relationships were likely captured through the measures of education and income.
Considering health status, people who reported good or better health, had no activity limitations due to health, did not require special equipment for their health problems and had no poor physical or mental health in the past month tended to have higher health literacy than their comparison groups. The health status variables are not novel to the BRFSS dataset; however, the association of the health status measures in relation to health literacy is unique. The association among health status and health literacy may be similar to that observed in the younger responders and health literacy in that healthier responders may not be in the health care system as much as their sicker counterparts who report poor health, activity limitations, use of special equipment, or poor physical and mental health. On the other hand, the association between health status and health literacy may be due to the responders having higher health literacy which, in turn, influenced their behavior and willingness to engage in healthy/preventive behaviors. In contrast, those in poor health and therefore having greater contact with the health care system had lower health literacy. This finding accentuates the importance of providers communicating health information in a manner that is understandable to patients. This is particularly important for patients with chronic diseases that require ongoing management to prevent complications and worsening health status. People with chronic conditions have multiple treatment regimens and office visits. Often, such patients rely on their providers to manage their illness because the information can become overwhelming [32-34]. Speaking to patients at their level of health literacy or communicating health information at a low literacy level to all patients is especially relevant for those patients managing chronic conditions. Ensuring understanding and empowering patients to manage their chronic disease could decrease complications and, in turn, the number of office visits and costs for numerous chronic conditions.
In this study, we used a three-item health literacy screening instrument to assess health literacy on a population level. This tool has been used in a wide range of patient populations although none have conducted a population-level health literacy assessment. However, health literacy continues to be a difficult construct to measure. To date, there is no one instrument that measures all aspects of health literacy. The three-item health literacy screening questions were selected for this study because of feasibility considerations. The questions provide a brief self-assessment of health literacy, are a viable option for a telephone survey, and provide a quick method to screen for health literacy on a population level. The three question screener is a validated tool that is growing in its use as a practical assessment of health literacy in research and clinical settings [16,17,19,20,35-37]. The use of more comprehensive assessments were precluded in this study as they take longer to complete, require more staff and participant time, and are not practical as a screening tool for population-based assessment.
This is the first study to incorporate a health literacy measure as part of a statewide BRFSS survey. Few studies have investigated health literacy with rural populations in the Midwest United States. This is also the first large-scale, population-based study to utilize the health literacy assessment created by Chew et al. . One study using a similar screening question with a large sample was recently published. The study used combined data from the Hawai'i Health Survey (HHS) years 2008 and 2010. Nearly 12,000 respondents across 37 communities completed the survey. Health literacy was measured using a validated self-reported question, "How confident are you filling out medical form by yourself?" Results indicated 18.2% self-reported low health literacy, and 14.7% self-reported having poor health. Those that self-reported poor health were significantly more likely to report: low health literacy, less education, older age, male gender, lower income, and unmarried marital status . Based on the findings from our study, incorporating a three question health literacy screening assessment is a viable option for other states to include in their state-specific BRFSS module. Our results indicate there are multiple demographic and health status characteristics that are related to health literacy. This has implications for the clinical setting where screening for health literacy may help facilitate better communication between the patient and health care providers. Our results also have ramifications for population health. By identifying factors associated with lower health literacy, public health interventions and programs could be designed to improve health literacy for these at-risk groups.
The three-item health literacy assessment used in this study identified sociodemographic factors and health status indicators that were associated with health literacy. However, the evidence supporting the use of a health literacy screening tool has yet to be established. Paasche-Orlow and Wolf (2008) acknowledged some possibility of harm to patients from shame and stigma and found little support to promote patient health literacy screening . Notifying physicians of patient health literacy has produced mixed results as well. In one study, physicians reported less satisfaction with their clinical encounter, and there were no differences in measured health outcomes . The model for success may be found in a combination of informing health care professionals about the prevalence of low health literacy in the population, launching public health campaigns aimed at improving health literacy, and utilizing public health organizations to provide community-level education and support services to residents. The methodology used in this study allows researchers and health care providers to begin to create and implement multi-level solutions. Improving health literacy could have downstream effects on many health outcomes and disease conditions from improved maintenance of chronic health conditions to reduced adverse effects from medication errors.
This study utilized cross-sectional data from one Midwestern state with validated methodology [41,42]. Several limitations should be considered. Self-reported survey data are subject to possible response bias, although self-reported health literacy rates have been demonstrated as reliable for a variety of skills . Due to the length, survey fatigue may have also affected responses. Furthermore, due to the nature of the screening test (only three items), the ability to objectively assess health literacy gaps was not possible. Additionally, as part of the study design, data were limited to specific questions that comprise the BRFSS module. Due to the proscribed nature of the questions included on the survey, exploration of additional questions that might further contribute to the findings was not possible. Finally, the cross-sectional nature of the data prohibits making causal judgments and study results may not be generalizable to other states with differing demographic characteristics.
Notwithstanding these limitations, this study has demonstrated the feasibility of assessing health literacy by incorporating a three-item health literacy assessment into an optional state-specific module of the BRFSS. Additionally, health literacy rates for the state have been identified and associations with variables typically collected through the BRFSS survey are confirmed. The implications of this study's findings are relevant to both health practitioners and researchers. Interventions and programs directed at multiple levels are necessary to improve health literacy and the poorer outcomes associated with a low health literate population. Assessing health literacy at the population-level is a necessary tool for health practitioners as this knowledge is vital to ensure healthcare accessibility and the successful use of health promotion and educational information.
Funding provided by the University of Kansas School of Medicine – Wichita Department of Family and Community Medicine, KDHE and the Kansas BRFSS as the source of such data, and partial support from the Health Resources and Services Administration Grant # D58HP20802 made this study possible.
Perlow E (2010) Accessibility: global gateway to health literacy. Health Promot Pract 11: 123-131.
Peterson PN, Shetterly SM, Clarke CL, Bekelman DB, Chan PS, et al. (2011) Health literacy and outcomes among patients with heart failure. JAMA 305: 1695-1701.
Sentell T, Baker KK, Onaka A, Braun K (2011) Low health literacy and poor health status in Asian Americans and Pacific Islanders in Hawai'i. J Health Commun 16 Suppl 3: 279-294.
Al Sayah F, Majumdar SR, Egede LE, Johnson JA (2015) Associations between health literacy and health outcomes in a predominantly low-income africanamerican population with type 2 diabetes. J Health Commun 20: 581-588.
Hahn EA, Burns JL, Jacobs EA, Ganschow PS, Garcia SF, et al. (2015) Health Literacy and Patient-Reported Outcomes: A Cross-Sectional Study of Underserved English- and Spanish-Speaking Patients With Type 2 Diabetes. J Health Commun 20 Suppl 2: 4-15.
Sarkar M, Asti L, Nacion KM, Chisolm DJ (2015) The Role of Health Literacy in Predicting Multiple Healthcare Outcomes Among Hispanics in a Nationally Representative Sample: A Comparative Analysis by English Proficiency Levels. J Immigr Minor Health 3: 608-615.
Wu JR, Moser DK, DeWalt DA, Rayens MK, Dracup K (2016) Health Literacy Mediates the Relationship Between Age and Health Outcomes in Patients With Heart Failure. Circ Heart Fail 9: e002250.
Leyla Mohadjer, Graham Kalton, Tom Krenzke, Benmei Liu, Wendy Van de Kerckhove, et al. (2009) National assessment of adult literacy: Indirect county and state estimates of the percentage of adults at the lowest literacy level for 1992 and 2003, NCfES US Department of Education, Editor, Washington, DC.
How Can Health Care Organizations Become More Health Literate: Workshop Summary (2012) National Academies Press, Washington (DC).
Brach C, Dreyer BP, Schillinger D (2014) Physicians' roles in creating health literate organizations: a call to action. J Gen Intern Med 29: 273-275.
Hahn EA, Du H, Garcia SF, Choi SW, Lai JS, et al. (2010) Literacy-fair measurement of health-related quality of life will facilitate comparative effectiveness research in Spanish-speaking cancer outpatients. Med care 48: S75-82.
Lawson JR, Al D (1962) Comparison of Wide Range Achievement Test and Gray Oral Reading Paragraphs reading scores of mentally retarded adults. Percept Mot Skills 14: 474.
Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, et al.(1993) Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med 25: 391-395.
Parker RM, Baker DW, Williams MV, Nurss JR (1995) The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med 10: 537-541.
Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, et al. (2005) Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med 3: 514-522.
Chew LD, Bradley KA, Boyko EJ (2004) Brief questions to identify patients with inadequate health literacy. Fam Med 36: 588-594.
Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, et al. (2008) Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med 23: 561-566.
Wallace LS, Cassada DC, Rogers ES, Freeman MB, Grandas OH, et al. (2007) Can screening items identify surgery patients at risk of limited health literacy? J Surg Res 140: 208-213.
Chew LD, Bradley KA, Flum DR, Cornia PB, Koepsell TD (2004) The impact of low health literacy on surgical practice. Am J Surg 188: 250-253.
Sarkar U, Schillinger D, López A, Sudore R (2011) Validation of self-reported health literacy questions among diverse English and Spanish-speaking populations. J Gen Intern Med 26: 265-271.
CDC. Behavioral Risk Factor Surveillance System Survey Data.
Chesser A, Jared Y, Kyle S, Nikki Keene W (2014) Integrating Health Literacy Questions into a Statewide Behavioral Risk Factor Surveillance System (BRFSS) Questionnaire. KUMC 7: 96-103.
Bursac Z, Gauss CH, Williams DK, Hosmer DW (2008) Purposeful selection of variables in logistic regression. Source Code Biol Med 3: 17.
Adams RJ, Piantadosi C, Ettridge K, Miller C, Wilson C, et al. (2013) Functional health literacy mediates the relationship between socio-economic status, perceptions and lifestyle behaviors related to cancer risk in an Australian population. Patient EducCouns 91: 206-212.
Wolf MS, Feinglass J, Thompson J, Baker DW (2010) In search of 'low health literacy': threshold vs. gradient effect of literacy on health status and mortality. SocSci Med 70: 1335-1341.
Guerra CE, Shea JA (2007) Health literacy and perceived health status in Latinos and African Americans. Ethn Dis 17: 305-312.
Schillinger D, Barton LR, Karter AJ, Wang F, Adler N (2006) Does literacy mediate the relationship between education and health outcomes? A study of a low-income population with diabetes. Public Health Rep 121: 245-254.
Kaphingst KA, Kreuter MW, Casey C, Leme L, Thompson T, et al. (2012) Health Literacy INDEX: development, reliability, and validity of a new tool for evaluating the health literacy demands of health information materials. Journal Health Commun 17: 203-221.
Sentell T (2012) Implications for reform: survey of California adults suggests low health literacy predicts likelihood of being uninsured. Health Aff (Millwood) 31: 1039-1048.
Mitchell SE, Sadikova E, Jack BW, Paasche-Orlow MK (2012) Health literacy and 30-day postdischarge hospital utilization. J Health Commun 17 Suppl 3: 325-338.
Bertakis KD, Azari R, Helms LJ, Callahan EJ, Robbins JA (2000) Gender differences in the utilization of health care services. J Fam Pract 49: 147-152.
Manning DL, Dickens C (2006) Health literacy: more choice, but do cancer patients have the skills to decide? Eur J Cancer Care (Engl) 15: 448-452.
Merriman B, Ades T, Seffrin JR (2002) Health literacy in the information age: communicating cancer information to patients and families. CA Cancer J Clin 52: 130-133.
Kim SP, Knight SJ, Tomori C, Colella KM, Schoor RA, et al. (2001) Health literacy and shared decision making for prostate cancer patients with low socioeconomic status. Cancer Invest 19: 684-691.
Johnson TV, Abbasi A, Kleris RS, Ehrlich SS, Barthwaite E, et al. (2013) Assessment of single-item literacy questions, age, and education level in the prediction of low health numeracy. JAAPA 26: 50-54.
Cordasco KM Homeier DC, Franco I, Wang PC, Sarkisian CA (2012) Health literacy screening of geriatric monolingual Spanish-speaking patients using single-item literacy screening questions and education. Health Educ J 71: 597-605.
Miller MJ, Allison JJ, Schmitt MR, Ray MN, Funkhouser EM, et al. (2010) Using single-item health literacy screening questions to identify patients who read written nonsteroidal anti-inflammatory medicine information provided at pharmacies. J Health Commun 15: 413-427.
Sentell T, Zhang W, Davis J, Baker KK, Braun KL (2014) The influence of community and individual health literacy on self-reported health status. J Gen Intern Med 29: 298-304.
Paasche-OrlowMK, Wolf MS (2008) Evidence does not support clinical screening of literacy. J Gen Intern Med 23: 100-102.
Seligman HK, Wang FF, Palacios JL, Wilson CC, Daher C, et al.(2005) Physician notification of their diabetes patients' limited health literacy. A randomized, controlled trial. J Gen Intern Med 20: 1001-1007.
Nelson DE, Holtzman D, Bolen J, Stanwyck CA, Mack KA (2001) Reliability and validity of measures from the Behavioral Risk Factor Surveillance System (BRFSS). SozPraventivmed 46: S3-42.
Kohlmann T (2001) BRFSS and beyond: methodological quality of the measures from the Behavioral Risk Factor Surveillance System. SozPraventivmed 46: S1-2.