In this Blog:
Think of the last time you gave a patient, community member, or study participant something to fill out. Maybe it was a questionnaire for public health surveillance or a research study, or perhaps you were collecting patient-reported outcomes or intake paperwork and forms. Maybe you developed a list of interview questions to ask verbally.
Questionnaires, forms, and surveys that are easy to understand and use can boost response rates and give you robust information with which to make decisions.1 But what happens if your audience doesn’t understand the questions or how to use the materials you want them to engage with?
You get bad data.
Inaccurate data collected for a research study could result in safety issues, displeased funders, and ultimately missed opportunities to advance your research agenda. In clinical care, inaccurate data collected via patient surveys can impact care and outcomes. In value-based reimbursement programs, bad data impacts the bottom line. If your work is in public health surveillance, inaccuracies could prevent you from truly understanding the needs of communities and designing impactful programs.
Why Questionnaires Need Different Assessments
As health literacy professionals, you’re likely familiar with the skills and tools needed to craft and assess health education and communication materials.
The health literacy professional’s typical tools to evaluate health materials aren’t validated for these types of communication. But there are a few ways you can check if your surveys and forms are easy to read, understand, and use:
- Readability Formulas. For any narrative text, such as an introduction or instructions, you can use your regular formulas to assess readability. You might also consider using a formula that is validated for non-narrative materials, such as FORCAST, 2 which was developed in the 1970s. FORCAST is ideal for assessing this type of content since it does not rely on complete sentences and instead measures the number of single-syllable words in a sample. The results of this formula are presented in U.S. grade-level terms and can come out a bit higher than those of other formulas.
- Understandability and Actionability Tools. In addition to readability, we also need to assess whether people understand and can act on these materials. Our normal tools to assess content, such as the Patient Education Materials Assessment Tool (PEMAT), 3 aren’t intended for surveys, forms, and questionnaires. You might consider using the ELF-Q 4 (which is short for Evaluative Linguistic Framework for Questionnaires). This framework is based on the ELF, which was developed to assess the quality of written health information using linguistic theory (how language works).5 While the ELF-Q is not as easy to use as the PEMAT (there is no user guide, for example), it does comprise 9 areas to assess the quality of questionnaires. Note that the ELF-Q was adapted to evaluate the quality of self-reported questionnaires, but there are elements that can inform questionnaires that are administered verbally as well.
- Audience Feedback. Another way you can ensure questionnaires (and other materials) are suitable for your audience is to ask them. But first, think deeply about the diverse people your audience may include and engage members who represent your audience, including those who may be likely to struggle with health information. (That group of retired nurses or your coworkers may give you useful feedback, but their perspectives and abilities could be different from other members of your audience!) Don’t underestimate the importance of getting feedback from potential users. This approach not only helps improve the material at hand, but may also teach you lessons that you can apply more broadly to future work. Engaging your audience is really a valuable experience.
How to Create Better Questionnaires, Surveys, and Forms
Moving forward, here are some specific strategies, established by the ELF-Q, that you can consider when writing and editing questionnaires:
- Establish and write a clear purpose for your questionnaire, form, or survey so respondents can easily and quickly determine if they should fill it out.
- Provide clear instructions on how to navigate questions and answer them. Often, instructions need to be included at the beginning of the material, and again as topics or response options change.
- Write short, easy-to-follow questions. Avoid double-barreled questions (in which 2 questions are asked at once) or questions that are overly complex or ambiguous.
- Ensure response options properly align with questions. Consider how the respondent may answer a question and whether response options are distinguishable. (The difference between “somewhat” and “a little” is probably not big enough.)
- Let formatting do some of the work in making questions easy to answer. As appropriate, use numbers, tables, and large font to present questions clearly and logically.
- Avoid overly long questionnaires.
Improve Survey Results by Focusing on Health Literacy
The ability to read and interpret questionnaires and other data-collection materials to provide accurate information is a critical health literacy skill. When people do not understand how to fill out a form or answer a survey question, this uncertainty or confusion introduces bias in the data we collect. It can even frustrate respondents to the point they don’t complete the survey or form.
As health literacy professionals, we must lower the barriers we place on individuals to fill out these forms by using clear communication strategies and appropriate tools and readability formulas to assess their quality. This supports the people we serve as they complete the forms and yields more accurate data and information for the solutions we deliver.
1Anhang Price, R., Quigley, D. D., Hargraves, J. L., Sorra, J., Becerra-Ornelas, A. U., Hays, R. D., Cleary, P. D., Brown, J., & Elliott, M. N. (2022). A Systematic Review of Strategies to Enhance Response Rates and Representativeness of Patient Experience Surveys. Medical care, 60(12), 910–918. https://doi.org/10.1097/MLR.000000000000178
2Caylor, J.S., Stitch, T.G., Fox, L.C. and Ford, J.P. (1973) Methodologies for Determining Reading Requirements of Military Occupational Specialties: Technical Report No. 73-5. Human Resources Research Organization, Alexander.
3The Patient Education Materials Assessment Tool (PEMAT) and User’s Guide. Content last reviewed November 2020. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/health-literacy/patient-education/pemat.html
4Clerehan, R., Guillemin, F., Epstein, J., & Buchbinder, R. (2016). Using the Evaluative Linguistic Framework for Questionnaires to Assess Comprehensibility of Self-Report Health Questionnaires. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research, 19(4), 335–342. https://doi.org/10.1016/j.jval.2016.01.008
5Rosemary Clerehan, Rachelle Buchbinder, Jane Moodie, A linguistic framework for assessing the quality of written patient information: its use in assessing methotrexate information for rheumatoid arthritis, Health Education Research, Volume 20, Issue 3, June 2005, Pages 334–344, https://doi.org/10.1093/her/cyg123
About the Author
Katie Leath is the director of the Center for Health Literacy at the University of Arkansas for Medical Sciences (UAMS). She is a passionate health communicator with nearly a decade of experience in making health information easy to read, understand, and use. She leads the team of health literacy experts at the UAMS Center for Health Literacy, where she has served in various capacities, in providing services to address health literacy at UAMS and beyond.