How Reliable Are Large Language Models for Health Literacy?

An Artificial Intelligence Practitioner’s Perspective on Trustworthiness and Responsibility, Part 2

By Temese Szalai | November 30, 2023

Read more in the series:

Part 1 | Can — and Should — We Trust Large Language Models for Health Literacy? | Publish Date: November 28, 2023

Part 3 | When LLMs May Not Be Appropriate for Health Literacy | Publish Date: December 5, 2023

Part 4 | Will LLMs Ever Really Be Trustworthy Enough for Health Literacy? | Publish Date: December 7, 2023

Today, we’ll continue our discussion of trustworthiness of large language models (LLMs) for health literacy by considering the reliability of outputs for health literacy purposes. This is the second post in a series inspired by a recent discussion on the IHA Health Literacy Solutions Center discussion list about the implications of artificial intelligence (AI) for the health literacy field.

As an AI practitioner, I believe this topic of reliability is especially relevant to health literacy professionals. Several people in the discussion list commented on issues surrounding accuracy, reliability, and suitability of AI chatbot outputs for health literacy. This post is intended to provide some insight on how we can think about reliability of LLMs for health literacy purposes today. 

Are LLMs the Right Tool for Health Literacy?

When considering the reliability of LLMs in various contexts, it’s important to think about what they were designed to do vis-a-vis the context at hand. It means asking, “Is this the right tool for the job?”

Tools like ChatGPT, Google Bard, Anthropic’s Claude, and Meta’s Llama 2, often called “foundational models,” were designed to help develop and understand general content. They can answer general user questions, generate business plans for cat cafés, write project descriptions for grant applications, and translate field values in a JavaScript Object Notation (JSON) file from English to French.

These foundational models were not designed for any specific domain. Importantly, they were not designed to be experts on medical content or health literacy. So, they aren’t the perfect tool for health literacy, even though they may still be a useful tool in some circumstances.

Generating Suitable Health Literacy Outputs

Further, LLMs are designed to mimic the data they were trained on. For foundational LLMs, this is generally crawlable digital data ¹ such as that found in the Common Crawl, the BookCorpus, and similar sources. 

The average reading level of these inputs is roughly college sophomore level, which results in outputs at a similar level. While recent research by Dr. Hana Haver et al., which is summarized in this post, indicates that ChatGPT can simplify responses to patient questions about lung cancer screening to about a 12th-grade level, outputs at this level aren’t typically simple enough for health literacy’s purposes.

As such studies, and many experiences described by folks in the discussion list, indicate, when we apply these tools to health literacy use cases, we may be disappointed with their results. Maybe the reading level is higher than it should be, or something simple is turned into something complex.

Even with healthcare-specific tools, like Grafi.ai, we may find their outputs unsatisfactory for health literacy’s purposes. Most of these tools are not designed with health literacy in mind. In fact, Grafi.ai was designed to help copywriters develop health content especially for search engine optimization (SEO).

As far as accuracy goes, at least one initial study concluded that “ChatGPT generated largely accurate information to diverse medical queries as judged by academic physician specialists, although with important limitations.” So, while suitably health literate content might not be generated, there’s a small ray of hope regarding medical accuracy.

^{1. “There’s controversy surrounding how public this crawlable data is and whether it is being used legally and legitimately. For this reason, we’ve opted to avoid the term “public” data and instead refer to “crawlable” data.↩}

#IHABlog
#ArtificialIntelligence