Health literacy is pivotal in enhancing individual agency, enabling informed choices regarding health matters, effective interaction with healthcare systems, and managing their well-being. Initially defined by basic reading and writing skills, the concept now encompasses higher-order abilities such as critical thinking, decision-making, and effective communication (1). Low health literacy has been consistently linked to adverse outcomes, including poor treatment adherence and increased mortality (2). Studies reveal that nearly half of U.S. adults—and a substantial portion of Europeans—struggle to comprehend health information, thereby contributing to disparities in healthcare access and utilization (3). In response, organizations such as the World Health Organization have advocated for improved health communication, developing literacy-friendly environments, and comprehensive policies to enhance health literacy. Despite significant research efforts, gaps remain in our understanding of how low health literacy exacerbates disparities and which interventions are most effective, underscoring the need for further exploration of screening methods, communication strategies, and the underlying causal mechanisms (4).

In rheumatology, health literacy plays a critical role in patient care by influencing disease management, treatment adherence, and overall health outcomes (5). For instance, approximately one in seven patients with rheumatoid arthritis (RA) may lack the skills required to engage in informed decision-making, potentially leading to suboptimal disease control (6). Limited health literacy in rheumatic conditions such as RA and systemic lupus erythematosus (SLE) has been associated with higher disease activity and reduced understanding of self-management strategies (5, 7). Patients with inadequate health literacy often encounter difficulties with complex treatment regimens, resulting in medication non-adherence and a reluctance to modify therapy when necessary (8). These issues are particularly pronounced among vulnerable groups—such as older adults, ethnic minorities, and individuals of lower socioeconomic status—who are more likely to experience limited health literacy (5). Consequently, these patients are at an increased risk of poorer outcomes and may have restricted access to advanced treatments, including biologic disease-modifying antirheumatic drugs (DMARDs) (8). Addressing health literacy through targeted patient education and tailored communication is crucial to improving patient engagement and optimizing treatment adherence in rheumatologic care.

The American Medical Association (AMA) and the National Institutes of Health (NIH) suggested that patient-directed health materials be written at a sixth- to eighth-grade reading level (9). This guideline is critical in the digital era since most U.S. population accesses health information online. Despite these recommendations, research shows that much online rheumatology-related health content, including materials produced by academic institutions, often exceeds the advised readability levels (10, 11). The advent of large language models (LLMs) such as ChatGPT has recently opened new avenues for enhancing the accessibility of complex health information (12, 13). Our study aimed to evaluate the usability and accuracy of ChatGPT in transforming technical rheumatology literature into patient-friendly materials that adhere to AMA and NIH readability guidelines.

To perform this analysis, we selected 12 open-access articles on diverse rheumatology topics authored by the two researchers (CMP and PMR). These articles spanned different study designs, including retrospective studies and systematic reviews. We then evaluated their readability using two well-established tools: the Flesch-Kincaid Grade Level (FKGL) and the Simple Measure of Gobbledygook (SMOG). The FKGL and the SMOG have emerged as the two most frequently employed readability assessment tools. Both formulas integrate metrics, including total word count, sentence count, syllable count, and the number of polysyllabic words. An online calculator (via the readability formulas website) was utilized to compute these readability scores. We subsequently provided ChatGPT 4.0 with the prompt: “Could you please simplify the following text, sourced from a peer-reviewed scientific paper? The objective is to achieve a 6th-grade reading level to make it fully understandable for a general reader. I will supply the text”. Appropriateness was assessed according to criteria previously published by two rheumatologist researchers (CMP and PMR), the authors of the selected articles. The outputs were classified as “appropriate” or “inappropriate” based on the evaluators’ clinical experience and understanding of the relevant literature.

The criteria for an appropriate response required the accurate simplification of the full text without introducing any false or misleading information. In contrast, an inappropriate response either contained inaccurate content or included material not intended by the original study authors. In cases of disagreement between the two evaluators, an independent opinion was sought from a third fellowship-trained uveitis physician. Statistical analyses were performed using SPSS software (version 26.0). A paired-sample t-test compared the original texts’ average readability scores and word counts with those of the ChatGPT-generated responses in achieving the advised readability level equivalent onto the sixth grade. At the same time, descriptive statistics summarized the remaining data. Given the absence of human participants and their data in this investigation, obtaining informed consent was unnecessary.

The application of ChatGPT significantly improved the readability of scientific articles (P < .0001). The mean readability level was reduced from approximately a 15th-grade level in the original texts to a 10th-grade level in the generated outputs. This reduction was observed consistently across all analyzed study types. Furthermore, the outputs generated by ChatGPT were evaluated as appropriate, accurately reflecting the source material in a simplified format without substantive errors. A significant decrease in length was also noted, with mean word counts reducing from 3,517 (SD 843) in the original articles to 446 (SD 172) in the generated responses (P = 0.047). Detailed data regarding response readability, appropriateness, original readability metrics, and study types are presented in Table 1.

Overall, ChatGPT demonstrated a significant capacity (P < .001) to improve the accessibility of open-access, peer-reviewed scientific literature by reducing its average readability score by five grade points from the 15th to the 10th-grade level. Over the past year, coinciding with the emergence of LLMs exemplified by ChatGPT, there has been heightened attention on integrating these artificial intelligence models into healthcare and clinical education. This includes research into their use for developing patient-facing health content, assisting with postoperative patient management, and automating responses to typical questions (14, 15).

Drawing from our prior research, which has been among the initial investigations into the capacity of LLMs to process and present health-related information comprehensibly for patients, this study serves as a direct extension. Our foundational work, focusing on using models including GPT-3.5, GPT-4, and Copilot to address patient inquiries regarding antimalarial therapy in SLE (16), demonstrated high accuracy and reproducibility across models. Importantly, that study revealed significant variability in the completeness of responses, with GPT-4 exhibiting superior performance, particularly for complex topics like mechanism of action. Grounded on these findings, which highlighted both the potential and limitations of LLMs in delivering accurate and complete patient information, the present analysis evaluates explicitly the effectiveness of ChatGPT in a related but distinct application: simplifying the readability of complex peer-reviewed scientific literature within the broader area of rheumatology. This progression, from evaluating the content quality of direct answers to assessing the readability of simplified published research, reflects our ongoing commitment to exploring how AI can best enhance health literacy in rheumatologic care.

While our results demonstrate that ChatGPT can markedly improve the readability of complex rheumatology literature (reducing the average grade level from 15th grade to 10th grade), this simplified output remains well above recommended standards. The AMA and NIH advise that patient education materials be prepared around a sixth-grade reading level (2), far lower than the high school level achieved by ChatGPT in our study. This discrepancy is not a trivial gap; educational content pitched at a 10th-grade level still exceeds the literacy of many patients, undermining accessibility. Patients with limited health literacy—who often belong to disadvantaged groups—may struggle to comprehend information at this level, leaving them disadvantaged in managing their care and participating in informed decision-making. Other evaluations of ChatGPT's medical explanations have noted similar shortcomings, with average readability scores in the high school to college range (e.g., FKGL 13 in one study of fibromyalgia queries) (17). These findings raise important clinical and policy considerations. Clinically, suppose providers rely on ChatGPT to generate patient education handouts or summarize research. In that case, they must recognize that the “simplified” text may still be too complex for many readers, potentially perpetuating the communication barriers it aims to bridge. From a policy standpoint, there is a need for guidelines and oversight on using AI tools in patient education—ensuring that content is vetted or further refined to meet health literacy best practices (ideally at or below the 6th-grade level). Although iterative prompting or more advanced models might further lower the reading level towards compliance with guideline (18) healthcare professionals bear the final responsibility for confirming understandability and straightforwardness. Ultimately, the limitation is that ChatGPT's output does not yet consistently reach recommended readability levels, which is significant: addressing it is critical for promoting health equity and truly informed decision-making among patients with rheumatic diseases (8). As the literature emphasizes, improving readability is not merely an editorial concern but a moral imperative to ensure all patients—regardless of literacy—can access, understand, and use health information effectively (2, 19).

Our findings indicate that utilizing ChatGPT to simplify peer-reviewed rheumatology literature reduced readability from an average of 15th to 10th grade. This outcome contrasts with a similar study in the field of ophthalmology, where ChatGPT reduced the readability of articles from 15th grade to a 7th-grade level (20). One potential factor contributing to this difference could be the inherent complexity of rheumatic diseases, which often involve intricate pathological processes, diverse manifestations, and complex treatment regimens, potentially simplifying to a lower reading level more challenging than specific topics in ophthalmology.

Historically, the scientific literature's inherent complexity and technical language have posed significant barriers to patient comprehension. Prior research establishes a positive correlation between improved understanding of medical information and health literacy and outcomes such as increased trust in scientific bodies and enhanced patient capacity for self-management of health conditions (21, 22). Consequently, the advent and utilization of LLMs, exemplified by platforms like ChatGPT, present a potential mechanism for improving patient engagement with scientific content, possibly fostering a greater societal perception of the reliability of scientific research.

Patients could derive advantages from consulting peer-reviewed literature, predicated on the notion that enhanced research comprehension can positively influence multiple facets of their care (23). Extensive scholarship confirms that grasping the underlying justification for medical interventions significantly improves patient adherence to treatment protocols throughout diverse clinical fields (24). Strategies to mitigate the proliferation of health misinformation include enhancing health literacy, promoting collaborative use of online resources by patients and physicians, and establishing more robust indicators of source quality, as Swire-Thompson and Lazer described (25). Furthermore, direct public engagement with primary research outputs, including clinical trial results and peer-reviewed literature, potentially empowers patients toward more informed healthcare decision-making.

This study has limitations affecting its interpretation and application. Despite verification steps, the recognized propensity of ChatGPT to occasionally produce inaccurate information must be acknowledged (26). Thus, ChatGPT summaries of scientific literature should not serve as the sole basis for patient health education. Direct consultation with physicians concerning individual health conditions is strongly recommended. A second limitation involves the inherent nature of single studies; one article, viewed in isolation, may provide insufficient context or lack the comprehensive information necessary for sound healthcare decision-making. Therefore, patients seeking to make informed choices should consult various trusted information sources. Furthermore, it is currently unknown whether the accuracy or appropriateness of ChatGPT's responses could be negatively impacted by follow-up requests aimed at further shortening the generated text.

Third, although content readability is fundamental to comprehension, assessing the real-world impact of ChatGPT requires direct patient evaluation. Future prospective studies involving patient participants are necessary to determine the effectiveness of ChatGPT in enhancing the understanding of medical information and influencing health-related decision-making. Finally, the analysis was restricted to peer-reviewed scientific articles focused primarily on some rheumatic diseases, mainly SLE, and authored by the two senior investigators of this study. Consequently, the generalizability of these findings is limited, and caution is warranted against extrapolating the results to the broader fields of rheumatology.

In conclusion, this study demonstrates that the LLM ChatGPT 4.0 can significantly reduce the reading complexity of peer-reviewed rheumatology literature, transforming text requiring graduate-level reading proficiency to a more accessible 10th-grade level. Expert rheumatologists found the generated summaries to be accurate and appropriate, indicating the potential of LLMs as tools to bridge the communication gap between complex scientific findings and lay audiences. However, despite this marked improvement, the resulting readability still falls short of the recommended sixth-grade level for optimal patient education materials. This highlights a relevant limitation: while LLMs offer a promising avenue for enhancing health literacy, their current outputs may not be sufficiently simplified for all patients, particularly those with lower literacy levels. Therefore, while ChatGPT can be a valuable aid, its use in generating patient-facing summaries requires careful implementation, likely involving expert review and potential further refinement to meet established health literacy guidelines. Future research should focus on optimizing LLM prompts for greater simplification, evaluating the impact of these tools on actual patient comprehension and decision-making, and exploring their application across a broader range of rheumatologic topics to fully harness their potential while mitigating risks like inaccuracy or perpetuating health disparities.