We conducted a cross-sectional, mixed-methods infodemiology study to systematically benchmark how LLMs explain the gut–liver axis, a biologically and clinically integrative concept spanning microbiology, metabolism, immunology, and clinical medicine. Rather than quantifying factual accuracy or hallucination frequency, this study focused on the structural properties of explanation that shape evidentiary signaling, an aspect critical for decision-adjacent applications. The evaluation was explicitly centered on two complementary dimensions with direct relevance to translational bioengineering and clinical decision support: informational reliability and patient-facing interpretability. The study design was constructed to approximate real-world information-seeking behavior while maintaining strict experimental control, thereby enabling reproducible, model-agnostic comparisons across platforms. As the analysis was restricted to model-generated text and did not involve human participants, clinical records, or biological specimens, institutional ethics review and informed consent were not required.

To probe model performance across increasing levels of mechanistic and clinical complexity, we developed a structured set of 20 clinically grounded questions through an iterative, expert-led process. Three oncology physicians with active roles in patient education and clinical decision-making integrated contemporary clinical guidelines, validated patient-education resources, and recurrent informational needs observed in routine outpatient practice to define representative queries (Table 1). Questions were refined to ensure neutral phrasing, conceptual precision, and patient-centered relevance, and were subsequently organized into five domains reflecting the hierarchical knowledge structure of the gut–liver axis: basic mechanism, microbiota–metabolism, disease association, clinical intervention, and detection and evaluation. This domain-based framework was designed to identify points at which fluent narrative generation diverges from mechanistic fidelity, a known limitation when LLM outputs are used in decision-adjacent contexts.

All questions were submitted verbatim to five widely accessible LLMs within a predefined collection window (December 10–20, 2025) to minimize version drift and enhance reproducibility. Each query was entered in a new, independent session without supplemental instructions, contextual priming, or follow-up prompts (Al-Thani et al., 2023). When multiple candidate responses were generated, the first complete output was retained to limit investigator degrees of freedom. Outputs were compiled into a standardized, anonymized corpus of 100 responses indexed by model and question domain, forming the basis for subsequent quantitative scoring and qualitative error-pattern analysis. The fixed sample size of 20 responses per platform was selected to enable controlled cross-platform and cross-domain comparison under standardized prompting conditions, prioritizing internal validity and pattern detection over statistical power for population-level inference regarding overall model performance. This framework was designed to function as a structured audit of explanatory performance rather than a benchmark of stylistic fluency.

To characterize the ease with which LLM-generated outputs can be processed by a lay audience, we assessed linguistic accessibility using a multi-metric readability framework. This analysis was intended to describe surface-level language burden rather than infer scientific validity, thereby explicitly separating textual form from informational substance.

Seven established readability indices were applied to each response (Supplementary Table 1), capturing complementary dimensions of linguistic complexity, including syntactic structure, lexical length, and sentence density: Coleman–Liau Score (CL), Linsear Write Index (LW), Automated Readability Index (ARI), Simple Measure of Gobbledygook (SMOG), Gunning Fog Index (GFOG), Flesch Reading Ease Score (FRES), and Flesch–Kincaid Grade Level (FKGL) (Mac et al., 2022; Özduran and Hanci, 2022; Yilmaz Hanci, 2023; Hanci et al., 2024). With the exception of FRES, higher values indicate greater textual complexity.

All calculations were performed on raw, unedited model outputs using uniform computational settings across models. This approach preserved intrinsic language characteristics and ensured that readability comparisons reflected model behavior rather than post-processing artifacts. Readability metrics were treated as descriptive indicators of user-facing communication complexity and were evaluated independently from content-level quality to avoid conflating accessibility with epistemic validity (Ojha et al., 2021; Gkikas et al., 2022). Accordingly, readability metrics were treated as descriptive indicators of linguistic complexity and user-facing communication burden, rather than as normative proxies for informational quality, accuracy, or epistemic reliability.

Beyond linguistic form, we independently examined the epistemic quality of model-generated explanations, focusing on transparency, reliability, and educational value. Informational integrity was operationalized using three complementary instruments with established use in digital health information assessment: the Journal of the American Medical Association (JAMA) Benchmark Criteria, the Global Quality Score (GQS), and a modified DISCERN framework (Supplementary Table 2) (Charnock et al., 1999; Azer, 2020; Kyarunts et al., 2022; Mohammadzadeh Azarabadi et al., 2025).

Transparency and source signaling were assessed using the JAMA Benchmark Criteria, which capture explicit disclosure of authorship, attribution of information sources, reporting of conflicts or sponsorship, and temporal currency (Bruce-Brand et al., 2013; Kara et al., 2024). Each criterion was scored dichotomously, yielding a composite score ranging from 0 to 4.

Educational value was evaluated using the GQS, a five-point ordinal scale reflecting overall coherence, logical organization, and potential usefulness for non-specialist audiences(Uzun, n.d.). Higher scores indicate content judged to be clearer and more informative from a patient-education perspective.

To further assess internal content reliability, we applied a modified DISCERN instrument restricted to its first domain, consistent with prior evaluations of non-interactive health information (Handler et al., 2021; Shan et al., 2023). The full DISCERN instrument was not applied because its subsequent domains primarily assess treatment options and decision-making quality, which are less applicable to non-interactive, explanatory model outputs rather than patient-facing treatment guidance. This domain comprises five binary items addressing clarity of purpose, relevance and balance, identification of sources, and acknowledgment of uncertainty. Affirmative responses were summed to generate a reliability score ranging from 0 to 5.

To mitigate subjectivity in content-level judgments, all responses underwent independent dual evaluation by medically trained reviewers following a structured calibration procedure. Prior to formal scoring, reviewers jointly assessed a set of pilot responses to harmonize interpretation of scoring criteria across all instruments. aInter-reviewer agreement was quantified using Cohen’s kappa statistic to measure consistency beyond chance, interpreted according to established benchmarks (McHugh, 2012; Rau and Shih, 2021). Discrepancies were resolved through structured discussion, with adjudication by a senior reviewer when necessary. This process ensured that observed differences in quality metrics reflected genuine variation in model outputs rather than inconsistency in evaluator judgment.

All analyses followed a prespecified workflow to ensure reproducible, model-agnostic inference while accounting for corpus hierarchy. Normality was assessed using the Shapiro–Wilk test and visual inspection. Between-model differences were evaluated using one-way analysis of variance with Bonferroni adjustment for approximately normal measures, and Kruskal–Wallis tests with Dunn–Bonferroni correction for non-normal measures, including most readability indices. Standardized effect sizes with confidence intervals were reported alongside P values. Associations between readability and informational reliability and quality were analyzed using Spearman correlations and domain-adjusted partial correlations. All tests were two-sided with P < 0.05. Statistical analyses were conducted in IBM SPSS Statistics (version 25), and figures were generated using GraphPad Prism (version 9).

A total of 100 LLM-generated responses were analyzed, evenly distributed across five predefined gut–liver axis knowledge domains. Linguistic accessibility varied systematically by domain, with statistically significant heterogeneity concentrated in indices reflecting grade-level demand and syntactic density (Table 2). ARI, GFOG, FKGL, and CL all demonstrated higher median values in domains oriented toward disease association, clinical intervention, and evaluation compared with foundational mechanistic domains, delineating a domain-dependent complexity gradient.

In contrast, surface-level fluency measures showed limited sensitivity to domain context, and no domain-dependent differences were observed for informational reliability or educational quality indicators, including the modified DISCERN score, JAMA Benchmark score, and Global Quality Score (all P > 0.05). Together, these findings indicate that increasing linguistic complexity across the gut–liver axis knowledge hierarchy is not accompanied by parallel gains in epistemic robustness. As a result, topic complexity alone cannot be used to infer informational quality, indicating that greater interpretive demands are not accompanied by corresponding gains in informational reliability.

Across all evaluated responses, platform identity accounted for the largest share of variation in informational reliability and educational quality (Figures 1A–C; Table 3). Statistically significant between-model differences were observed for the modified DISCERN score, JAMA Benchmark Criteria, and Global Quality Score (all P < 0.001), demonstrating that explanatory performance clustered systematically by platform rather than being evenly distributed across outputs. These cross-platform differences are not intended to constitute a performance ranking or hierarchy of model superiority, but rather to illustrate distinct explanatory strategies and epistemic signaling patterns across models under standardized conditions. GPT-5 consistently achieved the highest mean scores across all three instruments, indicating more robust transparency signaling, stronger reliability features, and greater educational utility, whereas Tongyi Qianwen ranked lowest overall, reflecting limited robustness in delivering trustworthy, decision-relevant explanations of the gut–liver axis. Other platforms occupied intermediate positions with broader score dispersion, suggesting greater variability in output-level reliability.

Readability likewise varied significantly by platform across all assessed indices, including ARI, FRES, GFOG, FKGL, CL, SMOG, and LW (all P < 0.001). Some platforms consistently generated explanations with higher grade-level demand and greater syntactic density, whereas others produced linguistically simpler outputs. Importantly, lower linguistic complexity was not associated with improved informational reliability, and higher language demand was not intrinsically detrimental. Together, these results indicate that platform-level optimization of readability and epistemic reliability proceeds along partially independent trajectories, establishing platform identity as the dominant determinant of explanation quality in LLM-generated gut–liver axis content.

Item-level analysis using the modified DISCERN framework showed that between-model differences were driven by reconfiguration of specific reliability components rather than uniform shifts in overall quality (Figure 2). Across platforms, attainment in presentation-oriented items, including clarity of purpose and informational balance, did not consistently coincide with attainment in credibility-signaling items, particularly source identification and acknowledgment of uncertainty. Consequently, composite reliability scores masked distinct internal configurations of epistemic strength and weakness, indicating that narrative adequacy can coexist with systematic gaps in evidentiary accountability.

Distinct reliability profiles emerged at the item level. GPT-5 demonstrated uniformly high attainment across all DISCERN items, reflecting consistent coupling of narrative clarity with source disclosure and uncertainty awareness. In contrast, other platforms exhibited selective or constrained patterns characterized by persistent underperformance in disclosure-oriented dimensions, resulting in attenuated composite reliability signals. These structured omission profiles underscore the value of item-level reliability assessment for identifying failure modes that are not captured by aggregate quality scores.

Correlation analysis demonstrated that linguistic accessibility and epistemic reliability occupy largely separable dimensions in LLM-generated gut–liver axis explanations (Figure 3). Readability indices formed a tightly coupled cluster, with strong concordance among grade-level demand and syntactic density measures, whereas integrity-oriented scores showed only moderate convergence, indicating that transparency signaling, reliability cues, and perceived educational value are related but non-identical constructs.

Cross-dimensional associations between readability and informational integrity were uniformly weak. Modified DISCERN, JAMA Benchmark, and Global Quality Score each exhibited only small or near-null correlations with grade-level demand and fluency measures, demonstrating that linguistic simplicity cannot be used as a proxy for credibility or trustworthiness. Together, these findings define a structural partition between language burden and epistemic safeguards, indicating that readability and informational reliability represent independent axes of explanatory performance rather than interchangeable indicators of explanation quality.