Ethical considerations the information utilized in this study was solely derived from publicly available WeChat data, with no involvement of any personal privacy concerns. Clinical data or human specimens were not included in the research, nor was there any direct interaction with users. Consequently, ethical review was deemed unnecessary.

On September 10, 2023, data was collected from WeChat, Baidu, Qingbo, and Sogou, using the keywords “Systemic Sclerosis” or “scleroderma” to identify relevant Chinese WeChat public accounts. Due to the limited number of SSc-related accounts, manual data collection was employed to ensure quality. Researchers manually searched and recorded details of public accounts related to systemic sclerosis or scleroderma, excluding deactivated accounts, those inactive for over 6 months, and accounts not focused on SSc. Account details were documented, such as registration date, operator, posting frequency, and article count.

Using the PythonScrapRedis crawler, all publicly published articles were retrieved and filtered according to specific inclusion and exclusion criteria. PythonScrapRedis, combining Python and Redis, facilitated efficient automated data scraping and processing. Inclusion criteria required articles to be original and provide treatment information related to rare diseases (RDs), while exclusion criteria eliminated irrelevant topics, content without treatment details, reposted links, duplicates, and articles in image format unsuitable for DISCERN evaluation (Figure 1).

This research was designed as a cross-sectional study, with sample size calculation based on previous studies (41–43). An average DISCERN score of 32.3 (SD = 11) was expected. With a 5% invalid data rate, a minimum of 218 cases was required. Proportional random sampling was conducted using R3.4.2 software, ensuring representativeness. Accounts with fewer than 10 processed articles were fully included to maintain sampling accuracy.

Articles are categorized based on the primary source of WeChat operations (how different entities or organizations publish and manage content on the platform) and the treatment methods introduced in each article. The sources of articles are divided into three categories: (1) public interest organizations (medical funds), (2) commercial companies, and (3) individuals/patients without medical professional backgrounds. The content categories are as follows: (1) functional rehabilitation; (2) nursing; (3) disease knowledge; (4) Western medicine; (5) psychology; (6) policy interpretation; (7) traditional Chinese medicine; and (8) comprehensive treatment involving two or more modalities. The articles are downloaded, and information extracted between September 10, 2023, and October 10, 2023.

This study assessed the quality of health information using the DISCERN tool and the GQS. DISCERN evaluates treatment-related health information through 16 items divided into three sections: Part 1 (questions 1–8) assesses reliability, Part 2 (questions 9–15) focuses on treatment quality, and Part 3 (question 16) provides an overall rating (43, 44). Each item is scored on a 5-point scale based on five key criteria: clarity and achievement of objectives, use of reliable sources such as published literature or expert opinion, balance and impartiality of content, availability of additional resources, and discussion of uncertainties. Total scores range from 16 to 80 and are classified as very poor (16–26), poor (27–38), fair (39–50), good (51–62), and excellent (63–80).

The GQS provides an overall assessment of patient content quality and educational value (43, 45). A score of 1 indicates poor quality, missing information, or misleading content with minimal educational value; 2 indicates limited and incomplete content with low technical quality; 3 reflects moderate quality, partially informative content, and basic technical adequacy; 4 indicates good quality with comprehensive and useful content; and 5 represents excellent quality, complete information, and strong educational value.

All evaluations were independently conducted by two senior rheumatologists with extensive clinical experience in systemic sclerosis (Haojie Chen and Yingying Gao). Each rater applied DISCERN and GQS to assess the selected content, with final scores calculated as the average. In cases of significant discrepancy, consensus was reached through team discussion. Inter-rater reliability was assessed using Pearson correlation analysis based on raw scores.

Two individuals recorded the data. The researchers were experienced senior rheumatologists proficient in diagnosing and treating SSc, utilizing the DISC method during the screening and grading process.

This study delves into understanding user satisfaction and concerns by collecting user feedback on SSc-related articles on the WeChat platform. The specific methods involved gathering user comments and questions and classifying and conducting sentiment analysis on these responses using natural language processing techniques. User comments and feedback were classified using a Naive Bayes classifier into positive, neutral, and negative categories. Sentiment analysis models and modern pre-trained models (e.g., BERT, GPT) were employed to capture and interpret emotional content in the comments. The study assessed post-article behaviors by analyzing user engagement data, including shares, discussions, and click-through rates. Data collection was facilitated through the data interfaces provided by the WeChat public platform, enabling user comments, shares, and click data extraction.

For a comprehensive and accurate evaluation of therapeutic information related to SSc on the WeChat platform, this study introduced various tools for assessing the quality of medical information. Initially, the accuracy and timeliness of medical information were assessed, evaluating the authenticity, reliability, and whether the content reflects the latest research or clinical guidelines. Rheumatology experts scored the articles to ensure content accuracy and relevance. A reader satisfaction questionnaire was also designed and distributed to collect readers’ evaluations of the articles’ content, format, and practicality through the WeChat public platform. The questionnaire results were compared and analyzed against the scores from other assessment tools to provide a comprehensive quality assessment. Finally, the survey results were combined with other assessment tools for a comprehensive quality evaluation. Finally, the study developed an integrated evaluation framework using DISCERN and GQS scores to rank and assess the overall quality of the articles.

Continuous variables were presented using the mean and standard deviation, while non-continuous variables were depicted using the median (interquartile range, IQR). Percentages represented categorical variables. Group differences were assessed using the Kruskal-Wallis test for non-normally distributed quantitative variables and Dunn’s multiple comparison test for pairwise comparisons. Spearman correlation analysis was employed to evaluate relationships between quantitative variables. Statistical significance was defined as p < 0.05. Statistical analyses were performed using GraphPad Prism 9.0.0 for Windows (GraphPad Software).

This study analyzed 6,408 articles from 9 WeChat public accounts, of which 59% were original content. A random sample of 375 articles was further examined (Table 1). The analysis showed high readership for SSc-related articles, with a total of 172,841 views and a median of 265 views per article. However, user interactions (likes, comments, and shares) were relatively low, with only 2,094 likes, 705 comments, and 1,377 shares. This suggests that despite significant interest in the topic, user engagement remains limited, indicating a one-way flow of information.

Among the accounts, “Multiple Sclerosis Home” showed high activity and readership but relied heavily on reposted content, with a low proportion of original articles. On the other hand, “Scleroderma Systemic Sclerosis” had high posting frequency but low engagement, possibly due to content quality or distribution issues. Notably, “Purple Conch Public Welfare Service Center” and “Scleroderma” demonstrated strong original content creation and good user interaction, highlighting their effective content strategies.

This study analyzed the time distribution of 6,408 articles published between January 1, 2015, and September 10, 2023, to monitor the activity of WeChat public accounts. Starting in 2019, publication frequency increased noticeably, with a significant peak observed during 2020–2021 (Figure 2).

The data shows that in 2015 and 2016, the number of articles was relatively low, with 25 and 91 articles, respectively. In 2017 and 2018, the number increased slightly to 101 and 193 articles, indicating modest growth. A significant rise occurred in 2019, with 550 articles published, marking increased activity in public accounts. In 2020 and 2021, articles surged to 1,277 and 2,002, respectively. This spike can be attributed to the expansion of the WeChat user base and a growing demand for SSc-related information. Although the number of articles slightly decreased in 2022 and 2023 to 1,049 and 1,120, respectively, they remained relatively high.

Based on the cited sources (Figure 3a), individuals account for 50% of the articles (189/375), followed by companies at 28% (105/375) and institutions at 21% (81/375). In terms of content, disease knowledge dominates, representing 53% of the total articles (198/375), while other content types include Western medicine (17%), traditional Chinese medicine (9%), and policy interpretation (6%) (Figure 3b). Table 2 presents descriptive statistics for these sources and content types.

Articles on policy interpretation (1,035 views) and functional rehabilitation (852 views) garnered the highest readership, indicating strong reader interest. Articles from institutions had an average reading volume of 538, significantly higher than those from individuals (110) and companies (265), suggesting greater reader engagement with authoritative sources.

Despite the many disease knowledge articles, their user engagement and readership were relatively low. Institutional articles had the highest readership but fewer likes and comments. Although individuals published the most articles, their engagement and readership were moderate. Company articles ranked between those from individuals and institutions across all metrics. Policy interpretation articles had the highest user interaction, followed by functional rehabilitation. Articles on Western medicine and psychology showed moderate readership but lower interaction, while traditional Chinese medicine and comprehensive therapy had the lowest engagement. In summary, policy interpretation and functional rehabilitation articles attracted more readership and engagement than other content types.

This study assessed the quality of systemic sclerosis-related articles on WeChat using the DISCERN tool (Table 3). The results showed an average score of 28.96 (SD = 7.13), with an overall quality rating of “poor.” The analysis indicated that scores were mainly concentrated in Parts 1 and 2, with few high scores (4 and 5), reflecting deficiencies in content reliability and treatment details (Figure 4).

In Part 1 (content reliability), the average score for 8 items was 16.23 (SD = 3.42). Better-performing questions included “Is the information objective and impartial?” (2.22, SD = 0.79) and “Does it address undefined areas?” (2.25, SD = 0.77). However, low scores for “Is the objective clear?” (1.64, SD = 0.63) and “Has the objective been achieved?” (1.96, SD = 0.78) indicated weaknesses in goal setting and achievement.

In Part 2 (treatment information details), the average score for 7 items was 10.69 (SD = 3.75). The lowest scores were for “Does it describe the benefits of each therapy?” (1.38, SD = 0.78) and “Does it describe the impact of treatment options on quality of life?” (1.37, SD = 1.31), indicating significant information gaps. Relatively higher scores were observed for “Does it describe each treatment option?” (1.83, SD = 0.76) and “Does it describe the risks of each therapy?” (1.64, SD = 0.76), but overall, these aspects were still lacking.

In Part 3 (overall article quality), the average score was 2.04 (SD = 0.69), indicating poor overall quality, particularly due to the lack of detailed descriptions of treatment benefits and quality of life impacts.

The DISCERN scores for articles from different sources (Figures 5a–d) and content types (Figures 5e–h) were evaluated (Figure 5). In Part 1 (Content Reliability), non-profit organizations scored significantly higher than businesses and individuals (p = 0.0007 and p = 0.016). Articles on common knowledge and psychology outperformed those on traditional Chinese medicine and disease knowledge (p = 0.046, p = 0.0005, and p = 0.042), highlighting the higher reliability of non-profit articles, particularly in these areas.

In Part 2 (Treatment Information Details), non-profit organizations again achieved significantly higher scores than businesses and individuals (p < 0.0001 and p = 0.0005). Articles on functional rehabilitation, Western medicine, and policy interpretation scored better than those on disease knowledge (p = 0.002, p = 0.0058, and p = 0.014), while psychology also outperformed both disease knowledge and traditional Chinese medicine (p < 0.001 and p = 0.013).

In Part 3 (Overall Quality), non-profit organizations continued to lead with significantly higher scores compared to businesses and individuals (p < 0.0001 in both comparisons). Articles on psychology and Western medicine scored better than disease knowledge (p = 0.006 and p = 0.017). Overall, non-profit organizations had superior total scores across all sections (p < 0.0001 and p = 0.009), with articles on functional rehabilitation, Western medicine, policy interpretation, and psychology consistently outscoring disease knowledge and traditional Chinese medicine (p = 0.007, p = 0.023, p = 0.019, p = 0.0002, and p = 0.043).

Based on the evaluation of article quality from different sources (a) and content (b), the overall average GQS score was 1.62 (SD = 0.72), indicating a classification of “poor.” The results depicted in Figure 6 show that the scores of articles from public organizations were significantly higher than those from companies and individuals (p < 0.0001 and p = 0.001). Functional rehabilitation, psychology, policy analysis, and general knowledge scores were notably higher than disease knowledge (with respective p-values of 0.01, 0.000, 0.04, and 0.017). The findings indicate that the quality of articles from public organizations surpasses those from businesses and individuals. Additionally, articles relating to functional rehabilitation, Western medicine, psychology, policy analysis, and general knowledge demonstrated higher quality than others.

Based on the DISCERN and GQS scores mentioned above, it was found that the quality of WeChat articles related to SSc was subpar, with a maximum DISCERN score of 199 (375 points, accounting for 53%) and a maximum GQS score of 196 (100 points, accounting for 52%). Upon pairwise comparison of the five levels in DISCERN and GQS, no significant differences were observed (Z = −0.105, p = 0.916), indicating that both scoring systems used for evaluation equally reflected article quality (Table 4 and Figure 7).

Spearman correlation analysis reveals significant relationships among the number of reads, likes, comments, and views of articles on SSc. Specifically, a strong positive correlation is observed between the number of reads and likes (r = 0.54, p < 0.0001), indicating that articles with higher readership tend to receive more likes. Furthermore, a moderate positive correlation exists between reads and views (r = 0.42, p < 0.0001), suggesting increased readership is associated with more views. While the correlation between comments and reads is weak (r = 0.11, p = 0.042), it is statistically significant. There is also a moderate positive correlation between likes and comments (r = 0.16, p = 0.003) and a strong correlation between likes and views (r = 0.55, p < 0.0001). Overall, there is a strong positive correlation between comments and views (r = 0.33, p < 0.0001), indicating a certain interplay of user interaction across different metrics (Table 5).

Furthermore, the study indicates a significant positive correlation between article quality scores (DISCERN score) and the number of reads (r = 0.37, p < 0.0001), likes (r = 0.33, p < 0.0001), and views (r = 0.28, p < 0.0001). GQS score also shows a positive correlation with reads (r = 0.35, p < 0.0001), likes (r = 0.29, p < 0.0001), and views (r = 0.26, p < 0.0001). These findings suggest that articles of higher quality are more likely to attract readers, likes, and views (Table 6).

The sentiment analysis of user comments reveals that out of a total of 300 comments, positive sentiments hold the highest proportion at 40% (120 comments), followed by neutral comments at 35% (105 comments) and negative comments at 25% (75 comments) (Table 7 and Figure 8A). Furthermore, the study examines user behavior data following article publication, encompassing sharing, discussing, and click-through rates. The analysis of user behavior data indicates that post-reading actions primarily concentrate on click-through rates (41.7%, 250 times), discussions (33.3%, 200 times), and sharing (25%, 150 times), signifying substantial user interest in the related articles (Table 8 and Figure 8B).

For a comprehensive and accurate evaluation of treatment-related information on the WeChat platform concerning SSc, this study introduced various tools for assessing the quality of medical information. Firstly, accuracy and timeliness assessments were conducted by experts in the rheumatology field to rate the content of the articles, ensuring the accuracy and timeliness of the information. The expert ratings showed an average accuracy score of 7.8 (SD = 1.2), and an average timeliness score of 8.1 (SD = 1.1) (Figure 9a). A reader satisfaction survey was also designed and distributed to collect readers’ feedback on the articles’ content, format, and utility through the WeChat public platform. The survey results were compared and analyzed alongside scores from other evaluation tools to provide a comprehensive quality assessment. The reader satisfaction survey indicated an average score of 4.2 for content quality (SD = 0.8), a 4.0 average score for design (SD = 0.7), and a 4.5 average score for utility (SD = 0.6), suggesting overall high satisfaction (Figure 9b). Finally, by combining DISCERN and GQS scores, a comprehensive evaluation index system was developed to rate and rank the overall quality of the articles. The results of the comprehensive evaluation index system demonstrated that the average DISCERN score was 28.96 (SD = 7.13), the average GQS score was 1.62 (SD = 0.72), and the average composite score was 6.3 (SD = 1.5) (Figure 9c).