Supporting behavioral change in individuals requires that the materials used for behavioral interventions are easy to read and understand. People with low health literacy are known to inappropriately use healthcare services, poorly manage chronic conditions, and use medical services inefficiently (1–3). Therefore, clarity and readability are essential in the guidance materials used to promote participation such as in health checkups; however, increasing the participation rate remains a key challenge. Previous studies have shown that improving the readability of screening invitations can increase the participation rates (4). Furthermore, creating easy-to-read health information not only makes it easier for the reader to understand but also enhances self-efficacy in performing health behaviors (5). It has also been reported that clear materials can increase the readers’ comprehension, sense of security, and satisfaction, and satisfaction positively influences decision-making (6). Of course, it is difficult to associate changes in participation rates solely with improved materials, as people’s perceptions and beliefs about health are not always rational and various factors influence their behavior. Still, improving communication materials remains a key element of public health interventions.

Practical guidelines for creating easy-to-understand health and medical materials in English (7) and practical kits to support the creators (8) are available. While similar support for creating health and medical materials in Japanese is not well-established, principles for crafting clear and persuasive health information have been established based on academic knowledge (9, 10) and are utilized as practical guides for creating materials in Japanese.

This study focused on materials that encouraged participation in the specific health checkups and health guidance provided under the municipal National Health Insurance (NHI) system. These programs aim to promote the prevention of lifestyle-related diseases and serve as an important strategy for addressing the increasing burden of healthcare costs associated with Japan’s rapidly aging population.

In Japan, all citizens are enrolled in one of several public health insurance schemes. NHI is one such scheme, operated by municipalities and primarily covering self-employed individuals and those working in agriculture, forestry, and fisheries. Recently, insurers have been increasingly expected to improve participation rates in specific health checkups and health guidance as part of government-led efforts to control rising healthcare expenditures.

One of the most critical initiatives in disease prevention is Japan’s specific health checkup and health guidance program, mandated for public health insurers since 2008 (11). In this program, insurers aim to prevent lifestyle-related diseases such as diabetes and hypertension by identifying individuals aged 40–74 with metabolic syndrome and providing tailored counseling based on their disease risks. The national targets for participation rates have been set at 70% for specific health checkups and 45% for specific health guidance. However, most insurers have not reached these targets (12). Moreover, public subsidies are allocated according to the participation rates, providing a strong incentive for insurers to raise them. The materials evaluated in this study—printed materials such as postcards and leaflets—are a key component of those efforts, intended to inform recipients about the importance of preventing lifestyle-related diseases and to encourage participation in the program.

In this study, the term “insurer” refers to entities responsible for implementing the specific health checkups under Japan’s National Health Insurance system. These include municipal governments (cities, wards, towns, and villages) and National Health Insurance Associations operated by specific occupational groups. They are primarily public administrative bodies and do not refer to private insurance companies.

Despite the widespread distribution of recommendation materials for specific halth checkups, several issues have been identified, such as failing to attract recipients’ attention or prompt action. Improvements have been suggested regarding content, information volume, layout, timing of distribution, and audience segmentation (13). However, there remains limited evidence on effective training approaches to enhance the quality of such materials.

To address this gap, the current study developed a checklist based on the Suitability Assessment of Materials (SAM) framework, which is used to evaluate the suitability of health communication materials (14). The SAM provides scores that categorize materials as Superior (70–100), Adequate (40–69), or Not Suitable (0–39). By adapting this tool to the perspective of material creators, we aimed to offer practical training to support the development of materials that effectively encourage participation in specific health checkups and health guidance. We evaluated changes in the clarity and readability of the materials before and after providing the training to municipal staff.

The checklist items were based on the Japanese version of the SAM. As presented in Table 1, the checklist consisted of 20 items. Each item was evaluated using the following criteria: excellent, good, needs some improvement, needs much improvement, and not applicable.

The participants were member insurers of the Tokyo Metropolitan National Health Insurance Organization (Tokyo NHIO) and were responsible for developing recommendation materials for specific health checkups. The Tokyo NHIO is a legally recognized organization established with the approval of the Governor of Tokyo under Article 83 of the National Health Insurance Act for insurers (wards, cities, towns, and National Health Insurance associations). Tokyo metropolitan government, special wards (“Ku”), cities, towns, villages, and National Health Insurance Associations for specific occupational groups in Tokyo are members (84 insurers in total) of Tokyo NHIO (15).

These member insurers were represented by municipal staff who were the actual senders of health checkup recommendation materials. Specifically, they were engaged in insurance-related services. The majority were administrative officers (74.5%), followed by public health nurses (21.6%) and others, such as registered dietitians (3.9%). While 69% had practical experience in preparing notification materials, only 26% had ever received any related training. These staff members participated in the training voluntarily after a call for participation was issued to all insurers in the Tokyo area. The intended receivers of the materials were primarily individuals aged 40 to 74 who are self-employed, unemployed, or employed in part-time or non-regular positions that are not covered by employee health insurance.

The training was conducted through individual and group sessions. In the individual sessions, a checklist was used to provide personalized feedback and specific suggestions for the improvement of the materials. The insurers were asked to submit specific health checkups and guidance recommendation materials (leaflets and postcards). Multiple researchers that are proficient in health communication reviewed these materials using the checklist developed in this study. The review included symbolic evaluations and detailed comments in the free description section.

The group sessions were organized to provide participants with hands-on experience in using the checklist. The session began with a lecture on the efforts to improve health checkup participation rates, application of behavioral science theories, and process evaluation of specific health checkups (Lecture 1). This was followed by a lecture on segmenting the target audience, clarifying the message concepts, conducting preliminary surveys, selecting communication channels, and creating messages and materials (Lecture 2). Each lecture, delivered by two experts, lasted 15 min. Subsequently, Lecture 3 covered the explanation and application of the checklist developed in this study. An exemplary case presented was a material with sufficient margins and a clear layout, placing the most important information for the recipient—such as what action to take and where to apply—in a prominent position. In contrast, problematic cases included materials that used indirect or bureaucratic expressions (e.g., “has become available”) instead of more conversational ones (e.g., “you can now use”), placed insurer-centered messages (e.g., “we need to raise our screening rate”) in the most visible areas, or used unrelated illustrations without explanatory captions, potentially confusing the reader. The session concluded with a group work activity.

Insurers who participated in individual sessions, group sessions, or both were asked to submit self-produced materials (flyers, postcards, leaflets, etc.) to recommend specific health checkups and health guidance in December 2020. These materials had actually been distributed to eligible residents in their respective municipalities or other insured populations covered by the submitting insurers. The materials before (2018 version) and after training (2020 version) were collected in an electronic file format. The suitability scores of the materials were calculated before and after training.

Of the 84 insurers, 49 (58%) participated in training. Among them, 31 were included in the analysis, after excluding those who did not submit materials (n = 13), those whose materials were not collected in pairs before and after the training (n = 2), those who did not report changes in their materials after the training (n = 2), and those who reported making changes without referring to the training (n = 1).

A total of 62 materials (31 before and 31 after training) collected from 31 target insurers were included in the analysis.

Pre-training materials were created in fiscal year 2018. However, insurers who reported not having created materials in 2018 used materials from 2017 as pre-training materials. Post-training materials were created in fiscal year 2020. For insurers that did not create materials in 2020, materials from 2019 were used as post-training materials.

For insurers who submitted multiple materials before and after the training, we thoroughly reviewed the purpose of each material (whether it was for a specific health checkup or specific health guidance, whether it was an initial notice or reminder, or whether there was segmentation) to select one pair with the same purpose. References indicating a specific year were removed from all the materials.

The evaluation items were expanded to 26 based on the original 20 checklist items while incorporating the specific content of the specific health checkup (Table 1). Each item was rated on the following scale: excellent = 3 points, good = 2 points, needs some improvement = 1 point, and needs much improvement = 0 points. The method for calculating the suitability score was the same as that used in the original version of SAM, using the following formula: total evaluation score/maximum possible score × 100. The suitability score for each material was adjusted for individual evaluator variability by calculating an adjusted score using the following formula: 50 + ((individual evaluator’s score – average score of individual evaluator) / individual evaluator’s standard deviation) × 10. The final suitability score for each material was the average score of the three evaluators.

Each material was evaluated by three evaluators. Before the evaluation, the evaluators underwent a two-hour online training session using the “Evaluation Manual for Specific Health Checkup Recommendation Materials Based on SAM” and sample materials. The evaluators were blinded to the materials used before and after the intervention. The key allocation condition was that the same evaluator would not evaluate both the pre- and post-intervention materials from the same insurer. It was noted that there is individual variability in the evaluation of Japanese texts (16), and involving multiple evaluators could help minimize the influence of individual differences (17). Therefore, it is desirable to have as many different combinations of the three evaluators as possible. Six evaluators (labeled A-F) were randomly assigned to the materials, as presented in Supplementary Table 1.

The Cronbach’s alpha coefficient for the scores from the six evaluators who assessed two types of materials, which were not part of this study, was 0.56 before adjustment and 0.60 after adjustment. Owing to low reliability, we further examined the inter-rater correlation (Spearman’s correlation coefficient and significance level) among the evaluators of the 31 insurers (62 materials) included in the analysis. The correlations were as follows: Evaluators A and B (r = 0.78, p < 0.0001), A and C (r = 0.69, p = 0.02), B and C (r = 0.80, p < 0.0001), and E and F (r = 0.53, p = 0.01). Evaluator D did not exhibit significant correlations with any of the other evaluators.

Therefore, in addition to analyzing the changes in the suitability scores for the materials before and after training using the results from all six evaluators, we conducted a sub-analysis excluding Evaluator D using the scores from the remaining five evaluators.

The means and standard deviations of the suitability scores for the “pre-training” and “post-training” materials were calculated and compared using a paired t-test (two-tailed). The differences in the mean values were also compared for each item in the evaluation manual. Statistical analyses were performed using the SAS software (Ver. 9.4), with a significance level of 5%.

This study adheres to standard reporting guidelines for research involving non-human subjects. In Japan, research such as ours—evaluating the clarity of health checkup recommendation materials (e.g., flyers, postcards, leaflets)—is not subject to review under the “Ethical Guidelines for Medical and Biological Research Involving Human Subjects.” As the work did not involve human participants but rather focused on organizational practices, institutional ethics committee or IRB approval was not required. Instead, an explanatory document was distributed to health insurers to notify them of the intent to evaluate the effectiveness of revised materials (Figure 1).