This was a one-year quality improvement project with implementation occurring in three general phases. The study team consisted of a researcher to lead study design and planning, data collection and analysis, and evaluation; a research assistant to support patient recruitment, data collection, and data analysis; and a physician champion to lead physician and staff education and practice-wide implementation. To meaningfully involve patients and end users throughout the process, the three phases included (1) focus groups with recruited older adult patients to review and modify the note sheet, (2) training and engagement with physicians, nurses, and staff, and (3) implementing the note sheet with three PDCA cycles over a 6-month period. Institutional Review Board approval was obtained from MedStar Health Research Institute/Georgetown University prior to focus group recruitment and implementation.

Focus group recruitment and project implementation occurred in a large family practice setting located in an urban area located in the Southeastern U.S. The practice location consists of 7 attending physicians, 3 registered or licensed practical nurses, 4 medical assistants and 11 other practice staff members (e.g., front desk and administrative staff, and patient navigators). It additionally serves as a graduate medical education (GME) training site for 12 family medicine residents and serves a diverse population of patients (8,000 patient visits in 2022; 22% adults who are 65 years of age and older, 15% Hispanic, 68% Black or African American).

Patient recruitment involved the distribution of informational flyers, receipt of physician/staff referrals, and conduct of eligibility screening from daily patient appointment information from October 2022 to December 2022. Adults who were unable to provide verbal consent, non-English speakers, or those with suspected or documented impaired mental capacity were excluded. Patients were informed that they would be compensated with a $100 gift card for their complete participation in the project.

Four 60-minute focus group sessions with 6 recruited patient participants were conducted in January and February 2023. Focus groups were held in-person and at the practice location, depending on patients' preferences and communication needs. Patient focus groups were facilitated by the project lead (AT). Prior to the start of each focus group session, participants were informed about the purpose of the study, the procedures involved, and the voluntary nature of their participation. Verbal consent was obtained from all participants after a detailed explanation of the study's objectives and confidentiality measures. Participants were given the opportunity to ask questions, and their consent was documented by the study facilitator before the focus group began.

Focus group questions were designed to elicit feedback about patient experiences with the diagnostic process, perceptions of diagnostic communication, and on the note sheet's clarity, usability, and perceived value. Focus group questions included: “What does the term ‘diagnostic error’ mean to you?”; “How do doctors include you in the diagnostic process?”; “Which aspects of the note sheet were challenging for you to complete?”; “How do you think this note sheet could be improved so that you, or another patient, could use this more readily?”; and “What advice would you give a patient who was seeing this note sheet for the first time?” Patient focus group discussions were audio recorded, transcribed, and reviewed for accuracy by the research team. All transcripts were de-identified and summarized by key themes by two reviewers. Key themes were generated by a research assistant and the project lead separately, then reviewed and compared in research team meetings. These summarized themes were then presented to the focus group participants to confirm accuracy and seek additional feedback and clarification. Table 1 presents a summary of the modifications made to the existing AHRQ “Be the Expert on You” note sheet from feedback in patient focus groups. Given the minor changes made to the tool itself, we planned to administer only the modified tool to all patients, believing that the modified tool could meet the diagnostic communication needs of both older adults and the general adult population.

A physician champion was responsible for leading implementation and was routinely updated on patient discussions and changes to the note sheet throughout the focus group phase. Because of the key roles that other patient-facing clinicians and staff members play in diagnostic safety efforts (1, 26), all practice physicians, nurses, and staff members (i.e., front desk, administrative, and patient navigation staff) received training from the physician champion about broad diagnostic safety concepts and the use of the note sheet in their new workflow. Physicians, nurses, and staff had not been previously familiar with the AHRQ note sheet or other diagnostic safety concepts. Training consisted of an introduction of the Toolkit Infographic and One-Page Handout for Staff Training (10) through in-service sessions and daily team huddles. In these sessions, clinicians and staff gave additional feedback on diagnostic safety concerns within the practice, explaining that patients occasionally failed to schedule follow-up appointments or complete care activities (e.g., laboratory tests, specialist referrals) which impacted the diagnostic process. They suggested adding a section to the note sheet to address clinical care needs prior to patient check-out to mitigate these concerns and better integrate the note sheet in the clinical workflow.

This was the first diagnostic safety intervention to be implemented by physicians, nurses, and staff in this practice, and also the first time a communication tool was introduced to patients at this practice. Based on physician and practice leadership feedback, implementation of the modified “Be the Expert on You” note sheet in clinical practice occurred in three stages over the course of 6 months and included three iterative PDCA cycles to ensure that a continuous loop of workflow improvement effectively occurred (27). The PDCA cycle is a continuous loop of planning, doing, checking (or studying), and acting where testing of quality improvement measures occurs on a small scale before updating procedures and working practices (28).

We had initially planned for front desk staff to introduce the note sheet and allow patients to complete it while they were in the waiting room. However, after speaking with patients and care team members, we decided to have the note sheet introduced by nurses during the triage process. Patients expressed a desire to have the note sheet introduced by a health professional, citing the need to be encouraged to share their perspective in the context of diagnostic safety. Physicians and staff expressed the need for the note sheet to be included in the workflow in a way that accommodated concerns about high front desk staff turnover and clinic workflows.

We worked with our patient participants and physician champion to develop an adapted script from the One-Page Handout for Staff Training (10), intended for nurses to introduce the modified note sheet to patients. Nurses were provided with the script by physicians at the start of their workday, and used the script to verbally introduce the note sheet to all English-speaking adult patients after their vital signs and intake were completed. Patients were asked to fill out the note sheet while waiting to see the physician. Patients and physicians were instructed to review the note sheet together and use it to guide the health history communication. Patients were then offered the opportunity to have a scanned copy of the note sheet for their own records and asked to leave the completed note sheet in a box after checkout. All patients were invited to complete a post-appointment survey to provide feedback about using the note sheet during their appointments.

To gather feedback on the implementation process across multiple PDCA cycles, practice leadership suggested a phased approach. Initially, the physician champion piloted the modified note sheet with a designated group of patients and nursing staff over a two-week period. Feedback from the first PDCA cycle was then discussed in staff meetings, leading to minor revisions of the note sheet and workflow processes to address staff concerns and questions. These included adjustments to font size and wording, the addition of future appointment and testing reminders to the note sheet, modifications to checkout procedures, and improvements in the process of scanning and recording completed note sheets in patients' charts. The second PDCA cycle expanded the implementation to include six additional physicians and six nurses over a three-month period. Feedback from patients, physicians, nurses, and staff during this phase was reviewed and used to refine the implementation plan for the subsequent practice-wide rollout. Nurses confirmed that the script was appropriate and could be implemented within their workflows, and staff felt satisfied with the modifications to the patient checkout process and checklist of care activities provided by the diagnostic communication tool. In the final PDCA cycle, the modified note sheet was implemented across the entire practice, involving all physicians and staff over a three-month period.

Surveys consisting of closed- and open-ended questions were developed by the study team to collect information about patients', physicians', and staff members' experiences with diagnostic communication throughout the project. No exclusion criteria were applied to patients during intervention implementation.

Fifteen physicians and 13 nurses and staff members responded to the pre-intervention survey; 12 physicians and 8 nurses and staff members responded to the post-intervention survey. We observed little change in the proportion of physicians who agreed with the statement, “my patients come to their appointments prepared” pre- and post- intervention; fewer staff agreed with the statement that patients came to appointments prepared after the note sheet intervention. Before implementing the “Be the Expert on You” note sheet, no physicians and only 1 nurse or staff member (3.6% of the total sample) agreed with the statement, “my patients are organized with the important health information needed for me to make a diagnosis”; after the intervention, 2 physicians and 3 nurses and staff members (25.0% of the total sample) agreed with this statement. Prior to the note sheet implementation, only 1 physician and 2 nurses and staff members (10.7% of the total sample) agreed with the statement, “patients share their health stories in an efficient manner”; after implementation, more than half of physicians and nurses and staff members (60.0% of the total sample) agreed with this statement. Additionally, after implementing the note sheet, more physicians and staff collectively agreed with the statement, “my patients effectively communicate their health needs” (75.0% vs. 42.9%). In the physician respondent subsample, 83.4% of physician respondents agreed with the statement, “my patients effectively communicate their health needs” after the note sheet implementation, compared to 33.3% of physician respondents before implementation. Table 2 presents the responses from our physician and nurse and staff subsamples.

We evaluated note sheet utilization and completion rates throughout the three phrases of the implementation period. A total of 143 patients completed at least one section of the “Be the Expert on You” note sheet. In the first phase of implementation (i.e., physician champion only), all 16 patients seen in clinic by the physician (100%) completed the note sheet in the 2-week period. In the second phase (i.e., 7 physicians), 39 of the 354 (11.0%) patients seen completed a note sheet in the 4-week period. During the 6-week implementation period occurring practice-wide, 88 of the 1,100 (8.0%) patients seen in clinic completed a note sheet.

Of those 143 patients who engaged with the note sheet in some capacity, 100 patients (69.9%) completed the note sheet in its entirety (i.e., completed each of the five questions). We found that all of these patients answered the first question (“Why are you here today?”) and third question (“Have you seen anyone else about your health?”). The lowest percentage of patients (80.4%) responded to the last question of the note sheet, “Is there anything else going on?”

We received 120 patient survey responses, with 31 surveys from patients 65 years of age or older (21.7%). Survey participants were predominantly women (71.7%) and Black or African American (81.7%) (see Table 3).

A majority of older adult patients who responded to survey questions (n = 31) agreed or strongly agreed that the “Be the Expert on You” note sheet helped them to organize their thoughts (71.0%) and helped their communication with their physician (77.4%). Most older adult patients reported being satisfied with the note sheet (80.6%) and would recommend the note sheet to other patients (77.4%). More than three-quarters of older adult patients who used the note sheet felt their provider listened to them carefully during their visit (77.5%) and addressed their main concerns (80.6%). The responses from the older adult subsample, as well as from the broader patient sample, are presented in Table 4. With the exception of the item regarding recommending the note sheet to other patients, we found that the responses to all other items were similar between the older adult subsample and the full patient sample, with differences within 10%.

We asked all patients to describe any challenges utilizing the note sheet and to identify ways that the note sheet helped them with their health visit. Of the 35 free-text comments received related to challenges using the note sheet, 30 patients (85.7%) responded “none” or “n/a”. One patient reported they had “nothing to write on” and that a clipboard might be helpful; two patients reported lost information or information-related challenges. The remaining two comments described appointment details rather than the note sheet itself.

Thirty-two patients responded to the question about how the note sheet helped with their health visit. Several patients (n = 6) stated the note sheet helped them focus, for example: “It helped me stay focused on what is ailing me and tell the doctor without rambling.” Other patients stated that the note sheet helped them communicate better (n = 4) and helped to prepare or get organized prior to meeting with the physician (n = 4): “It helped me think out what I needed to ask my physician.” A few patients (n = 3) expressed that the note sheet helped the physician with the diagnosis, “It allowed the doctor to immediately identify my issues,” and one patient wrote that the note sheet “Shows that the staff is concerned about its patients.”