This study of faculty experiences and beliefs took place in the context of a pragmatic educational trial, the results of which were previously published (24). We briefly summarize the trial design and intervention here. To evaluate the effectiveness of VCA feedback in resident error disclosure training, we conducted a multisite randomized trial in 2022–23 with 146 post-graduate year 2 (PGY2) residents in Internal Medicine (IM) and Family Medicine (FM). Residents received a lecture about error disclosure over Zoom from an investigator (AAW or THG), completed two VCA cases at Time 1, and were randomized to control or the intervention (crowdsourced feedback provided in the VCA app within 2 weeks for self-directed review). Residents then completed a second VCA exercise after 4 weeks (Time 2). The study found that self-directed review of crowdsourced feedback was associated with higher ratings of residents’ error disclosure skill. This manuscript describes the beliefs of residency faculty about the intervention (VCA use and crowdsourced feedback to residents) and its suitability for wider adoption.

Participants included internal medicine and family medicine faculty who teach patient safety and disclosure techniques at seven U.S. academic centers (list as follows): University of Washington (WA), University of Washington at Boise (ID), Washington State University (WA), Beaumont Health (MI), Dartmouth (NH), University of Massachusetts (MA), and Washington University (MO). All sites had participating IM programs, and two had FM program participation in addition to IM. Each program director was asked to identify at least four faculty (including core faculty, program directors, and assistant/associate program directors) responsible for training residents about medical error disclosure, who would be approached for recruitment. Up to three emails invitations were sent to eligible faculty. Faculty were paid a $60 incentive for completing the study; this amount was reviewed by the University of Washington IRB and deemed not to cause undue influence to participate.

To develop familiarity with the VCA, participants first used the VCA app to practice simulated error disclosure with the same medical error cases given to the residents in the randomized trial. They also completed a brief demographic survey within the VCA app, with questions about age, gender, race, specialty, and years in practice. After receiving personal feedback in the app in the same fashion as residents, they were invited to complete a survey and a research interview. We did not study the recordings or ratings of faculty VCA responses.

We assessed views on integrating the VCA into residency training using a widely used, reliable, brief, valid, 12-item tool (25). The tool has a 5-point scale ranging from completely disagree (1) to completely agree (5), with three subscales for Acceptability, Appropriateness, and Feasibility. Participants were also asked about their experience with coaching and instructing residents about error disclosure. The survey was coded into RedCAP and administered anonymously online (26, 27). Results were analyzed with descriptive statistics.

Two co-authors (KBB, AAW) with extensive interview experience conducted 20–30 min interviews with 25 key informants between January to June 2023. Interviews were conducted over Zoom and audio-recorded, then transcribed. All interviews followed a pilot-tested interview guide with 5 open-ended questions (Supplementary Material). These focused on general impressions, how the VCA could be incorporated into existing curriculum for teaching and assessing error disclosure, and how these assessments might be helpful for resident evaluation and/or feedback. No field notes were made, and transcripts were not reviewed by participants.

One co-author (EJG) reviewed a subset of six anonymized transcripts to develop an initial codebook. Questions from the interview guide framed initial codes, and subcodes were developed inductively. Two research associates (TM, KH), both trained in qualitative methods, reviewed five transcripts (including those used for initial codebook development and three new ones) to verify codebook comprehensiveness, confirm that key concepts and information needed for the study's objectives were identified and appropriately represented, and identify any additional emergent themes. They then refined the codebook in consultation with the original code book developers. To enhance trustworthiness, TM and KH independently coded a subset of five transcripts (20% of the total), met to reconcile discrepancies, and resolved outstanding issues through discussion with the broader team. Reflexivity was maintained through regular coder meetings and discussions of potential bias and interpretation. The team monitored for thematic saturation throughout the coding process, adding codes if additional themes emerged. Coding was performed using the John Snow NLP Lab (http://www.johnsnowlabs.com). The transcripts were coded manually without using NLP features in the coding software.

The University of Washington IRB approved all study procedures (Study 00015707). Participants provided verbal consent at the start of the interview and could opt out at any time.

Of 40 faculty members invited to participate, 29 completed the VCA exercise, and 25 completed every step, a 63% completion rate that is expected for busy clinicians. All 7 programs were represented, with a median of 4 faculty participants per site (range 1–5). Table 1 displays details of the 25 participants' demographic characteristics and prior experience with supervising and teaching error disclosure. All faculty reported previously communicating with patients about harm in care they had provided but had variable experience with coaching residents in error disclosure (Table 1).

Faculty rated the VCA with a mean AAF score of 4.23 (SD 0.78), corresponding to high acceptability with a rating of 5 as “completely acceptable”, although slight variation existed across domains (Table 2).

Faculty respondents identified several strengths and weaknesses of the VCA for error disclosure simulation. Key strengths included the VCA case quality and the tool's ability to fill a void in their current patient safety curriculum (Table 3). For example, most participants (19/25) commented that the cases were either relevant to their specialty or that they had encountered similar scenarios in their clinical work. One commented: “Almost all of those things, I’m sure have happened or will happen. I’ve experienced, almost, every one of the adverse events that was described.” Similarly, 15 of 25 faculty described the VCA actors and scripts as realistic, simulating the feeling of being in a clinical environment. Respondents noted that error disclosure was not a major part of their current curricula and that trainees should have more exposure to this content. One mentioned “We probably should have been doing this sooner, quite honestly.” The main limitation voiced by respondents (10/25) was the software's inability to create dynamic patient responses based upon the user's response, which contrasts to a standardized patient simulation with more fluid conversational exchanges.

Respondents agreed that repeated cycles of practice and feedback during residency would be ideal, if protected curricular time could be found. All faculty stated that they would like residents to use the VCA at least annually, and 19 of 25 believed practice should occur more than once a year to maintain skills. The preferred timing during the academic year varied, based upon the specific program's course schedule or rotation calendar. Regarding barriers to implementation, nearly all faculty reported that the lack of dedicated time for this type of curriculum would be a challenge. Either asking residents to complete the cases on their own time, or finding a dedicated time and physical space during other didactic time were identified to be barriers. Despite the appeal of on-demand availability, the faculty were wary of asking trainees to complete an educational task without protecting time to do so. When residents requested to complete the VCA out of class, one respondent was “kind of surprised at how many reminders I had to send.” A few programs also expressed a lack of expert faculty available to present a lecture on error disclosure.

Faculty believed the VCA might supplement other assessments of communication skills, providing data about broader progress towards communication milestones. As one envisioned, repeated VCA use would allow residents to “see their change over time…I think that'd be really meaningful to residents.” Eighteen respondents felt the VCA would be a valuable remediation tool, including one respondent who said, “For residents who have [a] challenge with patient communication, I think this could be one tool as a way to provide some sort of structured feedback.” No consensus existed about whether the tool would be more appropriate for formative vs. summative feedback. For example, one respondent preferred a formative role in which “Residents get to review their scores on their own, and it's sort of a self-driven development or learning process.” However, others envisioned gathering summative data on residents, seeing a novel opportunity to measure skills and progress towards milestones: “If we were actually, you know, acquiring summative data from this, it would be new. That said, I think particularly at a residency level, you want to demonstrate that people have competence in various domains, error disclosure being one of them.”

Regarding whether faculty should review the resident's feedback, the majority of faculty (18/25) suggested that a faculty coach would be helpful to discuss the resident's feedback with them. Reasons for this recommendation included a belief that residents would take the task more seriously, coaching would augment learning, and a desire to support learners with the lowest performance. Yet, they worried faculty involvement could increase the sense that residents were being evaluated. Faculty recommended a coach with a comfortable relationship with the resident, “not a program director.” Faculty also felt the current ∼2-week delay between VCA practice and feedback would be a barrier to coaching. Multiple faculty members expressed that more rapid crowdsourced feedback would facilitate in-the-moment coaching.

Participants requested other software features to support curricular implementation, including a summary of an individual's scores over the course of residency, and real-time tracking of whether the trainees completed the VCA for use in class. Additionally, interviewees wanted to gauge the cohort's aggregate performance and determine whether there were any outliers in the program who might need more support or resources. For example, one program director said: “I actually think the most important piece of that would be to identify outliers, especially on the low end.”