The incidence of cardiac arrests ranges between one and five cardiac arrests per 1,000 hospital admissions (1), and 67–170 per 1,000,000 citizens (2). Cardiopulmonary resuscitation (CPR) is a key skill that all health care professionals should master in order to improve patient outcomes after cardiac arrests. Especially faster and more efficient in-hospital cardiopulmonary resuscitation and early defibrillation can improve patient outcomes significantly (1–3).

All health care professionals should be proficient in basic life support measures (4, 5) and should be regularly trained in small groups with a maximum instructor to participant ratio of 1:6 (6). Health care professionals who regularly treat cardiac arrests need to participate in Advanced Life Support (ALS) courses to improve their patients’ outcomes (7, 8). Those health care professionals who are infrequently involved in cardiac arrest management should at least participate in Immediate Life Support (ILS) courses (4, 9).

During a cardiac arrest, physicians usually assume the role of the team leader and need to be proficient in guiding an interdisciplinary resuscitation team through the management of a cardiac arrest. The 2015 and 2021 Guidelines of the European Resuscitation Council (ERC) emphasize that team membership and leadership training are essential (5, 9–11).

Prerequisites for proper team performance were identified earlier, and include: team leadership, task management, teamwork, situational awareness, decision-making, adaptability, event and mission analysis, and communication (12, 13). ERC ALS courses include training of technical skills, adherence to current resuscitation guidelines, and training of human factors focusing on team membership and leadership, task management, communication, and situational awareness (14).

Different approaches to assessing human factors in different fields were described (e.g., the ANTS framework, the Oxford NOTECHS, the Mayo High performance teamwork scale, the adapted LOSA checklist from aviation, the Queen’s Simulation Assessment Tool, etc.) (12, 15–18). The paradigm shift to also include human factor training in cardiopulmonary resuscitation courses took place over the last years, however, the practical end-of-course summative assessment has remained largely unchanged and tests predominantly the participants’ ability to adhere to current resuscitation guidelines. Currently, the ERC uses simulated team assessments of one candidate during the summative practical end-of-course assessment. One instructor simulates a team, the assessed person leads this “team” through a cardiac arrest scenario test. Other organizations like the American Heart Association use real team assessments, where a group of participants forms a team. One of them is then assessed as the team leader; the team members are usually not assessed and act only on the commands of the team leader. There is currently no evidence available as to what extent different assessment methods can test human factors.

Therefore, the aim of this study was to investigate which variant of these two different summative end-of-course assessment methods is perceived as being superior in their ability to assess human factors, as judged by instructors and students. This is an evaluation of the opinions of course participants and instructors, the null hypothesis was that there is no difference between leadership skills comparing both assessment methods. The results of this study might influence the development of a different assessment approach in such cardiopulmonary resuscitation courses.

A total of 427 medical students participated in the study (in 19 courses), and one declined participation. Figure 1 shows the Consort Flow Diagram. Four students had to be excluded from the analysis because they withdrew their consent to publication after participating in the advanced life support course and assessment. In the 1-day ILS-course, 111 students were randomized to the simulated team assessment, and 116 to the real team assessment (5 courses each). In the 2-day ALS-course, 106 students to the simulated team assessment (5 courses), and 90 to the real team assessment (4 courses). The 1-year follow-up reached 172 students, 78 from the simulated team assessment and 94 from the real team assessment.

A total of 54 instructors participated in the study; they were aged 39 ± 8 years; 17 (31%) were women; the primary language reported was: 51 German (94%), 2 French (4%), and 1 (2%) other; mean experience as the instructor was 4 ± 4 years; courses taught before participating in the study: 3 ± 2; 17 (31%) were registered Bern First Responders. Most instructors participated more than once.

Participating instructors judged the real team assessment overall significantly better than the simulated team assessment (p = 0.029, Table 1). All human factors were judged significantly better in the real team assessment (p < 0.01 to 0.024), and so was the ability to test CPR skills (p < 0.01). In contrast, knowledge could be tested equally well with both assessment methods (p = 0.842, Table 1).

Table 2 shows details of the participating students’ characteristics. Of note, even though courses were randomized, we found in the 2-day ALS course cohort in the real team assessment group significantly more women (67% vs. 38% in the simulated team assessment group, p < 0.01). Also regardless of randomization, more registered Bern First Responders were found in the 2-day ALS course assessed with the simulated team assessment method (15% vs. 2% for team assessment, p < 0.01).

After 1 year, in the ILS cohort, 22 students reported having resuscitated a patient in real life after passing the last year’s ILS-course (15 in the simulated team assessment group, 7 in the real team assessment group). Additional 11 students were recruited as Bern First Responders (4 in the simulated team assessment group and 7 in the real team assessment group).

Details of the students’ subjective ratings of their assessments are displayed in Table 3. Self-assessed competence as a team member was judged significantly better for the real team assessment (ILS-course p = 0.032; ALS-course p < 0.01). Real team assessment was judged consistently significantly better in the ability to assess team membership, communication skills, team management skills, and CPR skills (all p < 0.01). Situational awareness was judged only in the ILS-course cohort in favor of real team assessments (p = 0.023, Table 3).

In the 1-year follow-up, both the self-assessed competence as a team member and team leader were comparable (p = 0.194 and 0.372). The team member and team leader competences dropped for both assessment methods (all p < 0.01). All students agreed at the time of the follow-up that further training is necessary for team members and leaders (all p > 0.05).

Table 4 shows the detailed results of the assessments. ILS course participants had a first attempt cardiac-arrest simulation-test passing rate of 80–84%; whereas ALS course participants in 92-95%. The detailed results are comparable between both course types and assessments, but some items scored differently (Table 4).

Instructors of advanced life support courses judge the real team assessment overall significantly better and rate its ability to test all human factors and CPR-skills better compared to the simulated team assessment. Participating students agree that team membership and management, communication, and CPR skills were better assessed by the real team assessment. However, students rated the assessments as comparable in the following categories: the assessment in general, team leadership, and knowledge. Situational awareness was only rated better in the 1-day ILS course, whereas judged as comparable in the 2-day ALS course.

Our course participants self-assessed their competence as team members higher than as team leaders, regardless of the assessment method used. They also agreed that further training as a team leader is more important than team membership training. These findings are consistent with findings from other researchers (21, 22).

The importance of training human factors in cardiopulmonary resuscitation courses is well established (23–25). Resuscitation teams who applied human factors efficiently also performed better in technical resuscitation skills (26). If a team leader performs hands-on procedures, the whole team is less effective (27). Prior participation in an advanced life support course improved the team leader’s ability to stay hands off (27). Nowadays, all courses include team leadership training, even though there is very little evidence that such specific leadership training during those courses leads to improved patient outcome (10). However, training in resuscitation skills, in general, was able to improve patient outcome (7).

It was unclear if current end-of-course summative assessments can test these human factor competences adequately. Therefore, the current study asked participants and instructors to judge subjectively two different assessment methods, the simulated team assessment, and the real team assessment. Participants found that both assessments are equally effective to test their team leadership skills. However, the included instructors who have experience with multiple course participants have a different opinions. The instructors clearly judge the real team assessment as superior in its ability to test team leadership skills. Training leadership skills prior to being involved in a real cardiac arrest is important (28, 29). Unfortunately, our data shows only the opinions of course participants and instructors. Further research is necessary to objectively assess the competences for such team leadership factors, and if one assessment approach is suitable to assess these competences in different advanced life support formats (e.g., for neonatal, pediatric, or adult resuscitation).

Participants of a qualitative analysis of human factors identified teamwork issues as most challenging during pediatric cardiac arrests (21). Our participants rated team membership competences consistently lower than team leadership competences. The real team assessment was judged to be better suitable to test team membership compared to the simulated team assessment. The lower scores for team membership compared to the other aspects of this study introduces the question if there is a need to adjust the current assessment method to allow better assessment of team membership competences. Regardless of the assessment method used, team members are not allowed to act as “real team members” during an assessment, they only act on team leaders’ commands. Assessment of team member competences is therefore not possible. In contrast, during cardiopulmonary resuscitation courses, team membership skills are trained, and course participants are encouraged to act as they would in real life and support their team leader with all their knowledge and skills.

Communication during a cardiac arrest needs to be clear, concise, brief, empathetic, and trigger a feedback loop. It needs to be targeted directly at a person and closing the loop is advisable (30, 31). ERC cardiopulmonary resuscitation courses emphasize proper communication and specifically focus on hindering factors for speaking up, which is key for success in cardiac arrest situations (29, 30). Research showed that simple and short ongoing educational interventions on leadership principles can improve team leadership and communication competences significantly (24, 32, 33). Our results show that such communication competences can be better assessed with the real team assessment compared to the simulated team assessment.

Shared situational awareness enables better teamwork (30). Both assessment methods were rated comparably by our ALS course participants. However, ILS course participants and instructors think situational awareness can be better assessed by real team assessments. The inconsistency in these findings suggests that situational awareness might depend more on the assessment setting itself, than on the assessment method.

Interestingly, over the 1 year between the initial study and the follow-up, 11 medical students (participants of the 1-day ILS course) were successfully recruited as Bern First Responders, some of them during the ILS courses (34) (A First Responder is a trained layperson or healthcare professional who is dispatched via an app to a medical emergency in addition to an ambulance). This should be considered a positive outcome given that health care professionals might get easily overwhelmed with the additional burden of being a First Responder (35).

This study has several limitations. It was a single-center study focusing on medical students as course participants in official European Resuscitation Council advanced life support courses. The University of Bern offers a large medical program with more than 340 medical students per study year. The medical students were randomly assigned by the University to their courses. There is a slight chance that medical students assessed together also worked together during course work or clinically as a team of students prior to their participation in the respective courses, therefore, we cannot exclude an effect of this on leadership and teamwork. Groups for the real team assessment were chosen randomly, and care was taken that students who had trained together during the course were not assessed together. Although there have been many study participants in this study, there are two cohorts of participants which limits the generalizability of the data, half of the study participants participated in a 1-day ILS course, and half in a 2-day ALS course. Randomization was performed for the whole course due to practical reasons and not for the individual course participant.

On the other hand, this study has several strengths as it was planned and executed as a large randomized controlled trial and each participant was only included once. Both the instructors’ and the participants’ perspectives are included.

In conclusion, instructors of advanced life support courses rated a summative course assessment in real teams significantly better regarding the ability to test team leadership, team membership, communication, team management, and situational awareness compared to simulated team assessments, which is the current European Resuscitation Council’s standard end-of-course assessment. Course participants rated team leadership and partly situational awareness as comparable between both assessment methods. These results might influence current summative assessment practices in advanced life support courses.

The original contributions presented in this study are included in the article/supplementary material, further inquiries can be directed to the corresponding author with a dedicated research question and local ethics committee approval.

The studies involving human participants were reviewed and approved by Cantonal Ethics Committee of Bern, Switzerland. The patients/participants provided their written informed consent to participate in this study.

SN had the idea of the study, performed data acquisition, data evaluation, first manuscript, and approved final manuscript. SH and LT contributed to data evaluation, significant contribution to manuscript, and approved final manuscript. CS and AH contributed to data acquisition, significant contribution to manuscript, and approved final manuscript. RG performed data acquisition, data evaluation, significant contribution to manuscript, and approved final manuscript. All authors contributed to the article and approved the submitted version.

SN was a European Resuscitation Council—Instructor Educator Support Science and Education Committee (SEC-IES) member, and the current education representative of the young ERC group of the European Resuscitation Council. SN was also a Canadian Anesthesiologists’ Society (CAS) CEPD (Continuing Education and Professional Development) Committee member. RG was the European Resuscitation Council’s Board Director of Guidelines and ILCOR and Chair of the ILCOR Task Force on Education, Implementation, and Team. SN, CS, AH, and RG were currently Life Support instructors with the European Resuscitation Council. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.