Standardized training for residents in medical colleges have become integral to postgraduate medical education. The education provided at the residency stage determines the career trajectory of young physicians (Ten Cate, 2017), and plays a critical role in nurturing high-level medical professionals and providing high-quality medical services.

Different from college education, standardized resident training requires a lot of clinical practice and emphasizes the cultivation of competence. This has become the core challenge and focus in current medical education reform studies. The late 20th century marked the third generation of medical education reform, and a key feature of this reform was the emphasis on competency-based medical education (CBME) (Pellegrini, 2006). CBME, an innovative practice in education, has been thoroughly examined and implemented in medical education reform in Europe and the United States, profoundly impacting the theory and practice of medical education in China (Calhoun et al., 2011). It stipulates that corresponding skills should be acquired by the conclusion of residency training (Holbrook and Kasales, 2020; Weller et al., 2020), which is the cornerstone of standardized resident training.

As a teaching hospital, teaching base of China-Japan Friendship Hospital has shouldered the clinical training responsibilities of numerous top-tier medical schools in China and has established a comprehensive evaluation system founded on CBME principles. One of the evaluation systems is Mini-CEX, a traditional teaching and formative evaluation method. The Mini-CEX grew out of the clinical evaluation exercise (CEX), a method introduced by the American Society of Internal Medicine in 1972 to assess the clinical competence of residents, especially for first-year residents. A resident is required to complete the inquiry, physical examination, diagnosis and treatment of a hospitalized patient, and the entire process is graded by a clinical teacher in about 2 h. The Mini-CEX focuses on real patients, which can make up for the defects caused by “simulation” in OSCE teaching.

However, as medical education continues to evolve, the current system has unveiled several issues. For instance, the delineation of competencies is becoming increasingly intricate (Ginsburg et al., 2010), the evaluation content and assessment procedures are cumbersome, and they consume a significant portion of the clinical work time (Leung, 2002). Furthermore, some research argued that the individual components or abilities within each competency domain do not add up to the whole of practice (Lurie, 2012). Mastery of abilities in individual competency domains does not ensure the capability to integrate them across domains or to appropriately apply them to patient care. Additionally, the ability to provide patient care in one context or clinical circumstance may not necessarily translate to other contexts and circumstances. Meanwhile, the focus on objective assessment of measurable abilities may detract attention from assessing how learners actually care for their patients in a variety of clinical work. These authors believed that performance outcomes should be framed in the context of clinical care, recognizing that professional development requires the integration of abilities across multiple competency domains and application within the health care environment (Brooks, 2009; Frank et al., 2010; Ten Cate, 2013).

In order to enable CBME to be implemented from an abstract competency framework in the ordinary clinical training process, professor Olle Ten Cate, a Dutch medical education expert, introduced the concept of Entrustable Professional Activities (EPAs) (Ten Cate, 2005). EPAs operationalize medical education outcomes as essential professional activities that one entrusts a professional to perform and involve observing residents’ professional behavior in clinical tasks such as communication, preliminary diagnosis, and differential diagnosis. Supervisors would then assign corresponding trust, clarify their rights and responsibilities, and perform a comprehensive evaluation. Whereas traditional competency frameworks focus on qualities of the person, EPAs focus on qualities of the work to be completed. EPAs therefore ground outcomes in the tasks of physicians and offer an approach to CBME that better addresses concerns around integration of competency domains and context than previous CBME frameworks. It transforms abstract ability assessment into the EPAs level evaluation of specific clinical tasks, which takes into account resident learning and patient safety, and is conducive to the implementation of CBME.

In 2016, Dwyer and colleagues utilized EPAs to gauge orthopaedic residents’ performance in managing ankle fractures, hip fractures, and total knee arthroplasty (Mulder et al., 2010). A group of seven orthopaedic surgeons, including Adam Watson from the University of Toronto, formulated a compulsory EPAs checklist for orthopaedic residency training in Ontario, Canada. This catalog of 49 EPAs sets a vital reference point for other programs to follow when constructing and assessing competency-based orthopaedic surgery curricula (Ten Cate et al., 2015). Therefore, based on previous research, we explored the application of Entrustable Professional Activities (EPAs) assessment system in orthopaedic residency training.

From January 1st 2022 to December 2nd 2022 during COVID-19 pandemic in China, postgraduate students undergoing standardized training in the orthopaedic teaching base were randomly segregated into two groups: an experimental group (Group A) and a control group (Group B), with 20 residents in each group. The training time was 40 h per week (8 h per day). General information, such as students’ age, gender, and grade, were documented. The study procedures were approved by the Ethics committee of China-Japan Friendship Hospital. Research procedures followed all relevant guidelines and met the criteria for the Declaration of Helsinki.

The Entrustable Professional Activities (EPAs) assessment system was implemented in Group A, while Group B used the Mini-Clinical Exercise Evaluation (Mini-CEX). Educators at the teaching base developed the teaching content and methods for Osteonecrosis of the Femoral Head (ONFH) to provide both theoretical instruction and practical guidance.

Upon the completion of the rotation, a theoretical and practical test was held to evaluate the effectiveness of the training. Additionally, a questionnaire survey which aimed to evaluate the effectiveness of EPAs was administered to both groups. The survey was designed around students’ levels of teaching satisfaction, knowledge mastery, and course engagement, with students providing scores based on their subjective perceptions. Each of these three categories contained five questions, each question contributing one point, making the total possible score for each category five points.

EPAs emphasize the direct observation of residents’ clinical behaviors by supervisors in a clinical setting. These observations encompass workplace-based assessments (Wagner et al., 2018) and competency-based evaluations, with judgments assigned on a scale ranging from level 1 to 5 (Table 1) (El-Haddad et al., 2016).

Five academic staff from the Education Department developed the EPAs, all with university teaching qualifications, who are responsible for the task of university education reform and optimization. The process of constructing EPAs mainly includes several steps. According to Ten Cate, EL-Haddad, Wagner et al. ‘s literature reporting method, the clinician in our teaching base constructed the EPAs by the following steps (Ten Cate and Scheele, 2007; Wagner et al., 2018; O'Dowd et al., 2019).

Forty residents participated and completed the study. Group A consisted of 13 males and 7 females with an average age of 26.4 ± 1.7 years. Group B included 9 males and 11 females, with an average age of 26.3 ± 1.8 years. No significant difference was observed in age, gender, and grade between the two groups (p > 0.05).

The content of theoretical test included choice question, short answer question and case analysis question and practical test contained basic operation skill, dressing change skill and physical examination skill. The total scores for theoretical and practical tests were 83.30 ± 8.69 in group A and 76.05 ± 10.58 in group B. The p value was 0.023, indicating a statistically significant difference between the two groups (Table 3).

Meanwhile, we conducted a questionnaire survey. For example, “The degree of participation in the admission of newly admitted patients” to gauge the level of course engagement, “The degree of acceptance of the teacher’s teaching style” to gauge the level of teaching satisfaction, “The degree of mastery of common orthopaedic diseases” to gauge the level of knowledge mastery. There were significant statistical differences in the level of teaching satisfaction, knowledge mastery, and course engagement between the two groups. Residents in group A scored significantly higher than those in group B.

Subsequently, we solicited feedback from participating clinicians. “I can really feel that the residents’ clinical behavior in the EPAs group is becoming more and more reliable and they are improving faster than I could have imagined. Meanwhile, as a teacher, the new evaluation system is simpler and easier to use,” one teacher said. They commended the new assessment system for its comprehensiveness, its alignment with actual clinical practice, and its graspable evaluation standards. The system was deemed to objectively and accurately evaluate the clinical competence of residents. EPAs efficiently mitigated randomness by observing residents’ daily clinical behaviors over time, offering a realistic reflection of their professional conduct. Furthermore, using EPAs, teaching physicians can keep track of residents’ progress and provide personalized training to those showing subpar performance. For instance, if a resident demonstrates significantly below-average basic surgical skills according to the EPAs assessment, teaching physicians can conduct additional specialized assessments through DOPS to reinforce targeted training. In summary, the EPAs received the clinical teacher’s approval, they agreed to further explore the application of EPAs in clinical teaching.

EPAs have emerged as a favored approach to bridge the gap between the Accreditation Council for Graduate Medical Education (ACGME) competencies and their practical application across various tasks (Nousiainen et al., 2017). They provide the scaffolding for CBME, aligning the anticipated performance of residents and supervisors in procedural skills (Dwyer et al., 2016). When paired with objective competency assessments, EPAs enhance the graduates’ capacity to deliver safe patient care (Watson et al., 2021). The successful completion of EPAs necessitates residents to exhibit a specific level of competence or proficiency across a range of clinical skills (Chang et al., 2013). For instance, executing surgical procedures demands not only a comprehensive understanding of anatomy, instruments, and procedural technicalities but also non-technical skills, like effective patient communication (Carraccio and Burke, 2010; Wagner et al., 2018). EPAs, being both workplace-based and competency-based, underscore the direct observation of residents’ conduct by their superiors during clinical practice. They facilitate a smoother transition from undergraduate to postgraduate and continuing education (el-Haddad et al., 2016). EPAs, as a novel competency evaluation model, concentrate on the continuous enhancement of crucial resident clinical behaviors, which are observable, measurable, and executable. Supervisors can assess resident physicians at any given time. Compared to other competency models, EPAs are more streamlined and user-friendly (Pangaro and Ten Cate, 2013). They evaluate competency levels based on the extent of completion and supervision of professional behaviors, thereby establishing a correlation between clinical “critical” behaviors and competencies (Wagner et al., 2018). Prompt feedback post-evaluation can significantly bolster the core competencies of residents (Duijn et al., 2017), and receiving multi-source feedback from residents, clinical teachers, and patients can further augment the quality of EPAs and medical care (Browne et al., 2010).

In essence, the key distinction between EPAs and other assessment systems lies in their contextual application. EPAs do not require the creation of separate assessment items, instead, they are directly implemented within clinical work, thereby reducing scoring errors (Meyer et al., 2019). EPAs emphasize the wholeness of residents’ medical behavior and its observability within the work environment, without the restrictions of a specific evaluation timeframe (Pangaro and Ten Cate, 2013). Moreover, EPAs are designed for easy comprehension, featuring straightforward evaluation criteria, and closely link residents’ clinical behavior with ability assessment (Beeson et al., 2014; Sharma et al., 2018). This approach makes EPAs an ideal tool for assessing residents’ competencies in a comprehensive and practical manner.

EPAs aim to be comprehensive, and as such, may not be as effective for first-year residents due to their relative inexperience in various clinical aspects. Therefore, evaluation methods such as Mini-CEX, SOAP, and DOPS may be more effective in assessing and nurturing specific areas of expertise. EPAs assessments can be more beneficial as a screening method in the second and third years to identify residents’ weaknesses and provide targeted training.

This study also has its limitations. Firstly, in order to ensure the residents in experimental and control groups do not interact with one another, we assigned participants to different medical groups and told them to follow their own evaluation content and assessment procedures and should not interact with residents of opposing groups. But despite this, different groups of residents still have chance to communicate with each other which may influence the results. Secondly, as a new evaluation system in our teaching base, different teachers have different degrees of mastery, which may affect the accuracy of results. Furthermore, as it was only conducted within our orthopaedic teaching base, the number of residents involved was relatively small because of COVID-19 pandemic and the 13 evaluation criteria for EPAs for orthopaedic residents require further validation through larger, multi-center studies. The quest to improve training quality, cultivate more exceptional medical talents, and incorporate EPA assessments into the existing residents training system warrants further exploration.

The EPAs assessment system in orthopaedic postgraduate residents training can improve the total theoretical and practical test score compared with traditional formative evaluation system, and can also raise the level of teaching satisfaction, knowledge mastery and course engagement. The EPAs can offer a reference way to our national orthopaedic residency training.

The datasets analyzed in this study are available on reasonable request from the corresponding author. Requests to access these datasets should be directed to FG, gaofuqiang@bjmu.edu.cn.

The studies involving humans were approved by the Ethics committee of China Japan Friendship Hospital. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

RQ: Data curation, Methodology, Software, Writing – original draft. XY: Software, Writing – original draft, Data curation, Methodology. YL: Conceptualization, Funding acquisition, Resources, Supervision, Writing – review & editing. FG: Conceptualization, Funding acquisition, Writing – review & editing, Data curation, Software. WS: Project administration, Supervision, Writing – review & editing. ZL: Project administration, Supervision, Writing – review & editing.