Eligibility criteria for this review were developed using the PICOS (Participant-Intervention-Comparator-Outcomes-Study design) format (12). Inclusion criteria for the review included original research publications related to methods used to develop competency standards in registered and self-regulating health disciplines including: Aboriginal and Torres Strait Islander Health Practice, Chinese Medicine, Chiropractic, Dental, Medical, Medical Radiation Practice, Nursing and Midwifery, Occupational Therapy, Optometry, Osteopathy, Pharmacy, Physiotherapy, Podiatry, Psychology, Nutrition and Dietetics, Exercise and Sports Science, Audiology and Speech Pathology. Development of specialist competency frameworks within a health profession were included if the results included the process of development of the framework. Specialist frameworks were defined as competency frameworks for a specialist role or those that are specific to a context, population or domain of practise (13). Method papers, such as occupational analysis, were excluded unless they were part of the methodology for the development of competency frameworks.

Publications that did not relate to the methods used to develop or revise competency frameworks, but merely published a framework, were excluded. Research related to competency framework development methodology for a health discipline not included above or pertaining to competency frameworks for preceptors were also excluded.

Literature was reviewed systematically to maximise rigour, and the protocol was registered on the PROSPERO International Prospective Register of Systematic Reviews under registration number CRD42019128350: https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=128350. Studies published between 1996 and 2021 were identified by searching MEDLINE, Pubmed, CINAHL Plus with full text, EMBASE and ERIC during March 2019. The search was updated in April 2021 to capture any additional frameworks published up until April 2021. The following key words were used to search:

[(“Aboriginal and Torres Strait Islander Health Practice” OR “chinese medicine” OR chiropract^* OR dent^* OR “radiation practice” OR nurs^* OR midwif^* OR “occupational therap^*” OR optometr^* OR osteopath^* OR pharmac^* OR physiotherap^* OR podiatr^* OR psycholog^* OR “social work^*” OR dietetic^* OR dieti#ian OR “exercise and sports science” OR audiolog^* OR “speech patholog^*” OR “medical education”) AND “competency standard^*” OR “competency framework” OR “curriculum development”].

The search was limited to English language papers published after 1996, the year of the formal adoption of CanMEDS Roles Framework, a globally recognised physician competency framework (6). This date was deemed relevant as the development of the CanMEDS framework, along with other national frameworks, are critical to the history of competency framework development in health professions, and most standards would have undergone revision since that time. Google Scholar was searched using the same search strategy (automatically sorted by relevance) to identify any publications that did not appear in the database searches. Reference lists of included literature were also hand searched.

Predefined topic and research questions guided data collection, extraction and analysis. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement guided the process of document identification, screening and eligibility assessment (14).

The electronic reference management tool EndNote X9 (Clarivate Analytics, Philadelphia, PA) was used to export and manage references. After the removal of duplicates, one author (BL) independently screened titles and abstracts and selected studies according to the pre-defined inclusion criteria. Risk of bias was managed by ensuring that the full texts of remaining papers were screened in duplicate amongst four researchers (BL, EB, KM, and CP) to identify documents and publications for inclusion. Any discrepancies were discussed in order to reach consensus. The primary author (BL) completed data extraction with oversight from EB.

To determine quality for review, the full text of each article was assessed independently by the primary researcher (BL), with a sample (20%) checked for accuracy by a second author (EB). Given the variation in research methodologies, the Critical Appraisal Skills Programme (CASP) tool was modified for use, as adapted by Halcomb et al. (15, 16). CASP is the most frequently used tool for quality appraisal and is recommended by the Cochrane Collaboration (17). Articles that did not achieve a “yes” on all items of the checklist were not excluded from the review, but the appraisal was taken into consideration in the overall rigour of the present review. Articles were included if the process of competency framework development process was described with sufficient detail.

Data was analysed using techniques of a narrative synthesis, given the heterogeneous nature of the literature (18). This process enables the researcher to construct greater meaning through a process of “re-interpretation” of published findings (19).

For the purposes of this review, we initially defined key concepts and stakeholder groups based on published definitions and author experience (Table 1).

The total yield from all databases was 10,625 results, including six articles identified through hand searching of reference lists. This was reduced to 7,918 results after the removal of duplicates. Using the exclusion criteria against title and abstract, a total of 239 full-text publications were assessed for eligibility (Figure 1). Following full-text review, 73 articles were included in the review (Table 2). The reasons for exclusion included insufficient detail on methodology and papers that did not discuss methods to develop competency frameworks.

Quality assessment identified variability in the reporting of study methodology. Rigor of data analysis was the main factor delineating methodological quality between included publications (Supplementary Material). We classified the sampling generously in the first instance, however even with this consideration, the sample size and participant profile were unclear in some publications. In many publications, while significant numbers of stakeholders may have taken part in the competency framework development process, often they were a very homogenous group. Few publications acknowledged study limitations or weaknesses.

Studies published between 1997 and 2021 were primarily peer-reviewed articles (n = 72), with one thesis (n = 1). Most publications related to specialty or context-specific competencies (n = 55), with 18 pertaining to occupational or professional practise competencies. The majority of studies were from Australia (n = 22, 30%), followed by the USA (n = 16, 22%), the United Kingdom and Northern Ireland (n = 12, 16%), and Canada (n = 7, 10%) (Table 2). Nursing competencies accounted for the majority (n = 25, 34%), followed by interdisciplinary frameworks (n = 12, 16%) and medicine (n = 11, 15%) and (Table 2).

Studies most frequently engaged four (n = 21, 29%) groups of stakeholders in the competency framework development process, albeit this varied considerably from one (n = 14, 19%) to seven groups (n = 1, 1%). There was considerable variation in the combination of stakeholder groups involved in the competency framework development process. None of the included studies provided a rationale for the number of stakeholder groups engaged in the competency framework development process.

Practitioners and academics were the most frequently engaged stakeholder groups in the development of competency frameworks in health professions outlined in the search strategy (Figure 2). Less than 20% of papers involved service users and only 14% considered the perspective of policy makers in the reported process to develop a competency framework. The least frequently involved stakeholder groups in competency framework development methodology in health professions include practitioners from other professions, new graduates or students and content and/or education experts (Figure 2). Few of the included studies provided a rationale for the stakeholder groups engaged in the competency framework development process in health professions. The findings for each stakeholder group are elaborated on below, with examples to illustrate a critical lack of diversity in stakeholder engagement in competency framework development in health professions.

Practitioners from the profession were the most frequently engaged stakeholder group in the development of competency frameworks in health professions (Figure 2). Researchers sought practitioner input across a number of methods used to develop a competency framework, in the development of a competency framework, to gather input on an existing framework and to revise, validate or reach consensus on a draught competency framework (Table 3). Practitioners were most frequently involved in studies which reported using the Delphi technique (n = 24), interviews, focus groups or a workshop (n = 17) as methods to develop competency frameworks. Though there were a small number of papers which reported using values clarification and the nominal group technique as methods to develop a competency framework, all of the papers which utilised these methods engaged with practitioners in the process (Table 3). Practitioners were often selected as Delphi panellists based on clinical expertise; with the assumption they had the necessary experience to be considered “experts” in their field (20, 37, 38, 53, 54, 72). In studies which utilised observation of practise in competency development methodology, there was little effort made to undertake observations in non-clinical settings (settings which do not include one-on-one care), such as roles in public health, management, academia, and policy.

The majority of studies (75%) involved academics in the competency framework development process. Academics participated in the initial development of a competency framework, to solicit input for changes to an existing competency framework and frequently, in the review process. Academics were most often participants in papers which reported using the Delphi technique (n = 24), or in a workshop (n = 18), and were infrequently engaged in papers which reported utilising observation of practise (n = 1). Academics were less likely to be engaged in the process via focus groups and interviews than practitioners (Table 3). In some articles, the development process was limited to the input of a small sample of local, and mostly senior, academics (39–41, 73, 82).

Healthcare employers were inconsistently (37%) involved in the competency framework development process. They were most frequently involved as a stakeholder in studies that reported use of the Delphi technique (n = 8) or a survey (n = 6), though considerably less frequently than practitioners and academics (Table 3). The perspective of healthcare employers or management was sought in the development of an initial competency framework and to revise or validate a preliminary framework. Jie and Wanyi reported including this stakeholder group to provide breadth of perspective on workforce expectation and development (70). The views of employers in the focus group methodology used by Palermo et al. were included to ensure that the competency framework conveyed the professionalism required of the future dietetic workforce (21). Vardanyan et al. reported using interview methodology to explore the perspective of medical co-ordinators on the personal and managerial skills required of pharmacists (74).

Representatives from credentialing and/or certification organisations were involved in the competency framework development process in thirty-three percent (33%) of papers, mostly via a stakeholder consultation process in the end-stages (review and validation) of competency framework development (59%) (13, 42–44, 55, 77). In some studies, the competency framework development process was limited to input from members of the working group, which often comprised of senior representatives from professional bodies engaged in credentialing (40, 56, 75).

Service users such as patients and their family, or consumer advocates were infrequently engaged (19%) in competency framework development, despite many authors claiming a focus on “patient-centred care” (Figure 2). Service users were most frequently engaged in the latter stages of the development process, via a stakeholder consultation (n = 5) to review or validate a framework and its application (Table 3). In observation of practise, there were no studies that utilised observation of interactions between clinicians and patients. Authors reported including service users, such as patient representatives and carers, in the development process to ensure that the resulting competency framework encompassed the needs of patients, families and carers (7, 9, 57). Framed as “experts by experience,” Carter et al. note that a focus group with service users helped to shape the content of the admiral nurse competency framework, based on their unique experience and expectations of care (57), though the sample was small (n = 5). Cashin et al. (4) state that partnering with service users is fundamental to the development of standards for practise for registered nurses providing person-centred health care. Yet, out of close to 10,000 stakeholders engaged in the redesign of the standards, only seven were patients or consumers. Similarly, Yates et al. (22) circulated a discussion paper outlining key roles and broad competency domains that constitute the specialist breast nursing role in Australia to 60 stakeholders for review, but only one was a representative from a consumer advocacy organisation and no other service users were engaged in the process. In the development of core competencies in cancer genetics for advanced practise oncology nurses, Calzone et al. (38) utilised the Delphi technique with an expert panel, which reportedly included consumer participants (n = 9). In this case the “consumers” were graduates of an advanced practise nursing oncology degree, selected to represent “novices in genetics” who are experienced in all aspects of oncology nursing care (38). In some studies, it was not clear how many service users were engaged in the competency framework development process (13, 68, 77). Few papers utilised values clarification as a method in the development of a competency framework, though Davis et al. (13) engaged users of diabetes services in this exercise to develop a competency framework for diabetes nursing. Kirk et al. note that engagement with patient representatives (n = 4) via a nominal group process validated the application of the competency framework to patient and family needs, and the role of the profession in meeting these needs (7).

Policy makers were rarely (14%) engaged as a stakeholder in competency framework development, despite the output of many studies containing competency items related to policy (40, 55, 62, 63). The input of policy makers was sought primarily to review a competency framework, via a workshop or consultation process, and less frequently in the initial construction of such frameworks (Table 3). Lehane et al. purposively included national policy makers (Department of Health) in focus groups in a “creative collaborative process” to develop a competency framework for clinical effectiveness education for health and social care professionals (84).

Only 12% of papers engaged practitioners from other professions in the process of development, despite competency frameworks containing competency items related to multi-disciplinary care or the ability to work in a team (22, 42, 55). The perspective of practitioners from other professions was most frequently sought using qualitative methods such as focus groups (n = 3) and interviews (n = 3) in the development of a competency framework (Table 3). Moaveni et al. (64) engaged interprofessional colleagues throughout the framework development process who were able to identify attributes essential for multidisciplinary care. In addition, these same authors reported utilising an interdisciplinary Delphi panel to provide a consensus on what the role of a family practise registered nurse might look like based on multiple perspectives (64). Similarly, Chen et al. interviewed providers from different disciplines, for variety of perspective on clinical experiences and educational needs for the in-patient provider workforce (9).

Only eight papers (11%) elected to include new graduates or students in the development of competency frameworks, most often using focus group methodology (n = 3) (Table 3). Authors reported sampling students or new graduates to capture the evolution of the profession, or to identify competencies required to work in emerging roles (1, 21).

Content or education experts (also referred to as educationalists) were engaged in 11% of publications, particularly in the end-stages of competency framework development, such as via stakeholder consultation to refine or validate a framework (Table 3). More specifically, authors reported consulting content and education experts to review a competency framework for scope, and to assist with consistency in style, language, and format of the framework. Dressler et al. note that a multiple review process including education experts brought consistency and cohesion to an initially disparate preliminary competency framework (42). Jidkov et al. interviewed stakeholders from a range of expert backgrounds, including educationalists, for diversity in attitude and perspective and to reduce bias in the development of health informatics competencies for postgraduate medical education and training in the UK (58).