As a complex adaptive system, health care and health systems demand competent managers and transformational leaders throughout its organizations to optimize health outcomes. Frontline health managers are often overlooked in health systems strengthening research and practice, although they directly influence the health care workers and the operational levels of the health care system responsible for delivering products and services to the population (1).

In an effort to improve coverage and the quality of health care services, the Government of Cambodia introduced health care reform in 1995, classifying the public health system into national and provincial levels. In each province, operational districts were formed based on the size of the population. Referral hospitals provide level one to level three specialist services depending upon the area. Health centers play an integral role in the Cambodian health system, providing frontline health services to the public; each health center provides basic primary care services to populations of 8,000 to 10,000 persons. Health centers receive feedback and overall guidance from the community through a health center management committee (HCMC) and are under the management and supervision of the operational district (2). A health center chief leads the daily activities of the health center, generally supervising a deputy chief and at least seven additional staff members. The health center chief also operationalizes health policy from national and sub-national levels to support health center staff and the communities they serve.

Despite their integral role, health center chiefs do not receive management and leadership training prior to assuming their positions. Thus, in 2017 the National Institute of Public Health (NIPH) in Cambodia collaborated with the U.S. Centers for Disease Control and Prevention (CDC) to address this need and provide management and leadership training for its health center managers and senior staff. With the approval of the Cambodia Ministry of Health, NIPH formed a technical working group to discuss the necessary knowledge and skills a health center manager needs. The technical working group included representatives from the national and sub-national offices of the Cambodia Ministry of Health, NIPH, CDC, and the Korea International Cooperation Agency (KOICA). After a series of workshops, the technical working group developed a 6-month management and leadership capacity building program, utilizing technical support from CDC and financial support from KOICA. The program was adapted from the Improving Public Health Management for Action (IMPACT) program, originally launched by the CDC in 2016 (3).

Previous thought scholarship in global health workforce development has acknowledged that better management is a necessary component to improving global health outcomes. Bradley et al. argued that a three-part strategy is needed to build the field of management as a key pillar of global health: training and education, practice, and research (4). Additionally, Yeager and Bertrand note that little evidence exists to inform public health management needs and best practices in global health settings (5–7). Thus, this paper will assess evaluation findings from the IMPACT Cambodia program and evaluate these findings against the World Health Organization's (WHO) key components of a well-functioning health system (Table 1). While these findings are specific to the context of Cambodian health centers, we hope this evaluation adds to the growing body of research around the impact of skilled public health management on health systems. In sharing the findings of our evaluation, we aim to contribute to the discussion around the importance of public health management in health systems strengthening.

The NIPH Technical Working Group selected Battambang and Preah Vihear provinces in northern Cambodia to pilot the IMPACT Cambodia program. Participants were nominated by their Provincial Health Directors and selected by NIPH. IMPACT Cambodia aimed to build capacity through interactive classroom training and regular mentorship at health center sites. Classwork was comprised of 6, 1-week modules covering topics listed in Table 1. The last module of the program was a symposium, providing participants with the opportunity to share how they have implemented lessons learned. After the first five training modules, a team of mentors visited each health center to provide technical guidance on fellows' activities, answer lingering questions from the modules, and provide direct coaching to the fellows. NIPH recruited seven individuals from its staff to serve on the mentorship team; mentors were not involved in the technical working group or other aspects of program implementation. However, mentors were provided with mentorship training by CDC prior to program initiation in August 2018.

The evaluation of the IMPACT Cambodia program utilized a mixed methods approach, employing a baseline and endpoint competency assessment, observational visits, and interviews. The technical working group provided direction and technical support for program implementation and evaluation design. The evaluation's goal was to determine the extent to which the program reached its intended outcome (participants' increased management competence) and what impact the program had on graduates and health centers.

The evaluation received a non-research determination by the CDC Center for Global Health's Institutional Review Board (Tracking number: 2018-169) and approved by the Cambodia National Ethics Committee for Health Research (#134NECHR; 21 June 2018). The key ethical principles of voluntary and informed participation, confidentiality and safety of participants was used in evaluator and participant interactions. Participants were provided with written information about the evaluation, informed that their participation was voluntary, and that they may withdraw from participation at any time.

From June 2018 to January 2019, data were collected from the 20 participants (nine health center chiefs and 11 health center staff members) from 10 health centers in Battambang and Preah Vihear provinces in Cambodia. Quantitative methods were used to measure the change in participant competence and at the health centers; qualitative methods were deployed to provide context to those changes. Analytic approaches varied by data source and type. Data sources and respective analytic approaches included the following:

From these findings, we determined that the program did achieve its intended outcome—participants reported an increase in competence over seven key management areas. Through observational visits and interviews, we found that participants applied what they learned from the program. This finding is consistent with findings from the Philippines Field Management Training Program, a management capacity building program for local health workers similar to IMPACT. The Philippines Field Management Training Program found that “the results…provide compelling near-term evidence of how the skills from the training are being used by participants to identify, prioritize, and solve day-to-day managerial problems” (14).

These gains in management competence may not have been possible without mentorship support. Post-training support is a necessary aspect of effective training design (15). IMPACT mentors provided guidance and support while participants were at the health centers, bridging the gap between classroom training and experiential learning. One participant shared that it was easier to apply what they learned in the classroom with the mentors' support and guidance. For future health workforce capacity building programs, funders and program implementers should consider mentorship an essential aspect to ensure participant understanding and skill application.

During baseline observational visits, we saw that health facilities largely did well in the areas of service delivery, health information system, and health financing. However, endpoint interviews captured how the participants used the training to improve factors outside of the indicators in the quantitative checklist, particularly in the areas of service delivery, community engagement, and self-efficacy. These factors are demonstrative of more comprehensive management and leadership capability.

The WHO's key components of a well-functioning health system can provide a useful guide for funders and program implementers in how they build and tailor their health workforce capacity building programs, as well as how they measure the impact of these programs on the health system. We found the training positively contributed toward building four out of six of the WHO's building blocks for a well-functioning health system—leadership and governance, health information systems, human resources for health, and service delivery (Table 6) (8). We found no evidence that capacity improved around health financing and essential medical products and technologies.

For leadership and governance, participants' creation of plans, and more importantly, staff involvement in creating those plans, can help improve health outcomes in the complex, ever-changing environment of the health system (16). Yet, effective leadership and governance are only possible with accurate information on the health challenges and context in which the health system operates (8). To that end, by identifying and analyzing the geographical locations within their catchment areas where residents who utilized the least amount of health services lived, then developing targeted plans for community outreach aimed at encouraging those community members to visit the health center, participants will be able to improve access to care and the quality of services provided.

Leadership, governance and health information systems are cross-cutting components that provide the foundation for all the other building blocks, yet human resources for health is a key input in a well-functioning health system (17). A strong health workforce responds to the needs and expectations of the community, and is fair and efficient in achieving health outcomes with available resources (8). Through conducting formal performance reviews with staff, using a more collaborative leadership approach, encouraging staff to share ideas about decisions that affected the health center, recognizing positive performance and providing encouragement and motivation to staff, participants are creating enabling work environments for the health workforce, which encourage them to focus on the needs and expectations of the community.

Health systems can only be successful in producing health outcomes if they deliver the necessary services. Service delivery crucially depends on the development of standards and guidance to ensure access and quality of care (8). Through developing and utilizing SOPs, instituting patient triage systems, improving WASH practices at the health centers, and creating and displaying a map directing patients where to access health services, participants are ensuring access, greater satisfaction, and higher quality of care for their patients and the community.

While the training did not appear to have an impact on the two remaining building blocks of a health system (health financing and essential medical products and technologies), this could be due to decentralization in the Cambodian health system. Implementation of health programs is the responsibility of provincial health departments rather than the Ministry of Health. Due to decentralization, provincial health departments are the focal point for allocation of resources to operational health districts and health centers (18). Therefore, we believe expanding public health management training to health center staff nationwide, particularly to provincial health department managers, would be beneficial for Cambodia's health system.

One strength of this evaluation is the utilization of a mixed methods approach, which permitted a triangulation of results that discretely quantitative or qualitative research approaches alone would not have achieved. A single methodology would have been inadequate to examine the impact of the training program in a complex environment such as a health system (19). While evaluation findings are not generalizable (20, 21), the mixed method approach is replicable and may prove beneficial for other countries or funders to evaluate their health workforce capacity building programs. Yet we agree with Yeager and Bertrand that evaluations of health system strengthening programs, particularly ones focused on management and leadership, require a significant investment of resources, which may be a deterrent for some funders or program implementers (5).

This evaluation has several limitations. First, self-report data may lead to a misestimation of abilities when compared with objective assessments. This may be compounded by issues of comprehension and memory recall. To address this response bias, we used the observational visit checklist as an objective assessment at baseline and endpoint to compare with the interview data. Second, these evaluation results are not generalizable outside of this program in Cambodia, but do demonstrate the short-term results of the implementation of such a program (20, 21). Third, since we used human translation to translate collected data from Khmer to English for analysis, it is possible that context was lost in translation. Fourth, we did not validate participants' interview responses with independent sources, so it's possible our own personal experiences may have resulted in methodological bias. To address this bias, we've attempted to clearly and accurately present participants' experiences in all cases (22). Lastly, the evaluation was constrained by time and our inability to assess the long-term impact of the training on graduates and the health outcomes in their communities. A long-term study that compares health centers with IMPACT graduates to those without IMPACT graduates and utilizes data from Cambodia's Health Equity and Quality Improvement Program could potentially measure the cascading effect of training participation on health outcomes.

The evaluation findings may have several practical implications for funders and program implementers, particularly those engaged in building and tailoring health workforce capacity building programs. In terms of program design, purposeful mentorship may be instrumental to ensuring participant understanding and skill application. In addition, trainings focused on competency development for frontline health managers and providers can be highly effective in skill-building and behavior change in health facility operations. Further, focused training of this kind may lead to demonstrations of enhanced leadership, self-efficacy, cooperation amongst staff, as well as improved service delivery and community engagement.

The evaluation approach may also provide lessons for global health capacity-building programs. Rigorous evaluations of such programs do require a significant investment of resources. However, this investment is critically important to understand the possible effects of these programs. Moreover, they are important to building the evidence-base around workforce capacity development efforts, particularly in the areas of health system management and leadership where there has been limited study.

We hope this evaluation encourages further research into the impact of skilled public health management on health systems, not only in low- and middle-income countries, but also in well-resourced nations. Although the Global Health Security Agenda identifies improved public health management capacity as essential for achieving its goals (23), many health systems still do not invest in or prioritize developing public health management capacity (24). The COVID-19 pandemic has elevated the urgency of this work, particularly in the managerial areas of coordination, community engagement and trust, planning, and adaptive leadership (25). Thus, continued investment and evaluation of workforce development efforts of this kind may be critical to addressing underlying health system issues and improving public health outcomes.

The de-identified data that supports the findings of the study are available on request from the corresponding author.

The studies involving human participants were reviewed and approved by U.S. Centers for Disease Control and Prevention Center for Global Health's Institutional Review Board and the Cambodia National Ethics Committee for Health Research. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

KH, PW, PI, CC, MK, VL, and EW equally involved in the conceptualization and design of the program. CH led the data collection, analysis, and interpretation. CH, KH, PW, and CC drafted the manuscript. CH, PI, MK, VL, and EW provided critical revisions to the manuscript. All authors were involved in designing the evaluation of the program and approved the final version of the manuscript.

This publication was supported by Grant or Cooperative Agreement numbes 5UGH001873 and 1NU2HGH000044-01-00, funded by the U.S. Centers for Disease Control and Prevention. This project was supported by grant numbers 5NU2GGH000990-05-00 and 6NU2GGH002124-01-01, funded by the Korea International Cooperation Agency. This project was supported in part by an appointment to the Research Participation Program at the U.S. Centers for Disease Control and Prevention administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. Department of Energy and the U.S. Centers for Disease Control and Prevention.

The views expressed in the submitted article are solely the responsibility of the authors and do not necessarily represent the official views of the U.S. Centers for Disease Control and Prevention, the U.S. Department of Health and Human Services, the Korea International Cooperation Agency, the Task Force for Global Health, Inc., or Training Programs in Epidemiology and Public Health Interventions Network.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.