Front. Public Health Frontiers in Public Health Front. Public Health 2296-2565 Frontiers Media S.A. 10.3389/fpubh.2024.1364798 Public Health Review Interventions addressing routine childhood immunization and its behavioral and social drivers Parsekar Shradha S. 1 * Vadrevu Lalitha 1 Jain Monica 1 Menon Shekhar 2 Taneja Gunjan 2 1International Initiative for Impact Evaluation (3ie), New Delhi, India 2Bill & Melinda Gates Foundation, New Delhi, India

Edited by: Chizoba Wonodi, Johns Hopkins University, United States

Reviewed by: Pragya Sharma, Maulana Azad Medical College, India

Rita Gill Singh, Hong Kong Baptist University, Hong Kong SAR, China

Mary Carol Jennings, Johns Hopkins University, United States

 *Correspondence: Shradha S. Parsekar, sparsekar@3ieimpact.org 19 06 2024 2024 12 1364798 16 01 2024 05 06 2024 Copyright © 2024 Parsekar, Vadrevu, Jain, Menon and Taneja. 2024 Parsekar, Vadrevu, Jain, Menon and Taneja

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

 Introduction

Despite the advances in vaccination, there are still several challenges in reaching millions of children in low- and middle-income countries (LMICs). In this review, we present an extensive summary of the various strategies used for improving routine immunization in LMICs to aid program implementers in designing vaccination interventions.

 Methods

Experimental and quasi-experimental impact evaluations conducted in LMICs evaluating the effectiveness of interventions in improving routine immunization of children aged 0–5 years or the intermediate outcomes were included from 3ie’s review of systematic reviews. Some additional impact evaluation studies published in recent years in select LMICs with large number of unvaccinated children were also included. Studies were coded to identify interventions and the barriers in the study context using the intervention framework developed in 3ie’s Evidence Gap Map and the WHO’s Behavioral and Social Drivers (BeSD) of vaccination framework, respectively. Qualitative analysis of the content was conducted to analyze the intervention strategies and the vaccination barriers that they addressed.

 Results and conclusion

One hundred and forty-two impact evaluations were included to summarize the interventions. To address attitudinal and knowledge related barriers to vaccination and to motivate caregivers, sensitization and educational programs, media campaigns, and monetary or non-monetary incentives to caregivers, that may or may not be conditional upon certain health behaviors, have been used across contexts. To improve knowledge of vaccination, its place, time, and schedule, automated voice messages and written or pictorial messages have been used as standalone or multicomponent strategies. Interventions used to improve service quality included training and education of health workers and providing monetary or non-monetary perks to them or sending reminders to them on different aspects of provision of vaccination services. Interventions like effective planning or outreach activities, follow-up of children, tracking of children that have missed vaccinations, pay-for-performance schemes and health system strengthening have also been used to improve service access and quality. Interventions aimed at mobilizing and collaborating with the community to impact social norms, attitudes, and empower communities to make health decisions have also been widely implemented.

 behavior and social drivers child immunization interventions low- and middle-income countries zero-dose children INV-030670 Bill and Melinda Gates Foundation (BMGF)10.13039/100000865 section-at-acceptance Public Health Education and Promotion Introduction

Despite the proven benefits of vaccination (1–3), a recent estimate shows that over 100 countries have stagnating or declining third dose of diphtheria-tetanus-pertussis (DTP3) coverage since 2019 (3, 4). An important factor in exacerbating this stagnation or decline is the COVID-19 pandemic and the resultant lockdown (5, 6). Even prior to 2019 and COVID-19, although there was a substantial improvement in immunization coverage, some of the countries struggled to improve or sustain coverage (2). Despite efforts to strengthen immunization services by national governments and international organizations (1, 3), childhood vaccination uptake differs across countries due to diversity in barriers to immunization, distribution of vaccination resources, and complexity in implementing the vaccination policies and programs (7). Identifying evidence on what interventions work for the uptake of childhood immunization, on whom, how, and why they work are important public health aims (7).

To improve the evidence base on community engagement interventions, International Initiative for Impact Evaluations (3ie) commissioned several evaluations of such interventions under the its Innovations in Increasing Immunization Evidence Program and prepared a series of synthesis products: Evidence Gap Map (EGM) (8) to map impact evaluations and systematic reviews of interventions for childhood immunization uptake, systematic review assessing effect of community engagement interventions on childhood immunization uptake (9), and a systematic review of systematic reviews (RoR) assessing effectiveness of different types of interventions for improving childhood immunization uptake in low- and middle-income countries (LMICs) (10). In the current review, we build on this existing work and use the intervention framework developed and described in 3ie EGM (8), to provide a compendium of interventions that have been implemented in LMICs to improve the routine child immunization outcomes.

We further analyze the barriers to vaccination uptake, classified using the WHO’s Behavioral and Social Drivers (BeSD) of Vaccination framework (11), that were potentially addressed through these interventions. The barriers to vaccination uptake which have been widely reported in the LMIC settings range from practical barriers such as time constraints, costs, health care access and caregivers’ knowledge to negative attitudes toward vaccination and fear of its adverse effects to social-cultural norms (12–16). A couple of recent review of reviews have systematically reviewed the evidence on interventions that have been effective in improving child vaccination uptake in LMICs. Besnier et al. (17) finds mixed effects of health promotion interventions but positive effects of interventions aiming to improve immunization communication, education, and social mobilization on vaccine uptake. The RoR (10) finds that caregiver-oriented interventions, especially those focusing on short-term sensitization and education campaigns as well as written messages to caregivers, are effective in improving vaccination uptake in LMICs. It also finds that community-oriented interventions are effective. Among the health system-oriented interventions it finds positive effects of home visits and mixed effects of pay-for-performance schemes.

While the existing literature provides evidence on barriers to vaccination uptake and to some extent on interventions to improve childhood vaccination uptake, it does not link the interventions to barriers being addressed in a structured manner, which we do in this paper. We also provide more information on key intervention features and implementation characteristics that are potentially useful for policy. Apart from standalone strategies, we also examine the different components of interventions that were co-implemented and whether the interventions were found to be successful in improving vaccination and/or its intermediate outcomes in the included evaluation studies.

 Materials and methods

This is a summative review of the evidence from impact evaluations of interventions that measured outcomes pertaining to vaccination or its causal mechanisms, and their associated literature.

Inclusion criteria and source of evidence

We included primary studies cited in the 62 systematic reviews included in 3ie’s RoR (10). These systematic reviews were published until October 2021 assessing effectiveness of interventions on childhood vaccination. We prepared a list of 1,062 primary studies that were cited in 62 systematic reviews. These studies were screened for inclusion based on following inclusion criteria:

Experimental and quasi-experimental studies establishing counterfactuals to determine the causal impact of an intervention in comparison to standard practice or alternative treatment.

Studies conducted in LMICs, determined by its World Bank classification at the time an intervention was carried out.

Studies assessing any intervention impacting routine immunization outcomes of children aged 0–5 years or the intermediate outcomes in the causal mechanisms (e.g., caregiver knowledge of childhood immunization).

Studies published in English.

Further, to make this review more expansive, we also searched 3ie’s EGM 1 (8) to include additional impact evaluations that were published between 2010 and 2020 and conducted in select LMICs with large number of unvaccinated children: India, Indonesia, and Pakistan from South and East Asia region; Ethiopia and Nigeria from Africa region; and Brazil and Guatemala from Latin America region. We ensured that these additional impact evaluation studies satisfied the above listed inclusion criteria and are not duplicates of the included primary studies from 3ie’s RoR.

 Data extraction and synthesis

The details of the study components, context, geographies, and implementation process was extracted from the studies using the NVIVO software (Version 1.6.1). The Intervention Framework developed by 3ie (8) was used to classify the interventions and extract data. The framework broadly groups the interventions into caregiver-oriented, health system-oriented, community level, and policies and institutions. These broad themes are further divided into multiple level-two and -three interventions (Appendix 1 in Supplementary material). Furthermore, information pertaining to type of intervention, year of intervention, intervention objective, outcomes, country, contextual characteristics of the population such as economic or educational status, etc. were mapped. We also extracted data on the barriers addressed by these interventions and categorized them as per BeSD framework. The BeSD framework (9) was adapted for coding the information pertaining to barriers of childhood vaccination. The framework (9) divides the factors driving vaccine uptake into four broad drivers (1) thinking and feeling, (2) motivation, (3) social processes, and (4) practical constraints. Additionally, political climate, natural calamity, migration, and socio-economic characteristics that may affect the uptake of vaccination were captured. Codes pertaining to program or intervention characteristics, reasons for intervention success or failure, and impacts on drivers of vaccination were also created. Coding framework can be found in Appendix 2 in the Supplementary material.

Two researchers coded the data by dividing the included impact evaluations between them. Prior to this, an iterative process was followed to achieve coding consistency between the coders by piloting the coding framework on six studies. A detailed comparison table containing the researchers’ coding was created and discussed during a weekly meeting to identify the differences in coding and to standardize the definitions of the codes used. This process continued until there were minimal differences in the coding done by the two researchers. The coding framework and the definitions used for each of the codes were finalized at this stage. Redundant codes were deleted, memos or notes were added as per need, and the merged file was finalized for use in analysis.

We used a qualitative approach to synthesize the findings. We assessed the context, geographies, and program implementation process of the included studies. In addition, we examined the year-wise trends of interventions in included studies. We also segregated standalone and multicomponent strategies that addressed barriers of childhood immunization using BeSD framework. Within multicomponent strategies, each component was further classified as per the intervention framework developed in 3ie’s EGM (8). Within each intervention category, we briefly defined and described the intervention, provided the frequencies of identified studies and the context where they were conducted. For multi-component strategies, we also provided details of co-interventions. Further, within each intervention category, we listed the studies that had positive influence of intervention on at least one vaccination outcome. For this review, the impact evaluations that assessed multiple arms, each arm was counted as a separate intervention. We also provided a summary of the interventions for childhood immunization in LMIC contexts from this evidence base by linking the programs to the BeSD barriers.

 Results Description of included studies

Overall, 142 impact evaluations [116 from 3ie’s RoR (10) and 26 from 3ie’s EGM (8)] were included to synthesize the information on interventions and barriers. Included studies were mostly conducted in low-income settings within the countries. Some interventions targeted underserved hard-to-reach (HTR) regions having predominantly indigenous populations, slums, or street children. Many included studies were conducted in India (n = 42), followed by Nigeria (n = 13) (Supplementary Figure 3A). Some studies were conducted in conflict affected countries like Afghanistan, Burundi, Democratic Republic of Congo, Pakistan, Palestine, Nigeria, Ethiopia, and Guatemala as well as those prone to natural calamities such as floods, like India, Bangladesh, Indonesia, and Nigeria. Impact evaluations of interventions in populations with high degree of migration were from countries like India, Indonesia, and Nigeria. Details of the context and interventions of included impact evaluations can be found in Appendix 3 in the Supplementary material. Majority of studies (n = 111) were published between 2011 and 2020 and the oldest study was published in the year 1986 (18) (Figure 1).

Year wise distribution of impact evaluations by intervention type.

 Description of included interventions

Evaluations of both standalone and multicomponent interventions have increased steadily since 2006 (Figure 1). In the 2001–2010 decade, the studies mostly focused on evaluating effectiveness of informing, educating, and incentivizing caregivers and training of health workers and health system strategic planning (Figure 2). In the 2011–2020 decade, there was an all-around increase in interventions oriented toward the caregiver, health system, and community. Among the caregiver-oriented interventions, there was a sharp jump in evaluations of reminder and recall interventions overlapping with increasing usage of mobile phones. Among the health system-oriented interventions, there was a big increase in evaluations of those training health workers, carrying out home visits and outreach as well as health system strategic planning. In addition, there were substantial number of evaluations of HMIS/Dashboard interventions. Furthermore, there was a sharp increase in evaluations of community engagement interventions.

Intervention components (both standalone and multicomponent).

The included impact evaluations investigated effects of multiple standalone or multicomponent strategies on immunization and its intermediate outcomes and one or more of the BeSD barriers to vaccination at various levels. We grouped these interventions based on the barriers that they address and described them below.

Strategies addressing barriers as per broad thematic areas of BeSD framework and their outcomes are listed in Table 1 (standalone strategies) and Appendix 4 (multicomponent strategies) in the Supplementary material. Table 2 is a summary of some of the multicomponent strategies that were co-implemented.

Standalone strategies addressing behavioral and social drivers of vaccination and their impact on immunization.

Interventions Barriers addressed Intervention effect on immunization as reported by included studies
Think and feel Social processes Motivation Practical barriers Statistically significant positive effect Insignificant effect
Short-term sensitization and education campaigns Yes – Yes Yes n = 8 (19–26) n = 1 (27)
Public information campaigns Yes – Yes - n = 1 (28) 0
Material/monetary incentives for caregivers – – Yes Yes n = 11 (29–39) n = 5 (18, 40–43)
Automated voice messages to caregivers Yes – Yes Yes n = 2 (44, 45) n = 1 (46)
Written or pictorial messages (SMS, stickers, flyers etc.) to caregivers Yes – Yes Yes n = 11 (37, 47–56) n = 2 (46, 57)
Formal/Community health worker training and education – – – Yes n = 3 (45, 58, 59) 0
National/sub-national immunization days – – – Yes n = 1 (60) 0
Material/monetary incentives for health workers – – – Yes n = 1 (61) 0
Non-monetary/material incentives – – – Yes n = 2^ (62) n = 1 (62)
Pay-for-performance schemes – – – Yes n = 6 (63–68) n = 8 (69–76)
Health system strategic planning – – – Yes n = 2 (77, 78) 0
Changes to broader governance systems (beyond health systems) – – – Yes n = 1 (79) 0
New HMIS/Dashboard systems (incl. Improved data collection) – – – Yes n = 1 (80) n = 1 (81)
Collaborating with whole community – Yes – – n = 1 (82) n = 1 (83)
Collaborating with selected community groups and networks – Yes – – n = 1 (37) 0
Non-health/education infrastructure (e.g., electrification) – – – Yes n = 4 (84–87) 0

In some of the included studies, it was not explicitly stated that the interventions addressed a particular barrier(s). We made reasonable assumptions on the barriers being addressed by considering the intervention components and causal pathways. For instance, pay-for-performance schemes can potentially improve performance of health workers thereby improving caregiver’s experiences of healthcare quality. Another example is prolonged exposure to media (public information campaigns) can potentially motivate caregivers. Furthermore, some of the strategies might also influence social processes, e.g., sensitization programs, but unless intervention included community mobilization activities or mentioned influencing the community, we did not mark them as intervention addressing social norms.^A study by Lee (62) assessed effect of three separate interventions (two of which were in Kenya and one in India).

Co-implemented multicomponent strategies.

Interventions Caregiver oriented interventions Healthcare provider-oriented interventions Community oriented interventions
Caregiver oriented interventions Information or education strategies Reminder/recall (88–90) Incentives for CGs (91) Incentives for HWs (92) Home visits/Outreach to vulnerable populations (88, 92–98) Health system strengthening and HW training (55, 89, 91, 92, 94, 96, 99–104) Community mobilization (55, 88, 96–98, 101–107)
Material/monetary incentives Reminder/recall (57) - Reminder/recall and Incentives for HWs (108–111) New HMIS/data collection systems or community tracking (57, 110) Health system strengthening and HW training (91, 110–112) -
Mobile phone-based voice or text messages Educational strategies (19, 88–90) Incentives for CGs (57) Reminder/recall (113–115) New HMIS/data collection systems or community tracking (45, 80, 89, 113–118) HW training (45, 89, 113) -
Healthcare provider-oriented interventions HW training and education Reminder/recall and Educational strategies (45, 89, 94, 99–104, 119, 120) Incentives for CGs (111) Incentives for HWs (92, 111, 119, 121) Home visits and other outreach activities (119, 120, 122–126) Health system strengthening (100, 102, 119–121, 123–125, 127–137) Community mobilization (101–104, 120, 124–126, 136, 137)
Outreach to vulnerable populations Educational strategies (92) Incentives for CGs (138) - - Health system strengthening and HW training (125, 126, 139–141) Community mobilization (125, 126, 139–142)
Home visits and other outreach activities Educational strategies (88, 92–98, 119, 120) - Incentives for HW (119) - Health system strengthening and HW training (96, 119, 120, 122–124, 126, 143, 144) Community mobilization (96–98, 120, 124, 126, 144–146)
Material/monetary incentives for HWs Educational strategies (92) Incentives for CGs (108, 109) - New HMIS/data collection systems (121) Health system strengthening and HW training (112, 121) -
Automated voice/written or pictorial messages to HWs Reminder/recall and Educational strategies (113–115) Incentives for CGs (110) - New HMIS/data collection systems or community tracking (113–115, 147) Health system strengthening and HW training (110, 113) -
Health system strategic planning and other health system strengthening interventions Reminder/recall and Educational strategies (92, 95, 100–103, 119, 120, 137) Incentives for CGs (110, 148) Incentives for HW (92, 119, 121) Home visits and other outreach activities (119, 120, 123–125, 139–141) HW training (100–103, 119–121, 123–125, 127–137, 149–152) Community mobilization (101–103, 120, 124, 125, 136, 137, 139–141, 153–156)
New HMIS/Dashboard systems Reminder/recall (80, 113–118) - Reminder/recall (113–115) Home visits and other outreach activities (143, 144) Health system strengthening and HW training (113, 144, 147) Community mobilization (144)
Community oriented interventions Community mobilization Reminder/recall and Educational strategies (88, 96–98, 101–107, 120, 137) - - Home visits and other outreach activities (88, 96–98, 120, 124–126, 139–141, 144–146) Health system strengthening and HW training (96, 101–104, 120, 124–126, 136, 137, 139–141, 144, 153–156) -

CG, Caregivers; HW, Health workers. Study-wise details on intervention components, intervention addressing broad thematic area of BeSD framework and whether these were successful in improving immunization outcomes can be found in Appendix 4 in Supplementary material.

 Interventions addressing attitudes and knowledge about immunization

These interventions include informational and educational strategies such as short-term and sustained sensitization campaigns as well as public information campaigns (8).

Short-term sensitization and education campaigns as a standalone strategy

Short-term sensitization and education campaigns are one-time interventions or those designed with a fixed end date in mind (8). There were nine studies using short-term sensitization campaigns as standalone strategies conducted in urban Indonesia (27), Pakistan (19, 20), China (21), and predominantly rural India (22–26). All of these studies, except one (27), found that the interventions were successful2 in improving immunization and intermediate outcomes such as maternal knowledge of immunization. These campaigns used video-assisted teaching packages, face-to-face health education and information, education and counseling strategies delivered via discussions, posters, leaflets, and flipcharts. The place of intervention was home, health facility, schools, places of worship or community centers. The interventions intended to improve knowledge about vaccine preventable diseases by focusing on benefits of vaccines, place of vaccination, and vaccination schedule. Interventions were either facilitated by physicians, medical students, trained researchers, community health facilitators or midwives. The focus of all interventions was on women during their pregnancy or childrearing, but one study also included school students (25).

 Short-term sensitization and education campaigns combined with other strategies

This review identified nine studies assessing short-term sensitization and education programs in combination with other strategies conducted in low-resource settings. Of these, six studies found that the interventions were successful in improving immunization and intermediate outcomes such as maternal knowledge of immunization outcomes (19, 88, 91, 93, 94, 105). The strategies co-implemented with short-term sensitization were media campaigns, mobile phone-based reminder/recall services, community engagement, and strategies to improve service quality by conducting outreach sessions like home visits and community health worker (CHW) trainings. For example, interventions in Ghana (93) and Nepal (99) included CHWs conducting short-term sensitization sessions during home visits. CHWs counseled the mothers regarding vaccination and reminded of the forthcoming appointments. The two studies from India conducted in the state of Uttar Pradesh combined short-term sensitization campaign with mass media campaigns, and reminder/recall services using technology. In one study, the intervention was delivered face-to-face in the community, and automated voice calls and a toll-free number were used to sensitize the community (88). While in another study, CHWs carried out video screening in the community (94). In a study conducted in Pakistan, center-based sessions were held to disseminate the information and motivate the caregivers about immunization, which were combined with reminder type immunization card for caregivers (19). While focus was on women, one intervention also included husbands and other family members, communities, and CHWs (94).

The other studies that had short-term sensitization intervention combined with other strategies (89, 91, 105, 157) are described under incentives and reminder/recalls to caregivers and community engagement strategies.

 Sustained sensitization and education campaigns combined with other strategies

Sensitization and education campaigns that were not designed with a fixed end date in mind were coded as sustained sensitization and education campaigns (8). Sustained sensitization and education of caregivers was used only in conjunction with other strategies. This review identified 16 studies that assessed these campaigns combined with other strategies. Of these, nine studies found that the interventions resulted in improved vaccination outcomes. The sustained sensitization strategies included health education by health professionals, trained peer educators, and women’s groups. The sustained sensitization strategies were combined with media campaigns, strategies for improving quality of services by conducting outreach vaccination services to vulnerable population, providing training to health workers, performance-based incentives, health check-ups, and tracking communities, and boosting political commitment and budgetary support. The examples of these programs were Growth and Development Monitoring Program in Colombia (158, 159), peer education intervention in Nigeria (95), and Muskan Ek Abhiyan in India (92).

 Public information campaigns as a standalone strategy

Public information campaigns refer to any mass media campaigns done through newspapers, social media, radio and television (8). A single study assessed the effect of social media platform, Twitter (now “X”), in Indonesia and demonstrated positive effect on knowledge and awareness about immunization. It was a nationwide dissemination campaign involving high-profile celebrities with millions of followers to retweet content focusing on improving knowledge and attitudes about immunization posted by the university students (28).

 Public information campaigns combined with other strategies

There were five multicomponent studies having one of the components as public information campaigns. Of these, all studies, except one (100), found that the interventions were successful in improving immunization outcomes. An example of public information campaigns was, a smiling sun campaign, a national health communication program in rural Bangladesh. It was launched to promote uptake of health services by disseminating key health messages through the logo posted at multiple clinic locations and through 26-episode television series, advertisements, radio spots, posters, newspapers, and local publicity (90).

 Behavior linked incentivized interventions for caregivers

Any incentives, such as material, monetary, or non-monetary perks that were used to nudge the mothers or caregivers to change their behavior toward immunization or health services were grouped under behavior linked incentives (8).

Monetary or non-monetary incentives for caregivers as standalone strategy

Most of the incentivized programs used conditional cash transfers (CCT) to nudge mothers toward certain healthy behaviors or provided financial aid in accessing health services. The programs established conditionalities on utilizing and retaining school attendance and/or utilizing key health services for children (e.g., immunization) and pregnant women, with varying cash benefits offered. Conditional or unconditional cash transfers or food coupons as standalone strategies were implemented to motivate caregivers and offset opportunity and transport cost, thereby intended to improve maternal and child health (MCH), basic education and alleviate poverty. The brief description of the strategies by broad types is given below.

Conditional monetary transfers at the household/individual level

This review identified 11 CCT programs, of which nine studies found that the intervention was successful in improving immunization outcomes, while two failed to improve immunization outcomes (40, 41). The Bolsa Familia Program in Brazil (29, 30, 42), the Program Keluarga Harapan (PKH or “Hopeful Family Program”) in Indonesia (31–33), Program Familias en Acción in Columbia (34), Progresa or Oportunidades Program in Mexico (35), Red de Protección Social in Nicaragua (35), and monetary program in Zimbabwe (41) transferred cash to poor households when they complied with conditions related to health and education. Similarly, the Subsidy Reinvestment and Empowerment Program (SURE-P) in Nigeria gave cash incentives to women residing in rural underserved areas, who sought maternal healthcare services such as the child births and post-natal care (PNC) at the facility (40). In India, caregivers were provided with a cash incentive for every girl child that was immunized and enrolled in the school (36).

 Village level performance incentives

Villages in Indonesia under the National Program for Community Empowerment (PNPM) Generasi received an annual block grant to be allocated to any activity that supported indicators of health and education service delivery. The study found that the program was successful in improving immunization outcomes (33).

 Conditional non-monetary transfers

This review identified four studies that provided non-monetary transfers to families upon receiving childhood immunization shots. Of these, three studies found that the interventions were successful in improving immunization outcomes while one study found that the intervention (18) was not successful. Incentives included mobile credit on the phone in India (37), food/medicine coupons or food supplements in Pakistan (38) and Nicaragua (18) and insecticide-treated bed nets in Malawi (39).

 Unconditional cash transfers

This review identified two unsuccessful unconditional cash transfers (UCTs) at individual or household level conducted in India (43) and Zimbabwe (41).

 Monetary or non-monetary incentives for caregivers combined with other strategies

This review identified 10 studies that implemented caregiver monetary or non-monetary incentive schemes as a multicomponent strategy conditional upon utilizing important health services. The other components combined with caregiver incentives were sensitization and education campaigns, mobile phone-based reminders to caregivers, improving healthcare service quality by introducing HMIS/Dashboard systems, training, incentivizing and reminding health workers and involving them in planning and monitoring, conducting outreach immunization services to vulnerable populations, tracking children for immunization, strengthening of peripheral health services that included setting up quality improvement teams and their training, and supplementary vaccination campaigns.

Conditional monetary transfers for caregivers

This review identified three studies demonstrating improved immunization outcomes by implementing CCT programs. A monetary voucher program (programa de asignación familiar or family allowance program), in Honduras, provided monthly vouchers to households belonging to vulnerable groups with pregnant women or children under 3 years complying preventive health visits and if the child aged 6–12 years were enrolled in school. The program also strengthened peripheral health services by equipping with essential medicines and supply, upgrading health centers, and recruiting and training quality improvement teams (91). The Janani Suraksha Yojana, a CCT program in India, gave cash incentives to mothers that sought maternal health care services such as the institutional child births and post-natal care. The program also incentivized CHWs (namely ASHAs) who encouraged and assisted mothers in receiving health services (108, 109).

 Conditional non-monetary transfers for caregivers

There were seven studies that assessed non-monetary incentives in the form of materials such as insecticide treated bed nets, hygiene kits, lentils, phone talk time, and rewards. Of these, four studies found that the interventions were successful in improving immunization outcomes, while three studies (110, 111, 160) found that the interventions were not successful. The examples of conditional non-monetary transfers were, integration of insecticide treated bed nets with measles vaccination campaign in Madagascar (148). In Nicaragua, under Red de Protección Social progam, households received a food security transfer in alternate months based on mothers attending health education workshops held every other month and children under 5 years being brought for scheduled preventive health check-ups (160). In India, immunization camps in underserved areas were held monthly to improve outreach services along with providing small incentives such as lentils and thalis when the child was fully immunized (138). SMS reminders were provided to mothers regarding the next appointment, and they received the mobile phone credit upon getting their child vaccinated in another Indian study (57). The program in Bangladesh distributed vouchers to pregnant women entitling them to key antenatal and PNC services and cash stipends for transportation (161). A study in Ethiopia implemented “Protected Children” posters that incorporated a stamp system upon completion of each immunization visit. After fully immunizing the child, the mother received a completion symbol representing non-monetary reward. The intervention was also combined with tracking children, and poster acted as a reminders for health workers (110).

 Other strategies

Intended to improve socio-economic and nutritional status of destitute women and their children, the Rural Maintenance Program (RMP) was implemented in Bangladesh. Women received skill training in multiple activities required for income-generation and maintaining health and wellbeing. They also received small loans and micro-credits for income generation. The RMP was also linked with health centers to receive priority services. The program was not successful in improving immunization-related outcomes (157).

 Interventions addressing practical constraints faced by caregivers

Interventions that addressed practical constraints faced by caregivers in the uptake of immunization services (8). Interventions such as reminder or recalls to address lack of knowledge of immunization time, place and schedule, and strategies or programs to address availability, accessibility and quality of vaccination or health services are described in this section.

Reminder or recall for caregivers as a standalone strategy

Reminder or recall strategies consist of reminding caregivers about upcoming immunization session, its place and time and may also contain motivational information. These strategies have been delivered via voice or text messages using mobile phone and/or paper-based printed material such as MCH handbook, flyer, or stickers.

Automated voice messages to caregivers

This review identified two studies conducted in Mumbai, India (44) and Ibadan, Nigeria (45) that found automated educational and/or reminder voice messages to caregivers were effective in improving vaccination outcomes.

 Written or pictorial messages to caregivers

These include SMS, flyers, or stickers provided to caregivers. There were 11 studies that used written and pictorial messages as standalone strategy to remind mothers, mostly from low-income status, about upcoming immunization schedule and/or to disseminate information related to immunization. Of these, except one (57), all demonstrated positive effect of intervention on immunization-related outcomes. SMS-based reminders prior to scheduled immunization date or informational text were disseminated to mothers/caregivers in the studies conducted in Nigeria (47, 48), Guatemala (49), Pakistan (50), India (57), and Zimbabwe (51). Furthermore, in Indonesia, stickers were added to the home-based records to remind caregivers of their upcoming appointments (52). Similarly, CHWs in Pakistan, with the help of pictorial cards disseminated educational informational about vaccination benefits, place of vaccination, and importance of retaining immunization card (53). Studies in Ethiopia assessed the impact of reminder stickers (54) and in Bangladesh and Kenya (55, 56) evaluated MCH handbook. The handbook served as a tool for health information and education, reminders, and health record keeping.

 Automated voice and written or pictorial messages to caregivers

Two studies used combination of automated voice calls and text messages to caregivers. Of these, one demonstrated positive effect of intervention on immunization (37). The Phone Reminder for Immunization (PRIMM) program in Nigeria combined text messages with automated voice messages (46). While a study in India, in addition to reminders, disseminated information offsetting misconception regarding immunization (37).

 Other innovative ways of reminder or recall for caregivers

Innovative interventions such as the bracelets were used to track children and act as reminder for caregivers in Pakistan. Each time the child came for vaccination with the bracelet, a hole was punched in the symbol denoting the vaccine that was received on that visit and on being fully vaccinated the “star” symbol was punched. The study found that the intervention was successful in improving immunization outcomes (162).

 Reminder or recall for caregivers combined with other strategies MCH handbook combined with other strategies

Maternal and child health handbook is a booklet containing information about pregnancy, childbirth, and child health and serves as health record, e.g., vaccination. There were two studies that assessed the effect of MCH handbook on immunization outcomes that co-implemented sensitization campaigns and strategies to improve service quality such as health worker training. Of the two studies, one found that the intervention was successful which has been described under short-term sensitization strategies (19). The other intervention was implemented in Palestine, wherein MCH handbook was delivered to women during pregnancy registration at the clinic and the nurse counseled the women about the pregnancy and child care services. They also instructed women to retain the MCH card to keep vaccination and growth records. The nursing staff received 1-day training to use MCH handbook effectively (89).

 Mobile-phone based reminders to caregivers combined with other strategies

This review identified 11 studies that used multicomponent strategies having one of the components as voice or text message reminders to caregivers about upcoming vaccination schedule and/or educational information. Of these, eight interventions were found to be effective in improving immunization-related outcomes. The other strategies used in addition to text or voice calls were sensitization and educational campaigns, improving health service quality by providing training to health workers, reminder/recall phone-based messages for health workers, implementing new HMIS or dashboard systems, capacity building for existing system, home visits and tracking children. For example, in Nigeria, one arm of the study assessed combination of cell phone reminder calls and training of health providers. Calls were made 1 and 2 days before the scheduled immunization date and repeat calls were made if the child missed immunization (45). The details of other multicomponent studies (57, 80, 113–118) that had voice or text message reminders to caregivers are described under sections new HMIS or dashboard system and interventions addressing attitude and knowledge about vaccination (88, 90).

 Strategies to improve access to vaccination center

Strategies that focused on improving access to immunization services such as road improvement programs, and outreach activities are discussed in this section. In India, two studies assessed the impact of Prime Minister’s Rural Road Program (Rural Gram Sadak Yojana). The studies found that the construction of roads would lead to increased accessibility to healthcare facility by caregivers (84) and improved health care supply, increased household income, increased awareness, and improved social interaction in the village (85).

In addition to improving access to roads, outreach to vulnerable groups and migrant populations was also used in some interventions. In Bangladesh and India, outreach activities were undertaken as a part of supplementary mass vaccination campaign to influence delivery of vaccines to the community (60, 163). Another study implemented community-based intervention in underserved urban migrant community living in slums of Punjab, India, through immunization outreach clinics to improve access. This was supported by the community nominated guardians to carry out post-immunization surveillance (142). All these studies demonstrated positive effect of intervention in improving immunization-related outcomes.

There were other multicomponent strategies focusing on outreach and home visits; these are described under interventions addressing health service quality and community engagement.

 Interventions to improve health systems New HMIS/dashboard systems (incl. improved data collection)

This review identified 14 studies that tested mobile-based applications to be used by health workers for data collection or tracking children. Most of these studies employed multicomponent strategies that also included reminder voice or text messages to caregivers and/or health workers, capacity building of health workers to use these applications and outreach activities. Of the 14 studies, six found that the intervention was effective in improving immunization-related outcomes.

Mobile application-based studies conducted in low resource settings of India (ImTeCHO), Bangladesh and China used reminder messages to caregivers and health workers. The m-health application was also used to improve data collection and tracking children (113, 114, 117). In ImTeCHO and ReMinD projects in India, CHWs (ASHAs) received daily work assignments and decisional support on the mobile application. ASHAs also used phone to show video clips to mothers during their home visits (81, 113). Similarly, a study in Ethiopia introduced m-health application for health extension workers (HEWs) to register pregnant mothers and children, receive reminder messages about scheduled antenatal or vaccination date, planning resources and tracking mothers and children who missed scheduled appointments (147). The RapidSMS program of Rwanda tracked mothers and children till 2-year of age and sent SMS reminders to both the providers and the mothers regarding follow-up visits (115). Whereas, in Guatemala under the Coverage extension program (Programa de Extensión de Cobertura) using dashboard, a list of mothers and children eligible for preventive health services was generated and shared with health workers. Health workers then visited the households to remind caregivers about the scheduled appointment (143). In a study conducted in Côte d’Ivoire, through mobile web service platform mothers received either SMS or voice call reminders about the upcoming immunization schedule and two more reminders for non-compliance (118). Another example is the Khushi Baby program, a digital pendant worn by child and voice reminder platform introduced in underserved areas of Udaipur, India. Automated voice reminder calls were sent to caregivers about the upcoming immunization camps using dashboard. Khushi baby app was used by the health workers to scan the pendant to update real-time summary statistics such as immunization records and document other necessary health system related information. The dashboard also helped CHWs in planning and decision-making (80, 116).

Prinja et al. (81) and one arm of Nagar et al. (80) assessed the effect of New HMIS or Dashboard systems as a single component, while the rest were combined with other interventions.

 Strategies to enhance capacity and/or performance of health workers Health worker training as standalone strategy

This review identified three studies which found employing health worker training as effective standalone strategy for improving immunization outcomes. Two studies conducted in Ibadan, Nigeria using a standard manual provided refresher training to primary healthcare immunization providers (viz. nurses, midwives, community health officers, and community HEWs) (45, 58). In Indonesia, on-the-job training was provided by the senior nurses to less-experienced or underperforming immunization nurses. Training was about maintaining quality of vaccines, practical advice, instruction on operating the information system and discussion of ways to increase coverage. Additionally, trainers received recognition, a paid trip to the host health center, and formal credit toward advancement (59).

 Health worker training combined with other strategies

This review identified 36 multicomponent interventions that recruited and/or trained human resources. Of these, majority, i.e., 22 studies found that the interventions were successful in improving immunization outcomes, while 14 studies found that the interventions were not successful in improving immunization outcomes (89, 99–101, 111, 119, 120, 122, 127–130, 149, 164). The training of healthcare providers was often combined with sustained sensitization programs, media campaigns, improving health service quality by improving outreach activities including home visits, supplementary immunization activities, nonmonetary incentives, pay-for-performance, supervising staff, involving health workers in planning and monitoring, upgrading the infrastructure, and new HMIS system and community engagement strategies. Training was done either as a part of a national health system strengthening initiatives or as an intervention to improve quality of childcare. For example, in India formal health workers and CHWs received trainings to strengthen their skills and knowledge, and encourage them to mobilize and motivate women for service utilization including immunization (111, 119, 122, 149, 150). Similar examples come from the continuous quality improvement initiatives (151) and Essential Services for Health project in Ethiopia (121), Midwife service scheme (100) and addressing missed opportunities for immunization (131) in Nigeria, the government and non-government collaborative project in Rwanda (164) and the program in Senegal (127). A study in Georgia, implemented a set of activities that included development of supportive supervision guidelines for district immunization managers, district-level training in continuous supportive supervision, monitoring and evaluation of performance, and funding for district center for public health to carry out the package of interventions (132).

There were other strategies that included training of frontline health workers, which are described under health system strategic planning and community engagement strategies.

 Standalone monetary incentives for health workers

In Pakistan, a standalone strategy on time preferences to customize for polio vaccinators was successfully used to motivate health workers (61).

 Standalone non-monetary incentives for health workers

This review identified three interventions conducted in Zambia and India wherein non-monetary or material incentives were provided to health workers. Two of these were found to be effective in improving immunization-related outcomes. In Zambia, intervention included promotion prospects and career advancement (e.g., nurse, clinical officer etc.) for newly recruited Community Health Assistants (CHWs). Based on performance, CHWs were eligible for in-service training for career advancement freely by the government. In the second experiment, nationwide 1-year training program was provided in Zambia for CHWs to be regularized in services. In the third experiment, a self-tracking app delivered information that made efforts more intrinsically rewarding for CHWs (ASHA) workers in India. ASHAs tracked their performance through data visualization (or audio version) and was dependent on their own submission via phone-based reporting tool (62).

 Monetary incentives for health workers combined with other strategies

Monetary incentives that are conditional on performance of the provider were used to incentivize health workers and other providers to improve health service delivery. Studies included five multicomponent strategies on monetary incentives for providers (92, 108, 109, 111, 161). These strategies were found in combination with sustained sensitization campaigns, incentivizing caregivers and improving health service quality by providing training and non-monetary incentives to health workers. Of the five studies, all studies, except one (111), found that the intervention was successful in improving immunization-related outcomes.

 Non-monetary incentives for health workers combined with other strategies

These include two studies in which material or non-material incentives were provided to motivate the healthcare providers to improve their performance. These were combined with service-level quality improvement strategies such as training and incentivizing health workers. Of these, one study found that the intervention improved vaccination outcomes (121). The other study was Team-based Goals and Incentives Initiative in India, in which CHWs (ASHAs and auxiliary nurse midwives) got quarterly incentives that typically consisted of stoves, casseroles, storage containers, or other similar household items. An additional so-called bumper prize (a pressure cooker) and a certificate was given at the end of the year to CHWs in those sub centers that successfully met their targets in all four quarters (165).

 Performance-based financing as standalone strategy

The performance-based incentives refer to monetary transfers to health center/facility for reaching a certain service delivery target(s) and may include some proportion of transfers going to the providers. This review identified 14 studies that assessed such strategies. Of these, six studies found that the strategies were successful in improving immunization-related outcomes (63–67, 69). These studies have been conducted in Haiti (66), Afghanistan (70), Rwanda (71–73), Burundi (63), Democratic Republic of Congo (69, 74, 75), Cameroon (65, 68), Tanzania (67, 76), and Zambia (64).

Performance-based financing as a multicomponent strategy was implemented in several contexts. For example, in India it was implemented as part of Integrated Management of Childhood Illnesses (IMCI) program (119), which found insignificant effect on immunization outcomes. Another example was the Nigeria State Health Investment Project to improve quality of care. Funds transferred to the health facility were used for operational costs and healthcare provider bonus. Results indicated that the immunization-related outcomes improved (112).

 Health system strategic planning Health system strategic planning as standalone strategies

Health system strategic planning refers to a broad array of strategies intended to strengthen health system. These include initiatives at the national or sub-national level to develop plans and governance structures designed to improve vaccination or other MCH services. Additionally, these include interventions that improve the human resource availability, strategies, policies, and plans in existing health governance and delivery structures that may or may not be directly related to immunization services (8). For example, as a part of Family Health Program in Brazil, provision of comprehensive care was done through multi-professional teams. These teams were responsible for promotion, prevention, and care of mothers and children in a designated area (77). In Mali, intermittent preventive treatment of malaria at the time of routine vaccination was introduced irrespective of presence of malaria symptoms or infection in children (78). Both interventions demonstrated improved immunization outcomes.

 Health system strategic planning as multicomponent strategies

This review identified 37 studies that implemented health system strategic planning components; of these 16 studies found that the intervention demonstrated improved immunization-related outcomes. The strategies identified from included studies were mostly large-scale programs intended to improve service delivery by addressing shortage of human resources and conducting outreach activities. The strategies included provision of comprehensive care, recruiting and training of CHWs, volunteers, religious leaders and other front-line workers, measures to improve quality of services, upgradation of services, and equipping with essential drugs and supply.

Many of the interventions using health system strategic planning were large-scale programs for engaging community and recruiting and training new cadres of CHWs to depute them to peripheral health centers to strengthen promotion and preventive health activities. These CHWs conducted home visits to provide essential services at the household-level, track children for immunization and encourage the community for uptake of health services (e.g., immunization). CHWs were paid honorarium for their services. To support the CHWs, peripheral health centers were created or upgraded and equipped with drugs and supply to manage basic health services. These programs were implemented in Bangladesh—IMCI (120), Ethiopia—Community-based Health Services Extension Program (123), Zambia—Community health assistant program (133), and South Africa—Ward-based outreach teams (134). Some of these programs in addition to above listed strategies also included other intervention components. For example, in Bangladesh, other stakeholders such as religious leaders, health workers and theater artists were trained to convey health messages to address IMCI issues (120). In India, under Integrated Child Development Services Scheme local village women, namely Anganwadi workers (AWWs) with the help of helpers delivered the health, nutrition and pre-school services (153). In China’s Gasu province, health sector reform by reducing the cost of care at the point of care was implemented. The strategies were also implemented to protect and finance old commune-based insurance scheme, and introduction of health expense safety nets for the poorest to cover MCH services (130).

The other components listed under health system strategic planning included introduction of auto-disabled syringes, reducing cost of care, intermittent malaria treatment, supply and replenishment of HIV testing commodities and cognitive behavioral therapy. For instance, a quality improvement approach namely, Client-Oriented, Provider Efficient Services was implemented to strengthen health system and support IMCI efforts in Guinea and Kenya. It was a participatory, problem-solving approach by trained health provider; wherein providers identified problem areas, suggested solutions and took actions. External facilitators oriented district supervisors and intervention site managers, who then supervised the intervention (135). A study in Madagascar introduced auto-disabled (AD) syringes in national immunization program. Additionally, training of health workers on national immunization policies, open vaccine vial, and AD syringe usage was provided (129). In Pakistan, in a program called Thinking health, a cognitive behavior therapy-based intervention was implemented for depressed perinatal women delivered by trained lady health workers. The intervention was guided by techniques of active listening, collaboration with the family, guided discovery, and homework (152). In Zambia, the intervention included integrating early infant HIV diagnosis with the expanded program on immunization. Supply and replenishment of HIV testing commodities and a sensitization meeting with facility staff was held by district health officials (128).

The other multicomponent studies that included health system strategic planning strategies are described under interventions addressing attitudes and knowledge of immunization, community mobilization and training of health workers.

 Other strategies

The Ethiopian government introduced Fiscal decentralization project, wherein more powers were given to district and urban administrators in order to improve service delivery (86). Another example was a rural electrification program (Jyotigram Yojana) in Gujarat, India. The motivation of the project was that by improving electricity to the rural areas, access to information via television and internet would improve and households would see time endowment effect. Electricity would also improve cold chain management at the facility level (87). Furthermore, in the state of Andhra Pradesh, India, women’s political reservation was assessed. The intervention was theorized on the basis that female leaders would favor policies useful to human capital, and they would increase investment in education and health care (e.g., vaccination) (79). All these three studies demonstrated positive effect of interventions on immunization-related outcomes.

 Intervention strategies used to mobilize communities and influence social norms

Interventions aimed at mobilizing and collaborating with the community to impact social norms, attitudes, and empower communities to make health decisions have been widely implemented. These strategies either collaborated with the whole community or select community groups such as self-help groups, village committees, religious groups, or women’s organizations. Most of these strategies were multicomponent in nature.

Community engagement or mobilization as standalone strategy

This review identified three studies that engaged community (whole or select groups) as a standalone strategy. All the three studies were conducted in India. Of these, two studies found that the interventions were successful, while one study found that the intervention (83) was not successful in improving immunization-related outcomes. In one study, Village Health and Sanitation Committees (VHSCs) were tested as part of the National Health Mission. VHSCs focused on political agency wherein community members were consulted for resource allocation at the village level. VHSCs developed village health plans and managed an untied fund to enable local planning and action. They organized health-promotion activities and mobilized pregnant women and children to access maternal and health care services (82). Another intervention was a participatory approach in Mumbai slums to identify problems, potential solutions, planning, implementing, and monitoring potential solutions and information sharing (83). Furthermore, one of the arms of a study conducted in Haryana, carried out a gossip seed (social network) experiment by engaging selected community members. The study identified key community members having wide community network (or gossip seeds), were trusted (trust seeds), were trusted with wide network (trusted gossip seeds) or any individual (random seeds) to spread information related to immunization (37).

 Community engagement or mobilization as multicomponent strategy

This review identified 27 multicomponent strategies that included components of collaboration with select community groups or the whole community. Of these, 21 studies found that the intervention improved immunization-related outcomes. Interventions included building community groups or leveraging existing groups to increase awareness and decision-making among caregivers. Collaboration strategies also included mobilizing the community to identify community resources and needs to demand health care. The strategies also engaged community to strengthen peripheral health service provision. These strategies were combined with sustained sensitization activities, media campaigns, interventions to improve service quality such as capacity building of health care workers, health system strengthening, introducing new HMIS or dashboard systems and improving outreach activities.

The community engagement strategies included identifying and training suitable community members/committees from different positions and background which could be newly selected members or members from recognized community groups/association, health providers and policymakers from a designated area/community. These committees discussed important health topics (e.g., barriers to vaccination or healthcare access) using participatory approaches and took actions to mitigate them. The committee cultivated accountability, shared the necessary information, used the resources responsibly and helped in improving governance. The health workers, research team, community themselves or NGO monitored and evaluated the progress made by community members/ committees. The program strategies also included dissemination of information to a larger community through sensitization and media campaigns or with the help of local leaders. The examples of community engagement strategies are Evidence-based discussion intervention in Pakistan (105), Bottom-up community engagement intervention in Ghana (154), Health governance intervention in Afghanistan (155), Community Health Strategy in Kenya as a part of National Health Sector Strategic Plan (136), and Community-based monitoring intervention in Uganda (156). Furthermore, in Bangladesh, participatory women’s groups (106) and existing rural self-help organizations (137) were trained and engaged to promote health in the community. In India, examples of community engagement initiatives were Mahila Samakhya, a women’s empowerment program to set up women’s group in underserved regions of Bihar belonging to socially and economically marginalized backgrounds (107) and social accountability interventions in Uttar Pradesh (145). A study in Nigeria, engaged and trained traditional and religious leaders to promote the active participation and positive engagement of communities in immunization. Leaders were also engaged in maintaining cold chain (96).

The multicomponent programs intended to improve targeted services in some of the underserved regions also engaged community. These programs identified and trained CHWs to provide essential services in underserved areas by carrying out outreach activities and home visits. The program also strengthened the peripheral services and ensured supply of vaccines and logistics. The local or religious leaders supported the CHWs or co-managed the multicomponent programs by initiating dialogs, mobilizing community, and outreach activities. The examples were Community Health Strategy in Kenya (124, 136), a multicomponent strategy using m-Reach platform in Uganda (125), and in Nigeria, sensitization and mobilization of mothers (97) and using trained community volunteers and CHWs as part of IMCI program (102). The other programs that included community engagement strategies alongside strengthening of peripheral health services and outreach activities were conducted in other contexts also. For example, in India a community resource center strategy in Mumbai slums (146), Stimulate, Appreciate, Learn, and Transfer community engagement approach project (98) and Reaching Every District strategy in Assam (144) and Community Mobilization and Integrated Planning (Bachpan Program) in underserved tribal regions of Madhya Pradesh (103). Similarly, in Guatemala, to address transport and informational barrier, under Coverage Extension Program (Programa de Extensión de Cobertura), provision of services in underserved community was made by contracting out of primary health services to non-governmental organizations (NGOs). Through mobile teams, the NGOs provided the health services to HTR areas at least once a month. Additionally, community facilitators recruited and supervised local staff (e.g., midwives), who helped in managing logistics of service provision (139–141). Furthermore, as part of Maternal Neonatal Child Survival program in Bangladesh, health services provision and sensitization were made at the community and facility by collaborating with whole community (101). A study in conflict-affected regions of Pakistan facing polio resistance engaged community members and political leaders and intensified immunization activities by conducting outreach camps with strict monitoring and supervision (126). In mountainous regions of Pakistan, community were engaged by forming community health committee as part of community-based perinatal and newborn preventive care package (104).

 Conclusion and implications for research and practice

The summative review article serves as a brief on the wide range of strategies used to improve immunization outcomes and links them with the BeSD barriers and sub-barriers being potentially addressed. Interventions included in this review are majorly focused on improving knowledge on immunization benefits, awareness regarding the schedule of vaccination, motivation of caregivers, and quality of health services. Interventions focusing on improving attitudinal and knowledge related barriers to vaccination, used sensitization programs and public campaigns alone or in combination with other strategies. Many of the interventions on conditional monetary or non-monetary incentives were government programs and were used for motivating caregivers into adopting healthy behaviors. Many interventions used m-health interventions for reminding caregivers regarding upcoming vaccinations and the place and time of vaccination. These strategies were also used to disseminate information related to the benefits of vaccinations and motivate caregivers. To address access related barriers, outreach services, home visits and road improvements strategies were implemented. Interventions used to improve service quality used training and education of frontline health workers, as well as providing monetary or non-monetary perks to encourage or remind them to provide vaccination services, pay-for-performance schemes and health system strategic planning. Most of these strategies were combined with others to improve service delivery. Many of these interventions were part of government programs that focused on existing resources, e.g., ASHAs in India, and health extension workers in Nigeria and Ethiopia. Most of the interventions on community mobilization were multi-component interventions that leveraged community to address the attitudinal and service delivery aspects of immunization.

It is to be noted that while we provide information on whether the program improved vaccination outcomes by studies, we cannot draw inferences on the effectiveness of a specific strategy as this evidence has not been critically appraised. Also, the review did not intend to assess the cost-effectiveness of the interventions as typically only a handful of evaluations provide the required information (9). Another limitation of this study is that while we summarize a large evidence base of 142 impact evaluations, because of resource constraints, we could only include primary studies which were cited in included reviews in the RoR (10) and some recent ones which were available in 3ie’s EGM (8) from select countries with large number of unvaccinated children. In addition, the information within the studies did not allow us to distinguish barriers to vaccine uptake by the vaccination status of children. For example, we could not assess if barriers for zero-dose children are different from those for drop-outs. Also, the findings were dependent on the study context and the degree of reporting in the manuscript. Not all studies clearly mentioned the barriers that were addressed by the intervention. Therefore, we made assumptions and identified the barriers in the study context from the objectives of the intervention strategies and causal pathways. For instance, pay-for-performance schemes can potentially improve performance of health workers thereby improving caregiver’s experiences of healthcare quality and hence the vaccine uptake.

While we acknowledge the limitations mentioned above, we also recognize that there is a dearth of high-quality systematic reviews that rigorously synthesize evidence on effectiveness of interventions by their types (10). In addition, the current literature on interventions provides only a very high-level overview of interventions (17, 166, 167). In such a situation, the current review fills an important learning gap by providing detailed summary of strategies from 142 impact evaluations. Considerable information on the intervention components, activities, and implementation features has been extracted and synthesized in a way that is easy for use by program managers and policy makers. It allows them to be informed on a range of strategies which have been tested to address a barrier along with the program contexts in which they have been tested. This review also separates the standalone strategies from those which have multi-components. For the latter it also provides information on the other major components as well as the broad barriers they potentially aimed to address. This allows the program managers to know the combinations in which multi-components interventions have been tested and their contexts. To summarize, this review allows the program managers and policy makers who are exploring various strategies to improve vaccination uptake (a) to know which strategies have been tested and in which combinations, (b) to know the context in which they have been tested, and (c) to easily identify and explore the studies as their references are provided.

This review also highlights critical areas for future policy relevant research design. Many of the national and regional level interventions included in this review had multi-components with complex impact pathways, but only a few included a detailed theory of change (ToC). To understand how and why an intervention impacts an important barrier subsequently influencing uptake of vaccination, a well-developed ToC with carefully laid out impact pathways is helpful. To evaluate these pathways and key underlying assumptions in a ToC, a mixed-methods evaluation design with an integrated process evaluation is important. However, only a few studies had this design and were mostly 3ie funded (37, 94, 96, 98, 116, 125). In addition, it is not only important to assess what works but also at what cost, therefore, future studies could explore this important research gap. Furthermore, better reporting in future research studies on various barriers in the study contexts as well as interventions components will be useful for identifying the link between the two. The article hopes to serve as a quick reckoner for program managers in LMIC context.

 Author contributions

SP: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing. LV: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing. MJ: Conceptualization, Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing. SM: Writing – review & editing. GT: Writing – review & editing.

 Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by the Bill and Melinda Gates Foundation (BMGF) through INV-030670.

We would like to thank the participants of the stakeholder engagement meeting in January 2023 for their significant feedback, Maren Duvendack, University of East Anglia, United Kingdom, for her inputs, and Sebastian Martinez at 3ie for his overall guidance and support to the review.

 Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

 Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

 Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2024.1364798/full#supplementary-material

1The EGM includes 226 completed and 26 ongoing impact evaluations, and 58 completed and one ongoing systematic reviews. The search of EGM was comprehensively done and was last updated in May 2020.

2We have considered an intervention to be effective/successful as long as the evaluation study found that it improved at least one immunization-related outcome, irrespective of whether it was primary or secondary outcome.

 References GAVI (2022). Reaching zero-dose children: GAVI: the vaccine alliance. Available at: https://www.gavi.org/our-alliance/strategy/phase-5-2021-2025/equity-goal/zero-dose-children-missed-communities UNICEF (2019). Leaving no one behind: All children immunized and healthy. Philippines Immunization Unit, Program Division, Data and Analytics Section, Division of Data, Research and Policy, and Division of Communication. The UNICEF Philippines Country Office. WHO (2022). Immunization coverage: Key facts Geneva: World Health Organization. Available at: https://www.who.int/news-room/fact-sheets/detail/immunization-coverage UNICEF (2022). Immunization: UNICEF. Available at: https://data.unicef.org/topic/child-health/immunization/ Lassi ZS Naseem R Salam RA Siddiqui F Das JK. The impact of the COVID-19 pandemic on immunization campaigns and programs: a systematic review. Int J Environ Res Public Health. (2021) 18:988. doi: 10.3390/ijerph18030988, PMID: 33499422 Muhoza P Danovaro-Holliday MC Diallo MS Murphy P Sodha SV Requejo JH . Routine vaccination coverage—worldwide, 2020. MMWR Morb Mortal Wkly Rep. (2021) 70:1495–500. doi: 10.15585/mmwr.mm7043a1, PMID: 34710074 Ekezie W Awwad S Krauchenberg A Karara N Dembiński Ł Grossman Z . Access to vaccination among disadvantaged, isolated and difficult-to-reach communities in the WHO European region: a systematic review. Vaccine. (2022) 10:1038. doi: 10.3390/vaccines10071038, PMID: 35891201 Engelbert M Jain M Bagai A Parsekar SS. Improving routine childhood immunisation outcomes in low-income and middle-income countries: an evidence gap map. BMJ Open. (2022) 12:e058258. doi: 10.1136/bmjopen-2021-058258, PMID: 36356993 Jain M Shisler S Lane C Bagai A Brown E Engelbert M. Use of community engagement interventions to improve child immunisation in low-income and middle-income countries: a systematic review and meta-analysis. BMJ Open. (2022) 12:e061568. doi: 10.1136/bmjopen-2022-061568, PMID: 36351718 Jain M Duvendack M Shisler S Parsekar SS Anda MD. Effective interventions for improving routine childhood immunisation in low- and middle-income countries: a systematic review of systematic reviews. BMJ Open. (2024) 14:e074370. doi: 10.1136/bmjopen-2023-074370, PMID: 38365291 Shapiro GK Kaufman J Brewer NT Wiley K Menning L Leask J. A critical review of measures of childhood vaccine confidence. Curr Opin Immunol. (2021) 71:34–45. doi: 10.1016/j.coi.2021.04.002, PMID: 34000455 Cardoso Pinto AM Shariq S Ranasinghe L Sundar Budhathoki S Skirrow H Whittaker E . Reasons for reductions in routine childhood immunisation uptake during the COVID-19 pandemic in low- and middle-income countries: a systematic review. PLoS Glob Public Health. (2023) 3:e0001415. doi: 10.1371/journal.pgph.0001415, PMID: 36963003 Galadima AN Zulkefli NAM Said SM Ahmad N. Factors influencing childhood immunisation uptake in Africa: a systematic review. BMC Public Health. (2021) 21:1475. doi: 10.1186/s12889-021-11466-5, PMID: 34320942 Guzman-Holst A DeAntonio R Prado-Cohrs D Juliao P. Barriers to vaccination in Latin America: a systematic literature review. Vaccine. (2020) 38:470–81. doi: 10.1016/j.vaccine.2019.10.088, PMID: 31767469 Périères L Séror V Boyer S Sokhna C Peretti-Watel P. Reasons given for non-vaccination and under-vaccination of children and adolescents in sub-Saharan Africa: a systematic review. Hum Vaccin Immunother. (2022) 18:2076524. doi: 10.1080/21645515.2022.2076524, PMID: 35709342 Phillips DE Dieleman JL Lim SS Shearer J. Determinants of effective vaccine coverage in low and middle-income countries: a systematic review and interpretive synthesis. BMC Health Serv Res. (2017) 17:681. doi: 10.1186/s12913-017-2626-0, PMID: 28950899 Besnier E Thomson K Stonkute D Mohammad T Akhter N Todd A . Which public health interventions are effective in reducing morbidity, mortality and health inequalities from infectious diseases amongst children in low- and middle-income countries (LMICs): an umbrella review. PLoS One. (2021) 16:e0251905. doi: 10.1371/journal.pone.0251905, PMID: 34111134 Loevinsohn BP Loevinsohn ME. Improvement in coverage of primary health care in a developing country through use of food incentives. Lancet. (1986) 1:1314–6. PMID: 2872440 Usman HR Rahbar MH Kristensen S Vermund SH Kirby RS Habib F . Randomized controlled trial to improve childhood immunization adherence in rural Pakistan: redesigned immunization card and maternal education. Trop Med Int Health. (2011) 16:334–42. doi: 10.1111/j.1365-3156.2010.02698.x, PMID: 21159080 Anjum Q Omair A Inam SN Ahmed Y Usman Y Shaikh S. Improving vaccination status of children under five through health education. J Pak Med Assoc. (2004) 54:610–3. PMID: 16104487 Hu Y Chen Y Wang Y Song Q Li Q. Prenatal vaccination education intervention improves both the mothers' knowledge and children's vaccination coverage: evidence from randomized controlled trial from eastern China. Hum Vaccin Immunother. (2017) 13:1–8. doi: 10.1080/21645515.2017.1285476, PMID: 28319453 D'Souza PV Umarani J. Teaching package improves mothers knowledge on vaccine preventable diseases and vaccination: a quasi experimental study. Int J Res Med Sci. (2014) 2:976–82. doi: 10.5455/2320-6012.ijrms20140850 Powell-Jackson T Fabbri C Dutt V Tougher S Singh K. Effect and cost-effectiveness of educating mothers about childhood DPT vaccination on immunisation uptake, knowledge, and perceptions in Uttar Pradesh, India: a randomised controlled trial. PLoS Med. (2018) 15:e1002519. doi: 10.1371/journal.pmed.1002519, PMID: 29509769 Saggurti N Atmavilas Y Porwal A Schooley J Das R Kande N . Effect of health intervention integration within women's self-help groups on collectivization and healthy practices around reproductive, maternal, neonatal and child health in rural India. PLoS One. (2018) 13:e0202562. doi: 10.1371/journal.pone.0202562, PMID: 30138397 Vaidyanathan R. Immunization coverage among under-five children living along a school student through child-to-child and child-to-parent information, education and communication strategy. Indian J Public Health. (2019) 63:334–40. doi: 10.4103/ijph.IJPH_424_18, PMID: 32189654 Pandey P Sehgal AR Riboud M Levine D Goyal M. Informing resource-poor populations and the delivery of entitled health and social services in rural India: a cluster randomized controlled trial. JAMA. (2007) 298:1867–75. doi: 10.1001/jama.298.16.1867, PMID: 17954538 Nasir NM Amran Y Nakamura Y. Changing knowledge and practices of mothers on newborn care through mother class: an intervention study in Indonesia. J Trop Pediatr. (2017) 63:440–6. doi: 10.1093/tropej/fmx010 Alatas V Chandrasekhar AG Mobius M Olken BA Paladines C (2019). When celebrities speak: A nationwide twitter experiment promoting vaccination in Indonesia. National Bureau of Economic Research, Cambridge. Contract No.: NBER Working Paper No. 25589. Rasella D Aquino R Santos CAT Paes-Sousa R Barreto ML. Effect of a conditional cash transfer programme on childhood mortality: a nationwide analysis of Brazilian municipalities. Lancet. (2013) 382:57–64. doi: 10.1016/S0140-6736(13)60715-1 Shei A Costa F Reis MG Ko AI. The impact of Brazil's Bolsa Família conditional cash transfer program on children's health care utilization and health outcomes. BMC Int Health Hum Rights. (2014) 14:10. doi: 10.1186/1472-698X-14-10, PMID: 24690131 Cahyadi N Hanna R Olken BA Adi Prima R Satriawan E Syamsulhakim E . (2018). Cumulative impacts of conditional cash transfer programs: Experimental evidence from Indonesia National Bureau of economic research. Contract No.: NBER Working Paper Series 24670. Kusuma D Thabrany H Hidayat B McConnell M Berman P Cohen J. New evidence on the impact of large-scale conditional cash transfers on child vaccination rates: the case of a clustered-randomized trial in Indonesia. World Dev. (2017) 98:497–505. doi: 10.1016/j.worlddev.2017.05.007 Olken BA Onishi J Wong S. Should aid reward performance? Evidence from a field experiment on health and education in Indonesia. Am Econ J Appl Econ. (2014) 6:1–34. doi: 10.1257/app.6.4.1, PMID: 25485039 Attanasio O.G.M.L. Heredia P Vera-Herna’ndez M (2005). The short-term impact of a conditional cash subsidy on child health and nutrition in Colombia. Centre for the Evaluation of Development Policies, The Institute for Fiscal Studies. Barham T Brenzel L Maluccio JA (2007). Beyond 80%: Are there new ways of increasing vaccination coverage? Evaluation of CCT programs in Mexico and Nicaragua. Washington, D.C.: The International Bank for Reconstruction and Development/The World Bank. Contract No.: 41537. Wadhwa S (2019). “Conditional cash transfers and parental investment in daughters: Evidence from India” in Population Association of America 2019 Annual Meeting, Austin, TX. Banerjee A Chandrasekhar A Duflo E Dalpath S Floretta J Jackson M . (2020). “Evaluating the impact of interventions to improve full immunisation rates in Haryana, India” in New Delhi: International Initiative for Impact Evaluation (3ie). Contract No.: 3ie Impact Evaluation Report 126. Chandir S Khan AJ Hussain H Usman HR Khowaja S Halsey NA . Effect of food coupon incentives on timely completion of DTP immunization series in children from a low-income area in Karachi, Pakistan: a longitudinal intervention study. Vaccine. (2010) 28:3473–8. doi: 10.1016/j.vaccine.2010.02.061, PMID: 20199756 Mathanga DP Luman ET Campbell CH Silwimba C Malenga G. Integration of insecticide-treated net distribution into routine immunization services in Malawi: a pilot study. Trop Med Int Health. (2009) 14:792–801. doi: 10.1111/j.1365-3156.2009.02295.x, PMID: 19497078 Okoli U Morris L Oshin A Pate MA Aigbe C Muhammad A. Conditional cash transfer schemes in Nigeria: potential gains for maternal and child health service uptake in a national pilot programme. BMC Pregnancy Childbirth. (2014) 14:408. doi: 10.1186/s12884-014-0408-9, PMID: 25495258 Robertson L Mushati P Eaton JW Dumba L Mavise G Makoni J . Effects of unconditional and conditional cash transfers on child health and development in Zimbabwe: a cluster-randomised trial. Lancet. (2013) 381:1283–92. doi: 10.1016/S0140-6736(12)62168-0, PMID: 23453283 Andrade MV Chein F Souza LR Puig-Junoy J. Income transfer policies and the impacts on the immunization of children: the Bolsa Família program. Cad Saud Publ. (2012) 28:1347–58. doi: 10.1590/S0102-311X2012000700013, PMID: 22729265 Beck S Pulkki-Brännström A-M San Sebastián M. Basic income—healthy outcome? Effects on health of an Indian basic income pilot project: a cluster randomised trial. J Dev Eff. (2015) 7:111–26. doi: 10.1080/19439342.2014.974200 Murthy N Chandrasekharan S Prakash MP Kaonga NN Peter J Ganju A . The impact of an mHealth voice message service (mMitra) on infant care knowledge, and practices among low-income women in India: findings from a Pseudo-randomized controlled trial. Matern Child Health J. (2019) 23:1658–69. doi: 10.1007/s10995-019-02805-5, PMID: 31584144 Brown VB Oluwatosin OA Akinyemi JO Adeyemo AA. Effects of community health nurse-led intervention on childhood routine immunization completion in primary health care centers in Ibadan, Nigeria. J Community Health. (2016) 41:265–73. doi: 10.1007/s10900-015-0092-3, PMID: 26395786 Ekhaguere OA Oluwafemi RO Badejoko B Oyeneyin LO Butali A Lowenthal ED . Automated phone call and text reminders for childhood immunisations (PRIMM): a randomised controlled trial in Nigeria. BMJ Glob Health. (2019) 4:e001232. doi: 10.1136/bmjgh-2018-001232, PMID: 31139442 Dipeolu IO (2017). Effect of mobile-phone reminder test messages on mothers' knowledge and completion of routine immunisation in rural areas of Oyo State, Nigeria University of Ibadan. Eze GU Adeleye OO. Enhancing routine immunization performance using innovative Technology in an Urban Area of Nigeria. West Afr J Med. (2015) 34:3–10. PMID: 26902809 Domek GJ Contreras-Roldan IL Bull S O'Leary ST Bolaños Ventura GA Bronsert M . Text message reminders to improve infant immunization in Guatemala: a randomized clinical trial. Vaccine. (2019) 37:6192–200. doi: 10.1016/j.vaccine.2019.08.046, PMID: 31492475 Kazi AM Ali M Zubair K Kalimuddin H Kazi AN Iqbal SP . Effect of Mobile phone text message reminders on routine immunization uptake in Pakistan: randomized controlled trial. JMIR Public Health Surveill. (2018) 4:e20. doi: 10.2196/publichealth.7026, PMID: 29514773 Bangure D Chirundu D Gombe N Marufu T Mandozana G Tshimanga M . Effectiveness of short message services reminder on childhood immunization programme in Kadoma, Zimbabwe—a randomized controlled trial, 2013. BMC Public Health. (2015) 15:137. doi: 10.1186/s12889-015-1470-6, PMID: 25885862 Wallace AS Peetosutan K Untung A Ricardo M Yosephine P Wannemuehler K . Home-based records and vaccination appointment stickers as parental reminders to reduce vaccination dropout in Indonesia: a cluster-randomized controlled trial. Vaccine. (2019) 37:6814–23. doi: 10.1016/j.vaccine.2019.09.040, PMID: 31564451 Owais A Hanif B Siddiqui AR Agha A Zaidi AKM. Does improving maternal knowledge of vaccines impact infant immunization rates? A community-based randomized-controlled trial in Karachi, Pakistan. BMC Public Health. (2011) 11:239. doi: 10.1186/1471-2458-11-239, PMID: 21496343 Berhane Y Pickering J. Are reminder stickers effective in reducing immunization dropout rates in Addis Ababa, Ethiopia? J Trop Med Hyg. (1993) 96:139–45. PMID: 8505766 Bhuiyan SU Nakamura Y Qureshi NA. Study on the development and assessment of maternal and child health (MCH) handbook in Bangladesh. J Public Health Dev. (2006) 4:45–60. Kawakatsu Y Sugishita T Oruenjo K Wakhule S Kibosia K Were E . Effectiveness of and factors related to possession of a mother and child health handbook: an analysis using propensity score matching. Health Educ Res. (2015) 30:935–46. doi: 10.1093/her/cyv048, PMID: 26491073 Seth R Akinboyo I Chhabra A Qaiyum Y Shet A Gupte N . Mobile phone incentives for childhood immunizations in rural India. Pediatrics. (2018) 141: e20173455. doi: 10.1542/peds.2017-3455, PMID: 29540571 Brown VB Oluwatosin OA, &, Ogundeji MA . Impact of training intervention on immunization providers’ knowledge and practice of routine immunization in Ibadan, South-Western Nigeria: a primary health care experience. PAMJ (2017) 26:216. doi: 10.11604/pamj.2017.26.216.11545 Robinson JS Burkhalter BR Rasmussen B Sugiono R. Low-cost on-the-job peer training of nurses improved immunization coverage in Indonesia. Bull World Health Organ. (2001) 79:150–8. PMID: 11242822 Helleringer S Asuming PO Abdelwahab J. The effect of mass vaccination campaigns against polio on the utilization of routine immunization services: a regression discontinuity design. Vaccine. (2016) 34:3817–22. doi: 10.1016/j.vaccine.2016.05.037, PMID: 27269060 Andreoni J Callen M. Khan M. Y. Jaffar K. Sprenger C. (2016). Using preference estimates to customize incentives: An application to polio vaccination drives in Pakistan. National Bureau of Economic Research. Contract No.: NBER Working Paper Series 22019. Lee SS. Three Field Experiments on Incentives for Health Workers. Cambridge, Massachusetts: Harvard University, Graduate School of Arts & Sciences (2015). Bonfrer I Van de Poel E Van Doorslaer E. The effects of performance incentives on the utilization and quality of maternal and child care in Burundi. Soc Sci Med. (2014) 123:96–104. doi: 10.1016/j.socscimed.2014.11.004, PMID: 25462610 Chansa C Das A Qamruddin J Friedman J Mkandawire A Vledder M (2015). Linking results to performance: Evidence from a results based financing pre-pilot project in Katete District, Zambia. Washington, D.C.: The World Bank, Health Nutrition and Population. Contract No.: 98265. de Walque D Robyn PJ Saidou H Ga Sorgho Steenland M (2017). Looking into the performance-based financing black box evidence from an impact evaluation in the health sector in Cameroon. Washington, D.C.: Development Research Group, Human Development and Public Services Team, Health Nutrition and Population Global Practice Group, The World Bank. Contract No.: 8162. Eichler R Auxila P U Antoine Desmangles B. Performance-based incentives for health: six years of results from supply-side programs in Haiti. Centre for Global Development. Contract No.: CGD Working Paper #121. Mayumana I Borghi J Anselmi L Mamdani M Lange S. Effects of payment for performance on accountability mechanisms: evidence from Pwani, Tanzania. Soc Sci Med. (2017) 179:61–73. doi: 10.1016/j.socscimed.2017.02.022, PMID: 28257886 Zang O Djienouassi S Sorgho G Taptueii JC. Impact of performance-based financing on health-care quality and utilization in urban areas of Cameroon. African Health Monitor. (2015) 20:10–14. Zeng W Shepard DS Rusatira JD Blaakman AP Nsitou BM. Evaluation of results-based financing in the Republic of the Congo: a comparison group pre-post study. Health Policy Plan. (2018) 33:392–400. doi: 10.1093/heapol/czx195, PMID: 29351604 Engineer CY Dale E Agarwal A Agarwal A Alonge O Edward A . Effectiveness of a pay-for-performance intervention to improve maternal and child health services in Afghanistan: a cluster-randomized trial. Int J Epidemiol. (2016) 45:451–9. doi: 10.1093/ije/dyv362, PMID: 26874927 Basinga P Gertler PJ Binagwaho A Soucat AL Sturdy J Vermeersch CM. Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet. (2011) 377:1421–8. doi: 10.1016/S0140-6736(11)60177-3, PMID: 21515164 Sherry TB Bauhoff S, &, Mohanan M . Multitasking and heterogeneous treatment effects in pay-for-performance in health care: evidence from Rwanda. Am J Health Econ (2017) 3: 192–226. doi: 10.1162/AJHE_a_00072 Rusa L Ngirabega Jde D Janssen W Van Bastelaere S Porignon D Vandenbulcke W. Performance-based financing for better quality of services in Rwandan health centres: 3-year experience. Trop Med Int Health. (2009) 14:830–7. doi: 10.1111/j.1365-3156.2009.02292.x, PMID: 19497081 Huillery E Seban J (2017). Money for nothing? The effect of financial incentives on efforts and performances in the health sector. J-PAL: Abdul Latif Jameel Poverty Action Lab. Soeters R Peerenboom PB Mushagalusa P Kimanuka C. Performance-based financing experiment improved health care in the Democratic Republic of Congo. Health Aff. (2011) 30:1518–27. doi: 10.1377/hlthaff.2009.0019, PMID: 21821568 Binyaruka P Patouillard E Powell-Jackson T Greco G Maestad O Borghi J. Effect of paying for performance on utilisation, quality, and user costs of health Services in Tanzania: a controlled before and after study. PLoS One. (2015) 10:e0135013. doi: 10.1371/journal.pone.0135013, PMID: 26317510 Aquino R de Oliveira NF Barreto ML. Impact of the family health program on infant mortality in Brazilian municipalities. Am J Public Health. (2009) 99:87–93. doi: 10.2105/AJPH.2007.127480, PMID: 19008516 Dicko A Toure SO Traore M Sagara I Toure OB Sissoko MS . Increase in EPI vaccines coverage after implementation of intermittent preventive treatment of malaria in infant with Sulfadoxine -pyrimethamine in the district of Kolokani, Mali: results from a cluster randomized control trial. BMC Int Health Hum Rights. (2011) 11:573. doi: 10.1186/1471-2458-11-573, PMID: 21767403 Pathak Y, &, Macours K. Women’s political reservation, early childhood development, and learning in India. Econ Dev Cult Chang (2017) 65: 741–766. doi: 10.1086/692114 Nagar R Venkat P Stone LD Engel KA Sadda P Shahnawaz M. A cluster randomized trial to determine the effectiveness of a novel, digital pendant and voice reminder platform on increasing infant immunization adherence in rural Udaipur, India. Vaccine. (2018) 36:6567–77. doi: 10.1016/j.vaccine.2017.11.023, PMID: 29162321 Prinja S Nimesh R Gupta A Bahuguna P Gupta M Thakur JS. Impact of m-health application used by community health volunteers on improving utilisation of maternal, new-born and child health care services in a rural area of Uttar Pradesh, India. Trop Med Int Health. (2017) 22:895–907. doi: 10.1111/tmi.12895, PMID: 28510997 Costa-Font J Parmar D. (2017). Political agency and public healthare: Evidence from India UNU-WIDER. Contract No.: CESifo Working Papers 6640. More NS Bapat U Das S Alcock G Patil S Porel M . Community mobilization in Mumbai slums to improve perinatal care and outcomes: a cluster randomized controlled trial. PLoS Med. (2012) 9:e1001257. doi: 10.1371/journal.pmed.1001257, PMID: 22802737 Aggarwal S. The long road to health: Healthcare utilization impacts of a road pavement policy in rural India. J Dev Econ. (2018) 151:102667. doi: 10.1016/j.jdeveco.2021.102667 Banerjee R, &, Sachdeva A (2015). Pathways to preventive health, evidence from India's rural road program. USC Dornsife Institute for New Economic Thinking. Contract No.: Working Paper No. 15–19. Weldemariam M. (2010). The impact of fiscal decentralization on education and health outcomes in Ethiopia: A regional panel data analysis Addis Ababa: Addis Ababa university. Chen YJ Chindarkar N Xiao Y. Effect of reliable electricity on health facilities, health information, and child and maternal health services utilization: evidence from rural Gujarat, India. J Health Popul Nutr. (2019) 38:7. doi: 10.1186/s41043-019-0164-6, PMID: 30782203 Johri M Chandra D Kone KG Sylvestre MP Mathur AK Harper S . Social and behavior change communication interventions delivered face-to-face and by a Mobile phone to strengthen vaccination uptake and improve child health in rural India: randomized pilot study. JMIR Mhealth Uhealth. (2020) 8:e20356. doi: 10.2196/20356, PMID: 32955455 Hagiwara A Ueyama M Ramlawi A Sawada Y. Is the maternal and child health (MCH) handbook effective in improving health-related behavior? Evidence from Palestine. J Public Health Policy. (2013) 34:31–45. doi: 10.1057/jphp.2012.56, PMID: 23151920 Hutchinson P Lance P Guilkey DK Shahjahan M Haque S. Measuring the cost-effectiveness of a national health communication program in rural Bangladesh. J Health Commun. (2006) 11:91–121. doi: 10.1080/10810730600974647, PMID: 17148101 Morris SS Flores R Olinto P Medina JM. Monetary incentives in primary health care and effects on use and coverage of preventive health care interventions in rural Honduras: cluster randomised trial. Lancet. (2004) 364:2030–7. doi: 10.1016/S0140-6736(04)17515-6, PMID: 15582060 Goel S Dogra V Gupta SK Lakshmi PV Varkey S Pradhan N . Effectiveness of Muskaan Ek Abhiyan (the smile campaign) for strengthening routine immunization in Bihar. Ind Pediatr. (2012) 49:103–8. doi: 10.1007/s13312-012-0023-7, PMID: 21555801 Brugha RF Kevany JP. Maximizing immunization coverage through home visits: a controlled trial in an urban area of Ghana. Bull World Health Organ. (1996) 74:517–24. PMID: 9002332 Gurley N Shearer J Srivastava Y Mahapatra S Desmond M (2020). Impacts of community-led video education to increase vaccination coverage in Uttar Pradesh, India. International initiative for impact evaluation (3ie). Contract No: Impact Evaluation Report 125. Banwat ME Lar LA Abok IA Yiltok ES. Effect of peer education on knowledge, attitude and completeness of childhood routine immunization in a rural community of plateau state. Res J Health Sci. (2015) 3:264–74. Oyo-Ita A Bosch-Capblanch X. Ross A. Hanlon P. Oku A. Esu E. . (2020). Impacts of engaging communities through traditional and religious leaders on vaccination coverage in Cross River state, Nigeria. International Initiative for Impact Evaluation (3ie). Contract No.: Impact Evaluation Report 127. Oche MO Umar AS Ibrahim MTO Sabitu K. An assessment of the impact of health education on maternal knowledge and practice of childhood immunization in Kware, Sokoto state. J Publ Health Epidemiol. (2011) 3:440–7. Pramanik S Ghosh A. Goswami A. Das T. Nanda R. Forth P. . (2020). Impacts of the stimulate, appreciate, learn, and transfer community engagement approach to increase immunisation coverage in Assam, India international initiative for impact evaluation (3ie). Contract No.: 3ie Impact Evaluation Report 13. Bolam A Manandhar DS Shrestha P Ellis M Costello AM. The effects of postnatal health education for mothers on infant care and family planning practices in Nepal: a randomised controlled trial. BMJ. (1998) 316:805–11. doi: 10.1136/bmj.316.7134.805, PMID: 9549449 Okeke E Glick P. Abubakar I. S. Chari A. V. Pitchforth E. Exley J. . (2017). Better obstetrics in rural Nigeria: Evaluating the midwives service scheme. International Initiative for Impact Evaluation (3ie). Contract No.: 3ie Impact Evaluation Report 56. Rahman M Yunus FM Shah R Jhohura FT Mistry SK Quayyum T . A controlled before-and-after perspective on the improving maternal, neonatal, and child survival program in rural Bangladesh: an impact analysis. PLoS One. (2016) 11:e0161647. doi: 10.1371/journal.pone.0161647, PMID: 27583478 Findley SE Uwemedimo OT Doctor HV Green C Adamu F Afenyadu GY. Early results of an integrated maternal, newborn, and child health program, northern Nigeria, 2009 to 2011. BMC Public Health. (2013) 13:1034. doi: 10.1186/1471-2458-13-1034, PMID: 24175944 Sankar D. (2013). Improving early childhood development through community mobilization and integrated planning for children. Results from the evaluation of Bachpan program, Ratlam District, Madhya Pradesh, India. The World Bank. Contract No.: Discussion Paper Series 59. Memon ZA Khan GN Soofi SB Baig IY Bhutta ZA. Impact of a community-based perinatal and newborn preventive care package on perinatal and neonatal mortality in a remote mountainous district in northern Pakistan. BMC Pregnancy Childbirth. (2015) 15:106. doi: 10.1186/s12884-015-0538-8, PMID: 25925407 Andersson N Cockcroft A Ansari NM Omer K Baloch M Ho Foster A . Evidence-based discussion increases childhood vaccination uptake: a randomised cluster controlled trial of knowledge translation in Pakistan. BMC Int Health Hum Rights. (2009) 9:S8. doi: 10.1186/1472-698X-9-S1-S8, PMID: 19828066 Younes L Houweling TAJ Azad K Kuddus A Shaha S Haq B . The effect of participatory women's groups on infant feeding and child health knowledge, behaviour and outcomes in rural Bangladesh: a controlled before-and-after study. J Epidemiol Community Health. (2015) 69:374–81. doi: 10.1136/jech-2014-204271, PMID: 25472635 Janssens W. Externalities in program evaluation: the impact of a women's empowerement program on immunization. J Eur Econ Assoc. (2011) 9:1082–113. doi: 10.1111/j.1542-4774.2011.01041.x Carvalho N Thacker N Gupta SS Salomon JA. More evidence on the impact of India's conditional cash transfer program, Janani Suraksha Yojana: quasi-experimental evaluation of the effects on childhood immunization and other reproductive and child health outcomes. PLoS One. (2014) 9:e109311. doi: 10.1371/journal.pone.0109311, PMID: 25303078 Rahman MM Pallikadavath S. Maternal and child health care services' utilization data from the fourth round of district level household survey in India. Data Brief. (2019) 23:103738. doi: 10.1016/j.dib.2019.103738, PMID: 31372404 Demilew A Girma M McElwee E Datta S Barofsky J Disasa T (2021). Impacts of supportive feedback and nonmonetary incentives on child immunisation in Ethiopia. International Initiative for Impact Evaluation (3ie). Contract No.: 3ie Impact Evaluation Report 134. Varghese B Roy R Saha S Roalkvam S. Fostering maternal and newborn Care in India the Yashoda way: does this improve maternal and newborn care practices during institutional delivery? PLoS One. (2014) 9:e84145. doi: 10.1371/journal.pone.0084145, PMID: 24454718 Sato R Belel A. The effect of performance-based financing on child vaccinations in northern Nigeria. Vaccine. (2020) 38:2209–15. doi: 10.1016/j.vaccine.2020.01.033, PMID: 31983583 Modi D Dholakia N Gopalan R Venkatraman S Dave K Shah S . mHealth intervention "ImTeCHO" to improve delivery of maternal, neonatal, and child care services-a cluster-randomized trial in tribal areas of Gujarat, India. PLoS Med. (2019) 16:e1002939. doi: 10.1371/journal.pmed.1002939, PMID: 31647821 Uddin MJ Shamsuzzaman M Horng L Labrique A Vasudevan L Zeller K . Use of mobile phones for improving vaccination coverage among children living in rural hard-to-reach areas and urban streets of Bangladesh. Vaccine. (2016) 34:276–83. doi: 10.1016/j.vaccine.2015.11.024, PMID: 26647290 Hategeka C Ruton H Law MR. Effect of a community health worker mHealth monitoring system on uptake of maternal and newborn health services in Rwanda. Glob Health Res Policy. (2019) 4:8. doi: 10.1186/s41256-019-0098-y Nagar R Ambiya M. S. Singh P. Abdullah H. Banshiwal V. Stone L. . (2020). Impacts of a novel mHealth platform to track maternal and child health in Udaipur, India. International Initiative for Impact Evaluation (3ie). Contract No.: 3ie Impact Evaluation Report 129. Chen L Du X Zhang L van Velthoven MH Wu Q Yang R . Effectiveness of a smartphone app on improving immunization of children in rural Sichuan Province, China: a cluster randomized controlled trial. BMC Public Health. (2016) 16:909. doi: 10.1186/s12889-016-3549-0, PMID: 27581655 Dissieka R Soohoo M Janmohamed A Doledec D. Providing mothers with mobile phone message reminders increases childhood immunization and vitamin a supplementation coverage in Côte d'Ivoire: a randomized controlled trial. J Public Health Africa. (2019) 10:1032. doi: 10.4081/jphia.2019.1032, PMID: 31285815 Mazumder S Taneja S Bahl R Mohan P Strand TA Sommerfelt H . Effect of implementation of integrated Management of Neonatal and Childhood Illness programme on treatment seeking practices for morbidities in infants: cluster randomised trial. BMJ. (2014) 349:g4988. doi: 10.1136/bmj.g4988, PMID: 25172514 Arifeen SE Hoque DM Akter T Rahman M Hoque ME Begum K . Effect of the integrated Management of Childhood Illness strategy on childhood mortality and nutrition in a rural area in Bangladesh: a cluster randomised trial. Lancet. (2009) 374:393–403. doi: 10.1016/S0140-6736(09)60828-X Carnell MA Dougherty L Pomeroy AM Karim AM Mekonnen YM Mulligan BE. Effectiveness of scaling up the 'three pillars' approach to accelerating MDG 4 progress in Ethiopia. J Health Popul Nutr. (2014) 32:549–63. PMID: 25895187 Mohan P Kishore B Singh S Bahl R Puri A Kumar R. Assessment of implementation of integrated management of neonatal and childhood illness in India. J Health Popul Nutr. (2011) 29:629–38. doi: 10.3329/jhpn.v29i6.9900 PMID: 22283037 Admassie A Abebaw D Woldemichael AD. Impact evaluation of the Ethiopian health services extension Programme. J Dev Eff. (2009) 1:430–49. doi: 10.1080/19439340903375724 Nzioki JM Ouma J Ombaka JH Onyango RO. Community health worker interventions are key to optimal infant immunization coverage, evidence from a pretest-posttest experiment in Mwingi, Kenya. Pan Afr Med J. (2017) 28:21. doi: 10.11604/pamj.2017.28.21.11255 Webster J Landegger J Bruce J Malunda D Chantler T Kumakech E . (2019). Impacts of IRC’s fifth child community engagement strategy to increase immunisation in northern Uganda. International Institute for Impact Evaluation. Habib MA Soofi S Cousens S Anwar S Haque N Ahmed I . Community engagement and integrated health and polio immunisation campaigns in conflict-affected areas of Pakistan: a cluster randomised controlled trial. Lancet Glob Health. (2017) 5:e593–603. doi: 10.1016/S2214-109X(17)30184-5 Unger JP. Can intensive campaigns dynamize front line health services? The evaluation of an immunization campaign in Thiès health district, Senegal. Soc Sci Med. (1991) 32:249–59. doi: 10.1016/0277-9536(91)90101-H, PMID: 2024134 Wang PC Mwango A Moberley S Brockman BJ Connor AL Kalesha-Masumbu P . A cluster randomised trial on the impact of integrating early infant HIV diagnosis with the expanded Programme on immunization on immunization and HIV testing rates in rural health facilities in southern Zambia. PLoS One. (2015) 10:e0141455. doi: 10.1371/journal.pone.0141455, PMID: 26513240 Drain PK Ralaivao JS Rakotonandrasana A Carnell MA. Introducing auto-disable syringes to the national immunization programme in Madagascar. Bull World Health Organ. (2003) 81:553–60. PMID: 14576886 Wagstaff A Yu S. Do health sector reforms have their intended impacts? The World Bank's health VIII project in Gansu province, China. J Health Econ. (2007) 26:505–35. doi: 10.1016/j.jhealeco.2006.10.006 Adamu AA Uthman OA Gadanya MA Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. (2019) 18:969–91. doi: 10.1080/14760584.2019.1647782, PMID: 31340663 Djibuti M Gotsadze G Zoidze A Mataradze G Esmail LC Kohler JC. The role of supportive supervision on immunization program outcome—a randomized field trial from Georgia. BMC Int Health Hum Rights. (2009) 9:S11. doi: 10.1186/1472-698X-9-S1-S11, PMID: 19828055 Biemba G Yeboah-Antwi K Vosburg KB Prust ML Keller B Worku Y . Effect of deploying community health assistants on appropriate treatment for diarrhoea, malaria and pneumonia: quasi-experimental study in two districts of Zambia. Trop Med Int Health. (2016) 21:985–94. doi: 10.1111/tmi.12730 Assegaai T Reagon G Schneider H. Evaluating the effect of ward-based outreach teams on primary healthcare performance in north West Province, South Africa: a plausibility design using routine data. S Afr Med J. (2018) 108:329–35. doi: 10.7196/SAMJ.2017.v108i4.12755, PMID: 29629685 Bradley J Igras S. Improving the quality of child health services: participatory action by providers. Int J Qual Health Care. (2005) 17:391–9. doi: 10.1093/intqhc/mzi057, PMID: 15951311 Olayo R Wafula C Aseyo E Loum C Kaseje D. A quasi-experimental assessment of the effectiveness of the community health strategy on health outcomes in Kenya. BMC Health Serv Res. (2014) 14:S3. doi: 10.1186/1472-6963-14-S1-S3, PMID: 25079378 Bhuiya A Hanifi SMA Hoque S. Unlocking community capability through promotion of self-help for health: experience from Chakaria, Bangladesh. BMC Health Serv Res. (2016) 16:624. doi: 10.1186/s12913-016-1865-9, PMID: 28185584 Banerjee AV Duflo E Glennerster R Kothari D. Improving immunisation coverage in rural India: clustered randomised controlled evaluation of immunisation campaigns with and without incentives. BMJ. (2010) 340:c2220. doi: 10.1136/bmj.c2220, PMID: 20478960 Cristia J Evans WN Kim B. Improving the health coverage of the rural poor: does contracting-out Mobile medical teams work? J Dev Stud. (2015) 51:1–15. doi: 10.1080/00220388.2014.976617 Cristia J Prado AG Peluffo C. The impact of contracting in and contracting out basic health services: the Guatemalan experience. World Dev. (2015) 70:215–27. doi: 10.1016/j.worlddev.2015.02.003 Cristia JP Evans W. N. Kim B. (2011). Does contracting-out primary care services work? The case of rural Guatemala. Contract No.: IDB Working Paper Series IDB-WP-273. Sengupta P Benjamin AI Myles PR Babu BV. Evaluation of a community-based intervention to improve routine childhood vaccination uptake among migrants in urban slums of Ludhiana, India. J Public Health. (2017) 39:805–12. doi: 10.1093/pubmed/fdw131, PMID: 27915261 Busso M Cristia J Humpage S. Did you get your shots? Experimental evidence on the role of reminders. J Health Econ. (2015) 44:226–37. doi: 10.1016/j.jhealeco.2015.08.005 Ryman TK Trakroo A Wallace A Gupta SK Wilkins K Mehta P . Implementation and evaluation of the reaching every district (RED) strategy in Assam, India, 2005–2008. Vaccine. (2011) 29:2555–60. doi: 10.1016/j.vaccine.2011.01.061, PMID: 21300101 Mohanan M Rajan V. S. Swanson K. Thirumurthy H. (2020). Information and facilitation interventions for accountability in health and nutrition: Evidence from a randomized trial in India Duke University. Contract No.: ERID Working Paper 295. More NS Das S Bapat U Alcock G Manjrekar S Kamble V . Community resource centres to improve the health of women and children in informal settlements in Mumbai: a cluster-randomised, controlled trial. Lancet Glob Health. (2017) 5:e335–49. doi: 10.1016/S2214-109X(16)30363-1, PMID: 28193399 Atnafu A Otto K Herbst CH. The role of mHealth intervention on maternal and child health service delivery: findings from a randomized controlled field trial in rural Ethiopia. mHealth. (2017) 3:39. doi: 10.21037/mhealth.2017.08.04, PMID: 29184891 Goodson JL Kulkarni MA Vanden Eng JL Wannemuehler KA Cotte AH Desrochers RE . Improved equity in measles vaccination from integrating insecticide-treated bednets in a vaccination campaign, Madagascar. Trop Med Int Health. (2012) 17:430–7. doi: 10.1111/j.1365-3156.2011.02953.x, PMID: 22273490 Balasubramaniam S Kumar S Sethi R Charurat E Lalchandani K Schuster A . Quasi-experimental study of systematic screening for family planning services among postpartum women Attending Village health and nutrition days in Jharkhand, India. Int J Integr Care. (2018) 18:7. doi: 10.5334/ijic.3078, PMID: 29588642 Rao T. (2014). The impact of a community health worker program on childhood immunization: Evidence from India's ‘ASHA’ workers. Cornell University. Manyazewal T Mekonnen A Demelew T Mengestu S Abdu Y Mammo D . Improving immunization capacity in Ethiopia through continuous quality improvement interventions: a prospective quasi-experimental study. Infect Dis Poverty. (2018) 7:119. doi: 10.1186/s40249-018-0502-8, PMID: 30497515 Rahman A Malik A Sikander S Roberts C Creed F. Cognitive behaviour therapy-based intervention by community health workers for mothers with depression and their infants in rural Pakistan: a cluster-randomised controlled trial. Lancet. (2008) 372:902–9. doi: 10.1016/S0140-6736(08)61400-2, PMID: 18790313 Tandon BN Sahai A. Immunization in India: contribution of integrated child development services scheme to expanded programme of immunization. J Trop Pediatr. (1988) 34:309–12. doi: 10.1093/tropej/34.6.309 Alhassan RK Nketiah-Amponsah E Ayanore MA Afaya A Salia SM Milipaak J . Impact of a bottom-up community engagement intervention on maternal and child health services utilization in Ghana: a cluster randomised trial. BMC Public Health. (2019) 19:791. doi: 10.1186/s12889-019-7180-8, PMID: 31226977 Shukla M. Impact of a health governance intervention on provincial health system performance in Afghanistan: a quasi-experimental study. Health Syst Reform. (2018) 4:249–66. doi: 10.1080/23288604.2018.1477536, PMID: 30059638 Björkman M Svensson J. Power to the people: evidence from a randomized field experiment on community-based monitoring in Uganda. Q J Econ. (2009) 124:735–69. doi: 10.1162/qjec.2009.124.2.735 Roy SK Bilkes F Islam K Ara G Tanner P Wosk I . Impact of pilot project of rural maintenance Programme (RMP) on destitute women: CARE, Bangladesh. Food Nutr Bull. (2008) 29:67–75. doi: 10.1177/156482650802900108, PMID: 18510207 Carrillo B Iglesias WJ Trujillo JC. Attainments and limitations of an early childhood programme in Colombia. Health Policy Plan. (2014) 30:906–16. doi: 10.1093/heapol/czu091, PMID: 25237133 Padilla Ade J Trujillo JC. An impact assessment of the child growth, development and care program in the Caribbean region of Colombia. Cad Saud Publ. (2015) 31:2099–109. doi: 10.1590/0102-311X00153514 Thomas RA (2011). Essays on ex ante evaluations of cash transfer programs. University of York, York. Talukder M Rob U., Musa S. A. Bajracharya A. Keya K. Noor F. . (2014). Evaluation of the impact of the voucher program for improving maternal health behavior and status in Bangladesh. Population Council, Dhaka. Siddiqi DA Ali RF Munir M Shah MT Khan AJ Chandir S. Effect of vaccine reminder and tracker bracelets on routine childhood immunization coverage and timeliness in urban Pakistan (2017-18): a randomized controlled trial. BMC Public Health. (2020) 20:1086. doi: 10.1186/s12889-020-09088-4, PMID: 32652969 Wong BKC Fadel SA Awasthi S Khera A Kumar R Menon G . The impact of measles immunization campaigns in India using a nationally representative sample of 27,000 child deaths. eLife. (2019) 8:e43290. doi: 10.7554/eLife.43290, PMID: 30834890 Thomson DR Amoroso C Atwood S Bonds MH Rwabukwisi FC Drobac P . Impact of a health system strengthening intervention on maternal and child health outputs and outcomes in rural Rwanda 2005–2010. BMJ Glob Health. (2018) 3:e000674. doi: 10.1136/bmjgh-2017-000674 Borkum E Rangarajan A., Rotz D., Sridharan S., Sethi S., & Manoranjini M (2014). Evaluation of the team-based goals and performance based incentives (TBGI) INNOVATion in Bihar Mathematica policy research. Dubé E Gagnon D MacDonald NE. Strategies intended to address vaccine hesitancy: review of published reviews. Vaccine. (2015) 33:4191–203. doi: 10.1016/j.vaccine.2015.04.041, PMID: 25896385 Heneghan C Plüddemann A Spencer E Brassey J Rosca E Onakpoya I . (2022). Interventions designed to improve vaccination uptake: scoping review of systematic reviews and meta-analyses—(version 1). medRxiv [Preprint]. doi: 10.1101/2022.01.26.22269890