This article studies the body of agriculture evidence synthesis in Nigeria as a basis for evidence-informed policymaking (EIPM). Politicians and decision-makers often face criticism from EIPM scholars for not using scientific evidence to support their decisions or for using it selectively to fit pre-existing beliefs (Buffardi et al., 2020; Fussy, 2022; Jones and Louis, 2018; Newman et al., 2015; Ouimet et al., 2023; Strassheim and Loer, 2019). Much EIPM research has focused on this low or non-use of evidence by policymakers. If policymakers are not using scientific evidence as expected, understanding these decision makers’ perspectives is crucial to enhancing science’s role in decision-making. Thus, a section of the EIPM literature has also been dedicated to exploring barriers and facilitators to using scientific evidence in policymaking to improve the role of science in the policy process. Part of the challenges identified in this literature is the fact that scientific research is often conducted for an audience of researchers, with little focus on policymakers’ needs (Oliver et al., 2014). Notwithstanding the literature on the facilitators and barriers of evidence use, a gap remains in understanding whether producers of scientific evidence consider the identified barriers and facilitators seriously when producing policy-relevant products. This study attempts to fill this gap in knowledge.

The push for using scientific evidence in policymaking gained popularity in the 1990s through the UK Labour Party’s efforts to base policies on research rather than political ideologies (Hadorn et al., 2022; Newman et al., 2015; Nutley et al., 2007; Sager et al., 2023). Advocates argue that scientific evidence helps objectively identify problems, propose solutions, achieve optimal results, and evaluate outcomes (Fussy, 2022; Ndu et al., 2022; Whitfield, 2012). They claim evidence provides a sound basis for decisions, encourages critical questioning, and prevents harmful outcomes (Haddaway and Pullin, 2014; Thomas-Walters et al., 2021; Yanovitzky and Weber, 2020). In Africa, the call for evidence-based agriculture policymaking began in the late 1980s, gaining popularity in the 2000s (Whitfield, 2012). In Nigeria, the 1988 civil service reform established departments of planning, research, and statistics (DPRS) in government ministries to address administrative data challenges (Ajakaiye, 2021). However, EIPM faces criticism for selective use and subjective interpretation of evidence (Newman et al., 2017; Schlaufer et al., 2018). Additionally, relying on single studies for conclusions can be misleading and biased (Siddaway et al., 2019; Tricco et al., 2015).

Consequently, within EIPM circles, evidence syntheses and randomized control trials are often praised for their rigor and objectivity (Bedard and Ouimet, 2017, Cairney and Oliver 2017, White and Waddington, 2012) and ranked highest among other evidence sources (“hierarchy of evidence”) when intervention effectiveness, appropriateness, and feasibility are of interest (Figure 1). A systematic review, often called research synthesis, involves strategies to limit bias in assembling, critically appraising, and synthesizing all relevant studies on a topic (Newcomer et al., 2015).¹ These reviews help assess multiple studies to understand or replicate interventions, generalize findings, and summarize complex evidence while limiting bias (Bedard and Ouimet, 2017). Although their use started in medical science, research syntheses (including meta-analysis and systematic reviews) and randomized control trails have been extended to environmental and social sciences (Haddaway and Pullin, 2014). They are praised as the industry standard and best source of evidence; providing policymakers with summaries on intervention impact and effectiveness and identifying points of consensus or dissent across a body of a literature (Bedard and Ouimet, 2017; Haddaway and Pullin, 2014). Compared to single studies, evidence synthesis provides a more objective source for informed policymaking. Recently, living syntheses have also been suggested as even better sources of evidence given that they are continuously updated with the latest evidence to increase value and validity of evidence syntheses in decision making (Elliott et al. 2017; Iannizzi et al., 2021). However, evidence syntheses are currently few (Bedard and Ouimet, 2017) and underutilized (Vale et al., 2015), despite indications that policymakers are interested in using them (Thomas-Walters et al., 2021; Wallace et al., 2012).

This study attempts to bridge the gap in knowledge with regards to the policy relevance of evidence syntheses by analysing systematic reviews and meta-analyses on agriculture in Nigeria using an analytical framework developed based on studies of facilitators and barriers to using scientific evidence. The study answers this question: are evidence syntheses on agriculture in Nigeria produced in a way that predisposes them to being used by decision makers? The paper proceed as follows: in the next section, I present the case of Nigerian agricultural policy and the role of evidence therein. The section after develops the analytical framework before I present our empirical strategy. I then present our findings and discuss them in the light of the analytical framework before I conclude with concrete policy recommendations. I conclude that to promote evidence-informed policymaking, agricultural evidence syntheses should be more policy relevant and available to decision makers in a timely manner and usable formats.

Nigeria, with thirty-six states and a federal capital territory divided into six geopolitical zones, is home to over 200 million people and has an economy over $440 billion (World Bank, 2022), making it the largest population and economy in Africa. About 80% of Nigeria’s 924,000 square kilometres of land is arable, but only half is currently cultivated (Tijani et al., 2015). Agriculture, primarily consisting of smallholder farms, employs over 36% of the labour force (Anugwa et al., 2022) and contributes 23–26% to the Gross Domestic Product (NBS, 2022; World Bank, 2022). By the early 1990s, the agriculture sector recognized the importance of reliable and timely data for planning, decision-making, and evaluation, leading to plans for a National Agricultural Information Management System (Ukpong and Alegieuno, 1992). Despite these efforts, gaps remain in generating timely and credible scientific information for policymakers (Delgado et al., 2019; Elueze, 2016; Liverpool-Tasie and Andam, 2021). The agriculture sector faces challenges such as low yield, low investment, land tenure issues, poor research-extension linkage, and limited access to finance (Olomola and Nwafor, 2018). Thus, there is a potentially significant role for research to drive the transformation of Nigeria’s agriculture sector.

While the body of knowledge on evidence-informed decision-making in Nigeria is growing, there is a notable gap of knowledge on EIPM in agriculture and a lack of understanding regarding the role of evidence synthesis in the policy process. Moreso, most EIPM research focus on whether policymakers use evidence (the demand side), with limited studies on whether scientific evidence is produced in ways that encourage use by decision makers (the supply side). Researchers have noted that research syntheses often face challenges that limit its uptake by policymakers (South and Lorenc, 2020; Vale et al., 2015; Wallace et al., 2014). These challenges include accessibility, length, policy relevance, transparency, implementation strategies, and information on benefits, costs, potential harms, and transferability of findings (Haddaway and Pullin, 2014; South and Lorenc, 2020; Lavis, 2009; Munthe-Kaas et al., 2020; Wallace et al., 2012; Wallace et al., 2014). Addressing these challenges could increase the use of evidence synthesis by policymakers.

As highlighted in the introduction, research syntheses are valuable for policymaking because they provide a comprehensive review of a literature through a transparent and rigorous process that minimises bias (Siddaway et al., 2019). However, producing these syntheses is often time-consuming and resource-intensive, typically taking 9–12 months to complete (Haddaway and Pullin, 2014; Thomas-Walters et al., 2021). Despite these, research syntheses are under-utilized partly because they often fail to offer useful implications for policy and practice (Wallace et al., 2014). While there is considerable evidence on the barriers and facilitators of using scientific evidence, including research syntheses, there is a lack of understanding of whether producers of evidence syntheses prepare them with the recommendations from this literature in mind. To explore this, we develop an analytical framework based on the literature on the barriers and facilitators of using scientific evidence and assess published meta-analysis and systematic reviews against this framework. We focus only on assessing the content of the research syntheses. This means, for instance, that even though the literature emphasizes the importance of policy-relevant dissemination strategies such as briefs, summaries, and targeted messaging (Lavis, 2009; South and Lorenc, 2020; Wallace et al., 2014), it is outside the scope of our study to assess if authors provided this to policymakers. Nonetheless, our theory-driven framework is useful in providing insights on whether evidence syntheses are being produced in ways that increase the likelihood of their use by decision makers.

With the framework, we first provide a general description of the evidence syntheses by examining their publication year, type (qualitative, quantitative, or mixed), number of primary studies included, length, open access status, funding support, and local authorship contribution (Table 1). Next, we analyse each synthesis based on the stage of the policy process it applies to, such as problem framing, policy option identification, or implementation strategies see Table 1 in Lavis (2009). For example, studies that identify indicators or factors contributing to a policy problem, compare data across different contexts, or offer alternative framings are useful for problem framing. Studies that identify policies or programs addressing the problem, including their impacts, cost-effectiveness, and success factors, inform policy option identification. Finally, reviews that identify barriers to implementation or compare the effects of different strategies are relevant for the implementation stage (Lavis, 2009).

We then evaluate the reviews’ potential usefulness to the policy process by examining their motivations, collaboration with policymakers, clear provision of policy implications, and inclusion of information on benefits, cost-effectiveness, or economic information. These factors are known to influence the use of evidence syntheses by policymakers [see Siddaway et al. (2019), South and Lorenc (2020), Thomas-Walters et al. (2021), and Wallace et al. (2012)]. Reviews are typically conducted either due to research gaps/curiosity or commissioned by policymakers (Haddaway and Pullin, 2014). Given that researchers tend to have different priorities than policymakers (Hadorn et al., 2022, Newman et al., 2015), it is likely that purely curiosity-driven syntheses focus on identifying research gaps, whereas commissioned syntheses address specific policy issues and are more useful to decision makers. We also assess whether the synthesis process involved collaboration with stakeholders, a factor that enhances the likelihood of evidence syntheses’ use (Munthe-Kaas et al., 2020; Thomas-Walters et al., 2021). Additionally, we examine if each synthesis clearly states policy implications, as research syntheses should target decision-makers and provide practical recommendations (Thomas-Walters et al., 2021; Siddaway et al., 2019). Finally, we analyse whether the syntheses include information on benefits, costs, and economic implication, which are crucial for policymakers (South and Lorenc, 2020; Wallace et al., 2012).

Lastly, we examine how transparent authors are in the evidence synthesis. Without high-quality evidence, necessary policy changes might not occur, or decisions could lead to harmful outcomes (Haddaway and Pullin, 2014). Recognizing the importance of quality evidence in policymaking, we assessed the syntheses’ quality based on their transparency in research methods and reporting of limitations and potential risks or harms (Haddaway and Pullin, 2014; Tricco et al., 2015; Wallace et al., 2012). A known facilitator of evidence synthesis use is if the synthesis increases the confidence of policymakers (Wallace et al., 2014). Policymakers are likely to trust and use a research synthesis if the authors clearly provide easily interpreted methods, assumptions, bias, and limitations involved in the research process.

Research synthesis can be broadly classified into quantitative and qualitative approaches (Siddaway et al., 2019). Boaz et al. (2006) also identified three classes of synthesis, including quantitative synthesis, qualitative or narrative synthesis, and meta-ethnography. This study employs a qualitative systematic review, or narrative review, which is suitable for addressing specific research questions posed. A narrative review is effective for synthesizing studies with different theoretical frameworks, constructs, and relationships [Baumeister 2013 in Siddaway et al., 2019]. Given that agriculture is very broad, encompassing sub-fields like crops, livestock, fisheries, and forestry, each with further subdivisions such as livestock diseases and production systems, a narrative review is the most appropriate synthesis method. We adhere to standard systematic review procedures: formulating a question, writing a protocol, searching for studies, screening, appraising, extracting data, and synthesizing findings (Haddaway and Pullin, 2014).

After a descriptive account of the sample, we assess the studies along the criteria developed in the analytical framework.

Despite the potential of systematic reviews and meta-analyses in decision-making, they are currently underutilized (Vale et al., 2015) while their status in a developing country like Nigeria, particularly in the agriculture sector, remains not well known. Our study shows that there are limited evidence syntheses (19 in total that meet the eligibility criteria) in the agriculture sector, especially in sub-sectors like fisheries and forestry. Although it is difficult to conclude definitively that Nigerian agricultural policymakers do not consult evidence syntheses (as our focus is on peer-reviewed evidence syntheses, while there may be commissioned ones not published or peer-reviewed), our findings suggest a likely low production and popularity of such evidence in the policymaking space. For example, despite yam, cassava, rice, and beans being staples in Nigeria, only yam production has been systematically reviewed, with two studies in total. Morse (2021), who conducted one of the quantitative syntheses analyzed in our study, had noted this low number of evidence syntheses published in Nigeria. This also aligns with findings elsewhere that evidence syntheses are often insufficient on topics important to policymakers (South and Lorenc, 2020). Nonetheless, the involvement of local researchers in conducting systematic reviews indicates local expertise in these methods, even though studies are likely concentrated among groups of the same researchers.

Additionally, there are few quantitative evidence syntheses (meta-analyses) available, which could be partly due to the limitations of data needed for this type of analysis. Authors often cite challenges such as lack of sufficient data, varying study designs, and lack of harmonized measurement methods (Morse, 2021; Odeniran and Ademola, 2018; Oloso et al., 2018). These challenges lead authors to conduct only qualitative syntheses (e.g., Begho et al., 2022; Esonu et al., 2022; Oloso et al., 2018), meta-analyses after improvisation or cumbersome harmonization of studies (e.g., Amadi et al., 2019; Morse, 2021), analyses with high heterogeneity (e.g., Karshima, 2019; Karshima et al., 2020; Odeniran and Ademola, 2018; Odeniran and Ademola, 2019), or meta-analyses without important sub-population analyses (e.g., Odeniran et al., 2021). It is likely that this primary data limitation is why other authors conduct meta-analysis using studies from outside Nigeria (c.f. Ogbuewu et al., 2020, 2021; Ogbuewu and Mbajiorgu, 2022).

Furthermore, most of the syntheses were published without funding support from Nigerian government agencies and with almost no collaboration with government stakeholders (at least from the list of authors of the primary studies). Moreover, more than 47% of the syntheses were not open access which could limit accessibility to policymakers, a known barrier to use of evidence (Cherney et al., 2015; Oliver et al., 2014; Williamson et al., 2019). Evidence synthesis published to fill research gap, with no funding support from the government, no collaboration in the production, and no clear recommendations for policy action will likely not be useful or usable for policymakers. Creating a space for relationship between researchers and policymakers could facilitate use of evidence (Tricco et al., 2015) as it can create legitimacy and trust for researchers and help them identify policy priorities to focus evidence synthesis on.

In this study, we made a distinction between the syntheses’ policy process applicability (i.e., whether the content of the synthesis can find application in one or more of the three stages of the policy process) and policy implications (i.e., if the syntheses provide explicit recommendations for policymakers). For applicability, our findings show that while many of the reviews could potentially be applied to problem definition, only a few focus on identifying options to address the policy problem or how to implement these options. The lack of implementation strategies in evidence syntheses is established in the literature as a barrier to their use (e.g., Wallace et al., 2014). This points to a gap either in evidence syntheses relevant to these stages of the policy process or a lack of relevant primary studies to conduct syntheses relevant to the stages of the policy process. For instance, although syntheses related to effectiveness of different policies could help policymakers identify suitable policy option or how to best implement policies, their almost non-existence among the studies reviewed hinders evidence-informed decision making, especially for public administrators whom this type of evidence is most suited for (Sager et al., 2023). In any case, the limited number of syntheses that offer policy solutions to policy problems or advice on how to implement policy solutions restricts the role that evidence can play in the policy process, at least from an ‘object-bound’ policy knowledge (Sager et al., 2023) point of view.

Similarly, few of the studies dedicate time to explain the relevance of their studies to policy and offer clear recommendations for policymakers, even though evidence syntheses are expected to do this (Haddaway and Pullin, 2014; Siddaway et al., 2019; Thomas-Walters et al., 2021). This could be because most of the syntheses were conducted to fill a research gap (rather than because they were commissioned by decision makers) and provided recommendations for future research more often than for policies. It could also be due to the differences in the priorities of researchers and policymakers: while researchers go after publications and publish for a research audience, policymakers are more interested in timely, and policy-relevant information (Hadorn et al., 2022). This aligns with previous finding by Oliver et al. (2014). The syntheses also hardly discuss economic implications even though some researchers (e.g., South and Lorenc, 2020) have noted that this kind of information are of importance to policymakers.

Even though evidence synthesis may be one of several types of scientific evidence in the policy process and may not be appropriate in all instances, they still present an objective way of summarising and making sense of the body of evidence to make more informed decision. This analysis has shown that despite the prospects of evidence synthesis in policymaking, there are currently few evidence syntheses relevant to agriculture in Nigeria and they are motivated by filling research gaps rather than meeting policy needs. While most of the reviewed syntheses were transparent on methods and limitations, they focused more on problem definition than other stages of the policy process and on livestock compared to other agriculture sub-sectors. Furthermore, a common issue highlighted across many of the syntheses is the challenge of availability of sufficient primary studies which limits the production of more quantitative evidence synthesis.

Following the gaps identified, this study offers a basis for concrete policy recommendations. Addressing the low number of evidence synthesis first requires educating policymakers on the usefulness of this type of evidence to create demand for them. Similarly, producers of the evidence synthesis could benefit from training on making evidence syntheses more policy-relevant (i.e., highlighting the policy implication of evidence synthesis and providing more recommendations on how to solve policy problems/how to implement policy solutions). Given the repeatedly mentioned challenge of primary data limitation, it will be useful to increase support for research across the country through funding, especially for primary studies that are based on rigorous experimental or quasi-experimental methods (or meta-analyses of these) that offer quantitative insights on (cost-)effectiveness of government policies. Research funders should dedicate special funding to cover publication costs of evidence syntheses so that more syntheses are published as open access and decision makers who need them may access them. Special research grant calls could be dedicated to areas which are of importance to policymakers but with little or no coverage by evidence syntheses. Additionally, internal review boards across universities in the country could work to ensure that there is some harmonization or comparability in variables measured in research proposals to avoid the complexities associated with variables measured differently across primary studies.

As agriculture policy in Nigeria leapfrogs from primary studies to evidence synthesis vis-à-vis evidence-informed policymaking, it also needs to quickly catch up with current trends in this space. Considering the time and other resource commitments involved in producing evidence syntheses (Haddaway and Pullin, 2014; Thomas-Walters et al., 2021), producing syntheses that quickly becomes outdated is not cost-effective and would not serve policymakers needs. Hence, there is need to shift from conducting static evidence syntheses to a paradigm of living evidence syntheses which is syntheses that are constantly updated and hence continuously relevant. Government agencies and development partners involved in funding research can support research institutions to gain expertise in this area to meet the current needs of policymakers. Such support should involve policymakers and identify key policy priorities so that syntheses produced are relevant, timely, and have immediate policy use.

Finally, the analytical framework used in this study applies to peer-reviewed studies. Future study may extend the data sources to include grey, unpublished, and non-peer reviewed articles to better understand if evidence syntheses are produced in ways that increase their likelihood of use by decision makers. Further research in this area may also be needed to understand the perception of policymakers about systematic reviews and their interest in using them in the decision-making process, evidence on which is currently missing. Beyond decision makers’ perception, future studies can also more objectively trace the use of this kind of evidence in policymaking using newly developed methods such as Jørgensen (2024) or Yanovitzky and Weber (2020). Given the limited knowledge on the effectiveness or cost-effectiveness of evidence-informed policies (Buffardi et al., 2020), future studies could also assess if policies supported by evidence syntheses produce better outcomes than those based on single studies.