For the general overview, the evaluator should put together a concise description of the background, objectives, key features, and rationale of the OH initiative under evaluation so that the user is aware of the important characteristics that can affect the evaluation.

Here, the focus is specifically on the system targeted by the OH initiative; in other words, the wider context within which the initiative operates. We will describe the initiative itself later. For the visual representation of the system, we propose a combination of the socio-ecological system framework by Ostrom and a causal loop diagram (13, 15).

To capture the socio-ecological system, three core subsystems are plotted first (Figure 2): the resource systems^T1 (blue ovals), the resource units^T1 they provide (dark blue boxes), and the governing systems^T1 (grey boxes). In the next step, further tangible and intangible components relevant to the system (white ovals, e.g., use of antibiotics, effectiveness of antimicrobials) are added. For legibility of the graph, it is recommended to use nouns that fit into phrases such as “the level of…,” to avoid verbs and to use neutral terms, e.g., “use of antimicrobials” rather than “increase of antimicrobial use.” Finally, relationships are added as arrows, namely governance relations (grey), membership relations (black), and causal relations (blue). For causal relations, it is useful to note the relation using S for “same direction change” and O for “opposite direction change,” in order to identify later reinforcing and balancing loops. Subscripts and explanatory text as well as annotations of time delays can be convenient for later reference.

Visual representation is powerful, but lacks any dimension beyond the plane and therefore hinders the depiction of overlapping subsystems or nested hierarchies. Hence, to explore further the system in which the OH initiative operates, it is recommended to include a textual description. It is guided by three questions formulated by Williams (28): (A) to understand interrelationships: what is the reality we are dealing with? (B) to engage with perspectives: how do we understand/how do we see that reality? (C) to reflect on boundaries: how do we decide to do what needs to be done? (28). In Table 2, we adapted the tabular system description by Boriani et al. (31) for a broader application. It allows capturing the aim of the system, the stakeholders and actors and their interactions, the system dimensions with corresponding boundaries, and the system evolution.

The aim and/or indicators of the system are not to be confused with the aim of the initiative and should answer the question “why does the system exist?” or “what does it produce?” e.g., the result of a food chain may be to “produce Salami.” A social-ecological system may not have an explicit aim, but it can be characterised by indicators that allow describing selected attributes, such as resilience, productivity, or health. In this evaluation framework, we differentiate among the declared aim by the system and the observed, enacted, and the perceived aims. The declared aim of a veterinary practice may be to provide animal health services. However, this will be enacted within a socio-economic context, which may result in therapeutic choices that prioritize practice income over animal welfare. These actions may be observed by a subset of clients, while others do not notice them. Each stakeholder may have a different perception of the declared aim and again, each of them can have a different way to interpret how the system is performing in relation to its aim (13). In socio-ecological systems, the perceptions differ mainly in regard to the way one verifies whether the system is healthy and/or intact. This is important as it explains the motivational background and sets of values of the concerned stakeholders. Indicators specific for the system aim should be identified in a participatory process and compared with indicators used by different stakeholders to assess their perceived aim(s), thereby shedding light on discrepancies and ways of resolving them.

Following the interactive terminology for Europe (17), we define stakeholders^T1 as “any individual, group, or organisation who may affect, be affected by, or perceive themselves to be affected by a decision or activity,” while actors^T1 are a subgroup of stakeholders such as “any individual, group, or organisation who acts, or takes part” in system activities. To gain clarity about roles of stakeholders, we recommend referring to the visual representation of the system exemplified in Figure 2 and probe for “who is involved in the system as an actor and who is merely affected?” For example, the pharmaceutical industry produces a certain compound, people can decide whether to take that compound or not, while animals are affected by a certain preparation distributed to them by an actor in the system (e.g., veterinarian or owner). An overview of relevant actors and stakeholders allows delimiting further the system under evaluation. Stakeholders could be actors at the same time, and in these situations, it should be differentiated in what capacity a group represents a stakeholder or actor, respectively.

In order to understand the context of the OH initiative, it is important to understand how the components of the system are arranged or interact (28). There are four aspects of relationships that should be considered and described: (a) the structure or arrangement of the links between the components (topology); (b) the nature of the processes between the components (e.g., information flow, transfer of goods, etc.); (c) the characteristics of the links (slow/fast, strong/weak, antagonistic/synergistic, etc.); and (d) identifying the links that are most important in the system.

Dimensions^T1 are defined as spaces in which levels of organisation according to Bunge occur (29). In other words, entities within a dimension feature the same quality (e.g., metric) but to a different degree. Examples include geographical space, time, governance/institutional, economic, linguistic, faith, and value dimensions. Within these dimensions, we consider scales^T1 or levels^T1 of analysis, e.g., cell—organism—population in the dimension of life (18). These levels are important, because they will determine the relation between the resolution of the analysis and the resolution of observations and what can be measured or evaluated in the system in a particular dimension. Due to their importance, geographical, temporal, and governance/institutional dimensions are included in Table 2. Particularly time is related to the scale in other dimensions, i.e., the larger the system the larger its characteristic time, which is the time at which the average change occurs (e.g., cells react within milliseconds, individuals within minutes to hours, ecosystems is within years or decades, the same applies to the adaptability of laws at different scales or the frequency that vocabulary is used in a language) (18). Together with geographical space, time is a particularly important dimension, because it will characterise if the system is evolving over seconds, hours, days, years, decades, or even longer. It can be considered in the past, present, or future, and opportunities to affect the system are highly dependent on time due to the system disposition (the same intervention may have different effects when applied at different times). Furthermore, causes and effects may occur in different time scales, where short actions may result in effects with a time lag of years. The governance/institutional dimension will determine which organisational levels (ranging from international governance mechanisms to household structures) are represented and addressed in an initiative. Considering scales is important, because initiatives may aim to change systems at different levels than where the necessary governance could be influenced and consequently, well intended initiatives may remain ineffective if they do not address all appropriate levels.

Further dimensions are the Dimension of Life (or Biology) comprising nested living entities from cells to biosphere with levels such as “cell,” “organ,” and “individual,” the Economic Dimension defined by rules and institutions involved in decisions on production, trade, and exchange of goods and services, the Linguistic Dimension delimited by languages and dialects used, the Faith/Value Dimension, which represents the values and beliefs underlying the system. Other dimensions may also be relevant to the system, such as communication, transportation, legal frame, sociocultural dimensions, and many others.

The primary importance of a systems approach to evaluation implies less the idea of being comprehensive, but rather being “thoughtful, smart, and aware about what you are leaving out” (28). The evaluator(s) will need to be transparent about the consequences of choices and declare their relation to the initiative, the system, and the evaluation per se. Although the dynamics and boundaries and stakeholders of a system are clear, they will be constrained by physical limits (e.g., a mountain range, river), social limits (e.g., country, community), regulations (e.g., quotas, prohibitions), and/or other norms (e.g., social norms, religious norms) that are either imposed by the systems nature or selected by the evaluators (23). Many restricting factors will lie in one of the system dimensions identified earlier. For example, a food system can be limited due to production regulations (e.g., the previous milk quotas system in Europe), food hygiene standards (e.g., restrictions on raw milk consumption), or cultural practices (e.g., no pork consumption in certain faith groups). The system boundaries characterise the interaction between the context of the initiative with the broader world in which it is imbedded, and determine how this affects the aim of the system (23). Finally, dimensions can also interact and may even be so closely correlated that it may not be useful to differentiate them (e.g., when religious beliefs are prescribed by the law).

The evolution of a system can be regarded as interaction of time with other dimensions in terms of iterations and pathways along those dimensions and time. Apart from the aim of the system, the interactions in the system may produce secondary effects within the system and “externalities” beyond the boundaries as it evolves. Highly self-organising systems may even change their (aim) dynamics and boundaries as time goes by.

In a next step, the OH initiative can be added to the visual representation of the context to illustrate its effects on various components^T1 and their interactions. If an affected component is missing, it is added and the system graph corrected accordingly. In the example in Figure 2, we have included a hypothetical OH initiative that involves new antimicrobial treatment guidelines for veterinarians and general practitioners (prescribers) that are assumed to impact directly on the amount and distributions of types of antimicrobials used in the system.

The user should now have a clear understanding of the system in which the OH initiative is situated. Next, the initiative itself is described using the template in Table 2 in analogy, namely as a nested subsystem of the context, which it aims to change. Many elements may be congruent, but the boundaries of the initiative will inevitably be smaller and there will be fewer actors, stakeholders, and more limitations than in the description of the system. Care should be taken, as actors and stakeholders and their particular roles, may not be identical in the initiative and in the wider system. The initiative may be likely to consider fewer dimensions compared with the system, but it is important to identify how it will influence the context and what the limitation of the actions are. A key question in this description is: how is OH conceptualised by the various participants and is there a common understanding?

Essentially, the TOC presents a roadmap with all building blocks required to bring about a desired (long-term) goal and hence spells out the logic behind the initiative. The presentation of the TOC can be assisted by a graphical presentation (e.g., Figure 3), or its description can refer back to the illustration of the system used in Element 1.

The impact^T1 is defined as the long-term effects (or goals) to be induced by an OH initiative. It is a change that continues to exist after the end of the initiative, and can be direct (first order) or indirect (second order) impacts. Outcomes^T1 are changes (e.g., improvement, learning) resulting from the initiative that can be considered to be stepping stones for progress towards the longer-term goals. In a transdisciplinary process, the outcomes are situated in societal and scientific practice and can be of multiple natures (e.g., technical, economic, social, sanitary, and political) (33). Outputs^T1 are products, goods, and services, which result from the transdisciplinary process of an OH initiative and are necessary for the achievement of outcomes. For illustration, we use an example from a fictive research project aiming to produce new knowledge and methods to combat the development of antimicrobial resistance (Figure 3): OH research outputs (new data and knowledge) result in new treatment guidelines (outcome for societal practice) leading to new regulations restricting (and hence lowering) the use of specific antimicrobials in farmed animals (first-order impact of political nature), which then may reduce the development of antimicrobial resistance in farmed animals and the associated transmission to people (second-order societal impact). The impacts can be realised at different political levels (e.g., individual, institutional, regional, national, and international) and can consist of different types of effects (positive or negative; direct or indirect). Outcomes for societal and scientific practice (e.g., an improved integrated surveillance programme for antimicrobial resistance or a new simulation model, respectively) are disseminated, adapted, and applied by other actors to result in societal impact or scientific progress. Between the initial problem formulation and the expected impact(s), new inputs might be required as a result of intermediary outcomes and will feed a further iteration of knowledge co-production. An example could be new research collaborations as the outcome of an OH initiative, which may lead to new knowledge or tools for improved control of infectious diseases in a second initiative. The sequence of inputs (i.e., resources needed to perform the actions), outputs, outcomes, and impact can be graphically represented by a change pathway also known as an impact pathway (34) or a logical framework^T1 or logic model^T1, which presents the flows in a “logical,” sequential way (19). Importantly, the classification into outputs, outcomes, and impacts depend on the perspective that is taken for the evaluation and may differ among stakeholders (35). It is therefore important to elaborate the TOC in collaboration with the entity contracting the evaluation.

A methodology related to the TOC is outcome mapping^T1, which can be used for planning and assessing (development) activities focusing on change and social transformation. It places people and learning at the centre of development and conceptualises unanticipated changes as potential for progress and innovation. Consequently, it can be a useful tool to use for OH initiatives, either in combination with TOC or on its own if it fulfills key assumptions of dependence on human behaviour, limits to the influence of interventions, active contribution of people to their well-being, co-existence of differing yet valid perspectives, and resilience dependent on interrelationships (36).

The description and definition of outcomes and impacts are dependent on the problem the OH initiative is addressing and the associated boundaries of the system, objective, rationale, and consequently the resulting TOC. Given the diversity of OH initiatives, there is no single outcome that summarises OH endeavours, but rather a wide range of different outcomes (37–39). However, at the longer-term impact level, there are commonalities OH endeavours appear to strive for (7). The outcomes and impacts to be measured need to be selected as best fit for the specific OH initiative and its TOC. Because of their nature, OH initiatives will commonly span different sectors and disciplines and therefore are likely to produce disciplinary, interdisciplinary, and OH outcomes and impacts. Evaluators consequently need to be aware of disciplinary paradigms, data, and approaches as well as methods of combining outcomes from different disciplines.

Disciplinary outcomes relate to outcomes that are measurable within a distinct discipline or sub-speciality within the natural or social sciences. Examples of disciplinary outcomes include health outcomes such as decreased levels of non-communicable or infectious diseases; nutrition outcomes such as reduced levels of undernutrition or obesity; economic outcomes such as increased productivity or savings in the health care system; social outcomes such as improved societal stability; and ecological outcomes such as slower rates of biodiversity reduction or improved water or air quality. Importantly, these outcomes can be achieved in disciplinary or sectoral approaches (e.g., promotion of a new anti-diabetes treatment or childhood vaccination in a national health service), but more often, they rely on collaborations across disciplines and sectors. Interdisciplinary activities by definition have an impact on multiple fields or disciplines and produce results that feed back into and enhance disciplinary or sectoral work. In these instances, the pathway to the outcome may be characterised by collaboration and contributions from different disciplines and sectors, but the outcomes may still be conceptualised (and consequently measured) at the level of a field or discipline. Combining these disciplinary outcomes in methods such as multi-criteria decision analysis gives a solid basis for an assessment of the achievements of the OH initiative. In interdisciplinary outcomes, individuals from different disciplines create new knowledge and understanding through sharing of ideas and bringing together different perspectives result in a product or measure, which explicitly reflect the shared responsibility among disciplines for outcomes (16, 22, 40). Consequently, interdisciplinary outcomes occur in the realm of at least two disciplines simultaneously, e.g., food security as an interdisciplinary outcome of successful alignment of multiple sectors (i.e., food availability, food access, and food utilisation), which contribute different skills and expertise (41). Other examples are the human development index, the environmental performance index, and the planetary boundaries, which combine a diversity of indicators into a single or a few measure(s). An improvement in the index cannot be achieved with a disciplinary approach, but needs activities in health (e.g., investment in health service capacity, public awareness campaigns), education (e.g., build infrastructure, attracted attract talented teachers, and provide incentives for school attendance), social protection (e.g., policies to reduce poverty and vulnerability of disadvantaged population groups), and economics (e.g., promotion of efficient labour markets, robust governance). Interdisciplinary outcomes are ideally measured in a common metric, i.e., they should rely on a consensus on how to assess and weigh the particular outcomes. Such metrics are even more policy relevant and effective if they are produced and measured in a transdisciplinary process, which transcends both horizontal boundaries between scientific disciplines, and vertical boundaries between science and other societal fields (private sector, public agencies, and civil society) (42). Like this, stakeholders share different perspectives and can therefore improve the contextualization of the problem and its potential solutions and targets (43).

One Health outcomes or impacts occur as result from a broader integration of activities in the system at stake. The main domains of OH outcomes are the three pillars of sustainability, i.e., society, environment, and economy. Typical examples are interspecies equity, health stewardship, human and animal welfare, efficiency, and effectiveness (7). Clear causal attribution to the OH initiative may be difficult, but a contribution of the OH initiative can be assessed.

Given the perspective chosen and the resource availability for the evaluation, the description of the TOC, and the selection of associated outcomes may be more or less comprehensive and complex. However, the evaluator should make sure to pay careful attention to the contributions from different disciplines and sectors, their integration and the resulting positive and negative effects.

By definition, unexpected outcomes and impacts cannot be planned or covered by a TOC, even though attempts are sometimes made to capture a wide range of eventualities. Throughout an OH initiative within its system, interactions among components and feedback loops frequently produce rapid, non-linear, and unanticipated changes (23, 44, 45). Typically, integrated approaches in complex systems generate unexpected added value, e.g., a new stakeholder organisation, but may also result in unexpected negative impacts, e.g., discrimination among stakeholders (23), which is why capturing unexpected outcomes constitutes an essential process of OH evaluation. Other examples would be emerging diseases due to new contact rates or closer contact between previously isolated populations, or due to new social behaviours in urbanised environments (46). If unexpected outcomes are not captured, evaluation fails in informing adaptive management that seeks to improve outcomes in complex dynamic environments (47). An expanding array of qualitative and quantitative methods for complexity-enabled monitoring, evaluation and learning is available for use in the fields of development and peacebuilding (48–50), many of which can be contextually adapted for OH projects and programmes.

One Health as a systemic approach with corresponding methodology is of little worth if not based on a foundation of systems thinking (14). This tool assesses how an OH initiative conceptualises the system in which it operates and in how far it considers features specific to complex adaptive systems. The fundamental idea is that a complex initiative addresses multiple dimensions of the system in which it operates (see Element 1 above). The first set of questions (Table S1 in Supplementary Material) measure the number of dimensions and the scales within each to gain a semi-quantitative appreciation of the context and the embedded OH initiative. The following questions assess the match between the dimensions of the initiative and its context. Particular attention is given to the scales in different dimensions and whether the initiative reflects the reality of the context in which it operates. A third set of questions probes for concepts and thoughts typically contained in a systems approach (13, 53). To assess systems thinking in written documents, e.g., in a retrospective evaluation or in a proposal, we refer to a method based on statistical semantics proposed by Whitehead and Scherer (54).

One Health planning is essentially the unfolding of the OH thinking into operational features of the initiative that should facilitate OH working towards achieving the aims and objectives during as well as after the OH initiative. The planning of OH initiatives go beyond the type of planning that is required for disciplinary and interdisciplinary projects in which it might be easier to maintain control of what tasks, engagement, and resources are required. For instance, OH initiatives typically require human resources with competences in transdisciplinary working methods and excellent communication skills to bridge disciplines and sectors (8). It is important that the planning includes appropriate methods to engage all of the essential actors and stakeholders, who should be aiming to reach a common goal. Part of the planning evaluation is to assess whether the planned structure, location, and timing of the initiative support the OH outcomes aimed for. Due to the complex and trans-domain characteristics of OH challenges, another important aspect of OH initiatives is the ability to self-assess, learn, reflect, and adapt to new knowledge and changing conditions, constraints, and opportunities over time (55). Therefore, adaptability features prominently in the evaluation of the planning of OH initiatives. Finally, the planning evaluation helps assessing the tasks and resources allocated to each task employed to achieve the specified objectives of the initiative. The questions in Table S1 in Supplementary Material were developed to probe if the challenges of complex initiatives described here are addressed in the planning phase and if funding as well as organisational aspects are set up to accommodate adaptive behaviour by the participants. High scores are recommended for a strong support of adaptability and flexibility.

Interdisciplinary collaboration brings together people with different skills and expertise to tackle complex problems, which often have a high-societal stake and require an understanding of the human behaviour (9, 56, 57). Appreciating potential contributions of multiple disciplines requires examining the limits imposed by a discipline, and rejecting or accepting different disciplinary theories based on their relevance and credibility in order to gain a new understanding about the defined challenge (12, 58). In the context of OH, interdisciplinarity^T1 has developed towards a participatory approach in the form of transdisciplinarity^T1 (57). Both inter- and transdisciplinarity rely on appropriate leadership and management to promote strategic dialogue and shared decision-making (40, 59), which in turn will foster a non-hierarchical relationship between the different disciplines and members within the team. It must also allow for self-reflection, flexibility, and recursiveness (40, 42, 57, 60), to be able to challenge and modify underlying assumptions and concepts and thereby enrich understanding. It must be emphasised that such transdisciplinary work demands a high level of commitment and collaboration of all participants to establish personal relationships founded within a climate of trust (9, 42, 59). The questions probing for transdisciplinarity (Table S1 in Supplementary Material) focus on disciplinary diversity, team building, and adaptability and were adapted based on the work cited above.

Further aspects of trans- and interdisciplinarity may be assessed, namely for (A) evaluating (academic) participants and (B) assessing scientific outputs of an OH initiative. However, because individuals may have different roles in an OH initiative, assessing their trans- and interdisciplinary capacity may not always be required or relevant. Also, printed scientific output may not be a primary objective of an OH initiative and occurs with some delay, thereby contributing more to the assessment of outputs than to the implementation per se: (A)

The transdisciplinarity of (academic) participants may be assessed based on the interdisciplinarity of publications [see method (B) below]; interdisciplinarity of teaching, other academic activity (e.g., teaching experience in other disciplines than the own, co-teaching with experts from other disciplines/sectors, etc.); previous experience with various non-academic communities (e.g., public debate, main stream media, sports and leisure organisations, politics, NGOs, volunteering, etc.); involvement in other disciplinary and interdisciplinary networks (e.g., social and natural science networks other than the own expertise, explicitly interdisciplinary initiatives, science policy, etc.);

In many complex settings, change-oriented leadership has helped to overcome the fallacies of conventions, norms, and traditions (61, 62). Complex systems have leverage points where they can be influenced according to their potential to modify a systems behaviour (53). The use of these points by an OH initiative determines the dimension(s) and scales at which the initiative is effective. However, in order to be effective, the implementation of the initiative needs to be facilitated by corresponding leadership behaviour. Yukl classifies leadership into four meta-categories with specific objectives (62): for (A) task-oriented behaviour, the primary objective is to accomplish work in an efficient and reliable way. For (B) relations-oriented behaviour, the primary objective is to increase the quality of human resources and relations, which is sometimes called “human capital.” For (C) change-oriented behaviour, the primary objectives are to increase innovation, collective learning, and adaptation to the external environment. For (D) external leadership behaviour, the primary objectives are to acquire necessary information and resources, and to promote and defend the interests of the team or organisation. These leadership behaviours can be related to the leverage points in a system according to their objectives (Table 3).

Yukl emphasises that all leadership behaviours and particularly their flexible applications are relevant for effective leadership. The table simply illustrates that the lack of a particular leadership behaviour may hamper the implementation of a well-conceived OH initiative. The effectiveness of leadership behaviours also depends on the extent to which the leader is trusted by people to be influenced. Most types of leadership behaviours can be used in ethical or unethical ways. Moreover, a leader, who is not trusted because of unethical behaviour will have less influence. Values, namely honesty, altruism, compassion, fairness, courage, and humility may further catalyse effects of good leadership behaviour. In contrast, excessive institutional structure and organisation can nullify these effects (62). Rooke and Torbert identify further common personality traits of leaders that effectively manage wicked problems: they can challenge the prevailing view without provoking outrage or cynicism; they can act on the big and small picture at the same time, and change course if their chosen path turns out to be incorrect; and they lead with inquiry as well as advocacy, with engagement as well as command, operating all the while from a deeply held humility, and respect for others (63).

A further challenge for leading OH projects is that there may be less interest, commitment, and collaboration if one discipline dominates. Consequently, other disciplines may retract their activity and reinforce the disciplinary silo mentality. To ensure that disciplines are effectively engaged and involved in decision-making from the planning to the implementation stages of projects, shared/distributed leadership, and governance should be implemented involving all stakeholders (64, 65).

Consequently, the selection of questions for the systemic organisation of OH initiatives focuses on the structure of teams, as well as management, social, and leadership skills of key players and its implementation (Table S1 in Supplementary Material). The questions were taken from the leadership assessment tools and the published questionnaires on team work and transdisciplinarity described in Section “OH Working: Transdisciplinarity”. High scores were recommended for strong teams, change-oriented leadership skills, clear competences, goals, and criteria of success.

Learning is a change in cognition, potential behaviour or actual behaviour through better knowledge, and understanding (66, 67). Organisations, such as OH initiatives, learn when they “encode inferences from history into routines that guide behaviour” (68). This is achieved when discoveries, evaluations, and insights by individuals are successfully embedded in the organisation’s mental models or cognitive systems and memories (69). This requires that organisational learning takes into account the learning that takes place at the individual, group, and organisational levels (70) and the interplay between them (69). The three levels of learning work together and influence each other and are thus not clearly distinct and mutually exclusive (71). Nevertheless, each level of learning has its characteristics for evaluation.

Individuals can engage in single-loop or double-loop learning. Single-loop learning happens when the output is corrected or existing competences, procedures, technologies, and paradigms are improved, without necessarily examining or challenging the underlying beliefs and assumptions. In contrast, double-loop learning involves seeing beyond the situation and questioning operating norms. It results in modification of the organisation’s underlying norms, policies, and objectives.

Individual learning is not a sufficient condition for organisational learning (72). Teams enable the interplay between individual and organisational learning, because they can better share the knowledge (72–74) and include more people in the learning process. As a result, team members share awareness of each individual member’s expertise, knowledge, and skills, and build a transactive memory system (8). Thus, the evaluation should examine the knowledge shared through teams, to what extent it is shared and how it is shared. The conclusion should show whether the teams provide the appropriate interplay between the individual and the OH initiative. Without supporting the development of a transactive memory system within and across teams, the initiative may have individuals who learn, but it cannot engage in organisational learning (75). It is important to assess how knowledge is gathered, stored, and distributed within an OH initiative (76), and if and how it provides working environments, technology, rewards, systems, structures, and policies that will support learning (73).

Finally, the context in which the OH initiative is located has influence on the organisational learning (77). The context can be divided into the direct system in which it operates and general environment (78). The direct system consists of other components with which the initiative interacts, e.g., actors and stakeholders with various relationships. The general environment consists of less specific elements that might affect learning like economic, technological, sociocultural, and other factors. The questions probing for learning are taken from a tool to change organisations towards learning organisations (79) and focus on the frequency single-loop and double-loop learning occur at the level of individuals, teams, and the OH initiative, as well as how the system and broader environment support learning (Table S1 in Supplementary Material).

In a broad sense, data and information sharing is a catalyser of knowledge generation (80). Data are often a pre-requisite for the operational gears to function. In OH initiatives, data and information are often the “raw material” that ultimately will lead to better understanding and a more inclusive and sustainable way of tackling the challenge. If managed appropriately, data and unbiased information sharing can foster trust between participants, as well as minimise misconduct in data management and reporting (81, 82). Additionally, this process can avoid duplication of data collection, ensuring an optimisation of resources (83).

A central benefit of data sharing is that the data can be analysed to a much greater extent than if only the data owner examines them. This brings benefits to the data owners themselves, as the analysis of others might lead them to further develop their knowledge on the systems the data originated from or the strengths and limitations of their datasets, as well as raising the awareness of the existence of the data in the wider community (80, 84, 85). Despite these benefits, data and information sharing often lead to barriers for establishing collaborations (86) and are hampered by confidentiality issues, time delays, and even mistrust in established collaborations. Consequently, data sharing is not as frequent as desirable, and needs to be incentivised to become a natural part of the science and governance cultures. For example, in some countries research relies on a tripartite agreement to share information and collaborate between academia, government institutions and industry, but public access to data may also be reinforced through legislation.

A frequent barrier to data procurement is the bureaucratic process to access data, particularly its complexity and duration. Moreover, fees and technical constraints may arise (87), and often too little resources are set aside to for data extraction from databases. Data accessibility and ownership are further critical factors, with data owned by collaborating parties contributing more to knowledge generation than public data or data owned by third parties. Data confidentiality may affect its sharing, as participant consent is usually collected for a specific purpose. This consent might not extend to new studies or alternative purposes, and therefore, security measures may be required to warrant confidentiality. Sharing sensitive data and information within a broader group might entail higher risks for confidentiality breaches (88). Alternatively, anonymization may reduce that risk, but may also reduce the utility of the data. Finally, it needs to be stressed that knowledge about the data origin and data collection processes is key for the quality and usefulness of stored data, and respective documentation must be available. For example, without knowledge about potential bias throughout the data generating process, it is extremely challenging to merge or combine data from multiple sectors in an OH initiative. The questions in Table S1 in Supplementary Material derive from a workshop held by NEOH on data and information sharing, in which critical aspects of data sharing were discussed. High scores are recommended for strong facilitation of sharing. The questions focus on the sharing mechanisms, available resources, data quality and accessibility, storage, and the resilience of these to change in the system.

Given the lack of current, commonly accepted benchmarks and the fact that OH initiatives are strongly context specific, it is recommended to assess them in relation to a context-specific benchmark. Hence, the evaluator should determine what the perfect situation in the given context would look like (using benchmarking data where they exist) and what proportion of this maximum is achieved with the OH initiative.

The aim of the OH-index (OHI) is to combine the assessments conducted in the previous sections of Element 3. To visualise the six assessments, we suggest a spider diagram (Figure 4), in which each assessment is represented by a spoke. The diagram depicts the operational aspects “OH thinking,” “OH planning,” and “OH working” opposed to the infrastructure for “learning,” “sharing,” and “systemic organisation.” Thus, the operational aspects on the top left of the diagonal are opposed to the infrastructure on the bottom right. Each spoke is scaled to cover a range of values between 0 and 1. Consequently, the plot not only illustrates the degree of integration by the surface, but it also shows the balance between the operation and the supporting means through its symmetry over the diagonal, numerically represented as the OHR.

In Figure 4, two exemplary fictive projects are depicted, an example with real data of a comparison of two OH initiatives can be found in the article by Buttigieg et al. (see text footnote 4). The fictive Project 1 depicted here has a highly developed transdisciplinary team with a very comprehensive multi-dimensional approach. However, it appears to lack learning and sharing infrastructure and has a mismatch between the responsibilities, authorities, and means which affects the transdisciplinary working and hence potentially the OH outcomes. On the other hand, Project 2 has well-developed infrastructure and well-defined tasks with sufficient funding, but does not explore the interdisciplinary space nor does it aim at serving multiple species.

The OHI corresponds to the ratio of the surface enclosed by the lines to the surface enclosed if all spokes were equal to 1 (a detailed derivation is provided in Data Sheet S1 in Supplementary Material). Thus, the OHI is (1)OHI={(ScP×ScL)+(ScL×ScP)+(ScS×ScL)+(ScO×ScS)+(ScW×ScO)+(ScT×ScW)}6 where Sc_P is the score obtained in OH planning, Sc_L is the score obtained in learning infrastructure, Sc_S is the score from sharing infrastructure, Sc_O is the score from systemic organisation, Sc_W is the score from OH working, and Sc_T is the score from OH thinking.

The OH-ratio (OHR) is the relation of the surface covered in the top left of the diagonal to the one in the lower right (a detailed derivation is provided in Data Sheet S1 in Supplementary Material). To compute the OHR, the surface of the top left surface (SUR_operation) is calculated (2)SUROperations=34{(ScO×ScW2ScO+ScW)+(ScW×ScT)+(ScT×ScP)+(ScP2×ScLScP+ScL)} and divided by the surface of the lower right (SUR_{infrastructure}) (3)SURInfrastructure=34{(ScP×ScL2ScP+ScL)+(ScL×ScS)+(ScS×ScO)+(ScO2×ScWScO+ScW)} resulting in the following equation: (4)OHR=(ScO×ScW2/(ScO+ScW))+(ScW×ScT)+(ScT×ScP)+(ScP2×ScL/(ScP+ScL))(ScP×ScL2/(ScP+ScL))+(ScL×ScS)+(ScS×ScO)+(ScO2×ScW/(ScO+ScW))