Separate literature surveys were conducted for each coexistence topic in the ISI Web of Knowledge database; only papers published in English were surveyed. For each survey, we first searched for the most influential relevant papers. In some cases, these influential papers were comprehensive reviews, and in others, we selected the original papers that established the theoretical foundation of the mechanism (Supplementary Table S1). Then, we complemented each survey for the missing years that were not covered previously by searching for additional papers that cited the influential papers or that included specific relevant topic terms in the title, the abstract, the author keywords, or the abstract keywords (see Supplementary Table S1 for the exact topic terms). To focus the survey on experimental coexistence studies, we decided that all the selected papers should include, in addition to the specific key terms and citations, the key terms “coexisten^*” or “co-existen^*,” and “experiment^*,” “manipulat^*,” or “treatment” (Supplementary Table S1). By using this approach, we might have missed some relevant papers that are not part of this search engine's database or that only included the specific key terms in the paper's body. However, it served to make the literature search uniform among the various tested topics and study systems. We then screened the titles of both the studies that were included in previous reviews and those identified from the database, and we removed irrelevant papers. The remaining papers were carefully checked, and only the papers that satisfied the theoretical criteria (details for each topic are provided below) were regarded as eligible studies (Figure 2 and Supplementary Table S2). Thus, the decision as to whether a paper was considered eligible or not was based on the design of the experiment and not on its results.

Initially, the papers related to each coexistence topic were scanned by a single reviewer (coauthors HH, MG, CC, SH, or SC), using the papers' titles and abstracts. Then, coauthor HH, who defined the theoretical criteria for each topic, made decisions regarding papers whose eligibility was in doubt, read the eligible papers, and extracted the required information from these papers. The design and reporting scheme of the surveys conform to PRISMA reporting guidelines (Liberati et al., 2009).

To estimate the prevalence of coexistence, we searched for experimental studies that included invasibility tests (see below). To choose the target coexistence mechanisms, we used Chesson's synthesis (2000b), which organizes the multiple classical coexistence theories under a unified framework. We selected theories (i) that deal with local coexistence, (ii) that have well-defined theoretical criteria, (iii) that are relatively simple and intuitive, yet constitute building blocks for more complex models, (iv) that encompass the essence of consumer-resource, enemy-consumer, and environment-consumer interactions, (v) that, to the best of our knowledge, are familiar to experimentalists, and (vi) for which a preliminary literature search detected at least five experimental studies.

Below, we briefly introduce each of the topics and the selection criteria that were applied to regard a paper as eligible for testing each topic. Theoretically, proper tests of coexistence mechanisms should associate the features of the mechanism under test with demonstrated stabilization, such as mutual invasibility or the density and frequency dependence of long-term growth rates. This is because the presence of a required set of conditions, by itself, does not imply that these conditions are the ones that promote coexistence; other stabilizing effects may dominate (Chesson, 2018). However, due to the paucity of studies that have included this association, a proper test of mutual invasibility was a sufficient criterion to select a paper as eligible for this topic, and proper tests for the features of each of the four mechanisms (see below) were sufficient criteria to select a paper as eligible for each one of them. In Supplementary Table S2, we indicated, for each eligible invasibility study, whether it was associated with a specific mechanism, and for the studies designed to test each of the four coexistence mechanisms, we indicated whether the purported mechanism was associated with mutual invasibility tests or any other coexistence proxies.

Three obstacles prevented us from presenting a standard mathematical model for each topic, namely, the models' complexity levels, their multifold techniques [e.g., compare the approaches used by Roughgarden and Feldman (1975), Hutson (1984), and Kuang and Chesson (2010) to investigate the stabilizing effect of the frequency-dependent exploitation by generalist enemies mechanism), and the original absence of a mathematical approach for the natural enemy partitioning mechanism (Janzen, 1970; Connell, 1971)]. Thus, instead, below, we refer the readers to the papers that include the most simple and basic models for each of the topics. To illustrate the experimental design required for testing the various coexistence topics, we refer, in each topic, to the same basic design elements, namely when each species is maintained alone and in a species mixture, at low and high densities (Figures 3A,C). The search methods and terms, the number of papers retrieved in each specific search, and the number of eligible papers from the selected pool are presented in Figure 2 and Supplementary Table S1.

For each eligible paper, we recorded: (i) the number of target species (i.e., species hypothesized to coexist) used to test the hypothesis (invasibility criterion or specific mechanism), (ii) the type of environment (laboratory, field, or outdoor experimental systems in which the species compositions were determined by the researcher, e.g., microcosm, mesocosm, or macrocosm), (iii) the manipulated factors (factors related to the competitor species, their resources and enemies, and their environmental conditions), (iv) whether a model was embedded in the study, (v) the experimental design, (vi) whether the hypothesis was supported by the experimental results, in which, for the invasibility tests that had an a priori prediction about the likelihood that the target species coexist, we recorded whether the prediction was supported or not, and (vii) whether the purported mechanism was associated with mutual invasibility tests (Supplementary Table S2). Some of the studies conducted multiple experiments with either multiple species combinations or with the same species combination under a variety of conditions. For these, we reported “support” when all multiple experiments supported the hypothesis, “no support” for the opposite case, and “partial support” when some experiments supported and others refuted the hypothesis. Furthermore, to account for the variability within each paper, in addition to the exploration at the study level, we extracted, from each eligible paper, all tests conducted for an interacting species combination and gave it a score of either “0” or “1” if less than 60% or at least 60% of the tests supported the theoretical hypotheses, respectively. For each species combination, we also indicated the organisms' group, habitat type, and size (Figures 4B, 5). Considering the small number of eligible studies detected for each coexistence topic and the clear link between particular mechanisms and study systems (Figures 2, 4, 5 and Supplementary Table S2), below we provide a qualitative summary of the results, without statistical analyses.

The initial screening covered 2,510 papers; of these, 89 had been identified by previous reviews and 2,422 were new studies identified through the ISI Web of Knowledge search. Of the initial 2,510 papers screened, 2,035 were excluded after screening of the titles and abstracts, and an additional 379 papers were excluded as their design did not satisfy the theoretical criteria (Figure 2). Among the latter group of ineligible papers, three types of study were rejected because they failed to satisfy the theoretical criteria for invasibility tests: most did not include a manipulation of frequency- or density-dependent interspecific competition (70%); others did not determine mutual invasibility through growth rate assessments (15%), and yet others had not established the resident community before introducing the invader (15%). Thus, in sum, 96% of the papers that incorporated the invasibility terms comprised neither invasibility experiments nor model parameterization studies that evaluate their parameters based on intra- and interspecies competition assays (Figures 1, 2 and Supplementary Table S1). For the resource-ratio mechanism, most studies that failed to satisfy the theoretical criteria included only one resource (54%) or had not calculated R^* for all tested species (40%). Others did not include a mixed culture of the species (1%) or did not test any of the predictions about community composition (5%). For the natural enemy partitioning mechanism, most studies that failed to satisfy the theoretical criteria used a generalist predator (36%) or multiple predators that interacted among themselves (28%). The others were observational (8%), did not manipulate the enemy presence (17%) or the densities of the target species (7%), or did not assess the interaction between the victim species (4%). For the FD-exploitation mechanism, most studies that failed to satisfy the theoretical criteria used victim species frequencies of three or less (92%), and one study focused on the predator's perspective using virtual prey (8%). For the storage effect mechanism, most studies that failed to satisfy the theoretical criteria did not test the two storage effect criteria (75%), whereas the others (25%) were observational.

In total, 96 papers were retrieved for detailed evaluation; 14 of them had already been identified by previous reviews (Supplementary Tables S1, S2). The limited number of eligible studies precludes a proper analysis of the effect sizes, and thus, the description of the results provided below is qualitative. The eligible papers and their distribution among categories, with regard to the organisms' group, habitat type, and size, are provided in Figures 4, 5.

Invasibility tests: Of the 24 eligible studies, 42% comprised invasibility experiments, and 58% included model parameterization studies (Figure 4A). Only 50% of these 24 studies associated the mutual invasibility tests with the features of a specific mechanism (column “mechanism-invasibility” in Supplementary Table S2). Importantly, an invasibility-criterion-based examination of the eligible studies that were conducted on species co-occurring in nature revealed that only 37% of the species combinations were predicted to maintain long-term coexistence, suggesting that the current co-occurrence of the other 63% of species combinations may be a transient phenomenon (Figure 4B). The two main tested groups were phytoplankton and terrestrial plants (22 and 73% of the 140 tested species combinations, respectively), and their tendency toward long-term coexistence was similar (Figure 4B).

Four coexistence mechanisms: Support for the resource-ratio (81%, N = 47) and storage effect (74%, N = 19) mechanisms was relatively higher than for the natural enemy partitioning (46%, N = 164) and FD-exploitation (44%, N = 18) mechanisms (Figure 5A). The species combinations that supported each mechanism were not randomly distributed among groups, but were dominated by phytoplankton, terrestrial plants, aquatic invertebrates, and terrestrial plants, for the resource-ratio, natural enemy partitioning, FD-exploitation, and storage effect mechanisms, respectively (Figure 5B). However, the division of supporting evidence among organismal groups was similar to their division among all eligible studies (Figure 5C).

The storage effect mechanism was best associated with the mutual invasibility tests; 73% of its eligible studies included these tests (column “experimental design” in Supplementary Table S2). In contrast, none of the eligible studies of the FD-exploitation mechanism included mutual invasibility tests, while only two studies exploring the resource-ratio mechanism and only one study exploring the natural enemy partitioning mechanism included these tests (Chung and Rudgers, 2016; Letten et al., 2017; column “mechanism-invasibility” in Supplementary Table S2). Studies exploring the three latter mechanisms sometimes used alternative methods to associate the feature of their mechanisms with coexistence proxies. For example, to explore the effect of FD-exploitation on coexistence, Ishii and Shimada (2012) manipulated the attack rate and correlated it with the time until competitive exclusion. Similarly, in all eligible studies of the resource-ratio mechanism, the outcomes of interspecific competition were experimentally tested with respect to the limiting resources and supply rates in question, and they were compared with the corresponding predictions. Finally, in nine of the 27 eligible studies of the natural enemy partitioning mechanism, either the density-dependent effect was associated with an increased species diversity, the enemy effect was associated with the strength of interspecific competition, or the long-term community response was estimated through model trajectories (column “experimental design” in Supplementary Table S2).

A mechanistic exploration of species coexistence may have overarching implications for our understanding of the structure and function of natural communities. Here, we used the reviewed papers on the five coexistence topics to highlight examples from which insights could be gained by applying a mechanistic approach.

Mutual invasibility tests are derived from the theoretical distinction between stable coexisting and co-occurring species and therefore play an important role in bridging theoretical-experimental gaps. In addition to their direct function in predicting species' long-term coexistence, mutual invasibility tests can improve our ability to determine the spatial scale of coexistence (Hart et al., 2017), assess changes in species dynamics (Stein et al., 2016), and predict where and when biodiversity will persist. Invasibility tests can also be used to investigate evolutionary questions and phenological effects on species coexistence (Hart et al., 2019; Blackford et al., 2020).

Investigating the resource-ratio mechanism adds insights into bottom-up community regulation, highlighting the most important resources that determine the community structure and identifying the species-specific limiting factors. Such information can improve our ability to forecast succession, disturbances, trophic interactions, and global change effects (e.g., Luimstra et al., 2020), and to evaluate the role of resource competition in habitat changes and disease (Bell et al., 1990; Zhang et al., 2015). An exploration of the resource-ratio mechanism can also be used to predict higher-level community patterns, such as the associations between productivity and species diversity (Harpole and Suding, 2011). Insights into top-down community regulation may be drawn from the experimental exploration of the natural enemy partitioning and FD-exploitation mechanisms. A better understanding of the natural enemy partitioning mechanism can improve our ability to predict the changes in community structure and dynamics in the face of specialized pathogens and other invasive enemies and to design potential solutions to regulate them (Liu and Stiling, 2006). A better understanding of the FD-exploitation mechanism can improve our understanding of frequency-dependent evolutionary feedbacks, apparent mutualism and commensalism, and predator coexistence (Gaymer et al., 2001; Clay et al., 2007; Long et al., 2012) and may reveal the causes underlying long-term patterns of population dynamics, such as population cycles (Hanski et al., 1991). Finally, the reviewed papers demonstrate that insights into fluctuation-dependent mechanisms, such as the storage effect, can improve our understanding of community responses to changing conditions, which is fundamental to the study of natural communities. The specific attention given in this mechanism to the potential change in interaction outcomes, under different habitat and temporal conditions, can shed light on various natural phenomena, such as Hutchinson's paradox (Descamps-Julien and Gonzalez, 2005), and human-induced changes, as in the case of invasive species (Epstein et al., 2019).

In contrast to the growing interest in coexistence and the remarkable progress in the theoretical and phenomenological approach (e.g., Levine et al., 2017; Saavedra et al., 2017; Godoy et al., 2018; Ke and Letten, 2018; Ocampo-Ariza et al., 2018; Gallego et al., 2019; Spaak and De Laender, 2020), our review suggests that experimental support for specific coexistence mechanisms lags behind (Figures 1, 2). The mechanistic approach lags behind to such an extent that we do not have sufficient experimental evidence to assess the prevalence of coexistence for the following reasons: First, while coexistence theory suggests that the invasibility criterion is one of the key methods to distinguish coexistence from co-occurrence, our literature search revealed that most of the papers that incorporated the invasibility terms did not include invasibility tests. Second, the theoretical-experimental gaps (e.g., the occurrence of demographic stochasticity and short experimental periods), the scarcity of tests with a complete design that reflects the natural settings (e.g., multiple species and mutual tests), and the study bias toward terrestrial plants (Figure 4B) restrict our ability to quantify the extent of the experimental support for the invasibility criterion in co-occurring species.

Considering the assessment of each of the coexistence mechanisms' prevalence, support for the resource-ratio and storage effect mechanisms was higher than that for the natural enemy partitioning and FD-exploitation mechanisms (Figure 5A). This difference suggests that the prevalence of a mechanism is not related to the nature of the mechanism's dependence on environmental variation. Instead, the results may reflect a tendency toward lower support for mechanisms mediated by enemies (i.e., top-down control) than for mechanisms mediated by resources (i.e., bottom-up control), reinforcing the conclusion that top-down interactions act mainly as fitness equalizing mechanisms (Bartomeus and Godoy, 2018); hence, without mechanisms that enhance the niche differences between competing species, these interactions are less likely to maintain stable coexistence. However, here too, the theoretical-experimental gaps (e.g., restricting the experiment to short periods, not testing core theoretical predictions, and not associating each mechanism's features with demonstrated stabilization), the scarcity of tests with a complete design that reflects the natural setting (e.g., outdoor experiments, field observations, and field manipulations), and the bias of specific mechanisms toward specific study systems (Figure 5C) did not permit a comprehensive analysis.

Similarly, our review suggests that we do not have sufficient experimental evidence to determine the dependence of the mechanisms on the natural history of the competing organisms, partly due to a linkage between the target mechanism and the study system. In particular, the resource-ratio mechanism was supported mainly in plankton, the natural enemy partitioning mechanism in plant-soil microbial systems, the FD-exploitation mechanism in invertebrates, and the storage effect hypothesis in short-lived organisms (Figure 5B). However, the division of supporting evidence among organismal groups was similar to their division among all eligible studies (Figure 5C). Considering that, in general, studies supporting a theory are more likely to be reported than studies with negative results, we may speculate that the mechanism-system linkage reflects a tendency of specific groups to be controlled by particular coexistence mechanisms. However, an inherent system-specific bias precludes such an explanation, and currently, we cannot reject the alternative explanation that the bias had already been introduced at the stage of model system choice. Although our systematic review used general, rather than system-specific, ecological key terms, it revealed that coexistence experiments are limited to a handful of study systems, mostly annual plants, aquatic invertebrates, and plankton (Figure 5C). This bias may reflect both the research interests of the leading research groups and practical methodological difficulties.

Though we intentionally chose to focus the review on a set of well-known mechanisms, our synthesis suggests that the experimental focus is neither uniformly distributed among mechanisms, nor is it determined by their theoretical coverage or the publication of influential papers (Figure 1). For example, over the years, the resource-ratio mechanism has attracted significantly more interest than the FD-exploitation mechanism, but the eligible experimental papers represent only 1 and 2% of the relevant studies of these two mechanisms, respectively. This narrow range of studies, together with methodological variability (e.g., the variation in the percentage use of laboratory experiments) and the tendency of experimental studies to focus on a single mechanism, preclude definitive inferences about the relative importance of the various mechanisms in maintaining species coexistence in nature.

Reviewing the above five topics allowed us to identify three main reasons for the current lack of coexistence experiments. Below, we discuss each of these reasons.

Below, we offer strategies to overcome the above-mentioned interdisciplinary, accessibility, and theoretical challenges in future work.