A literature review was conducted to identify publications describing variability analyses for in vivo toxicology studies. Initially, a literature search was conducted using PubMed (including MEDLINE) and Causaly databases. Medical Subject Heading (MeSH) terms and text words included “variability”, “reproducibility” and their variants combined with “in vivo”, “animal studies”, “experimental studies” and other relevant alternatives. The literature search focused on publications from 1990 to April 2023 (the date when the original search was conducted). This broad search, without restriction to toxicology, TG, or type of variability/reproducibility analysis yielded approximately 8,500 publications, confirming that evaluation of variability in the life sciences is a widely addressed topic.

Limiting to only publications containing variability analyses, systematic reviews, and meta-analyses of in vivo studies dramatically reduced the literature set. Subsequent manual screening, first by title and abstract, then subsequently by reviewing full text for a prioritized subset, was conducted to exclude non-relevant articles and adhere to the defined scope. This tiered manual screening resulted in the identification of approximately 100 relevant manuscripts; this corpus of literature was reviewed for data extraction and inclusion based on either quantitative analyses or relevant discussion addressing sources of variability in bioassays. This provided a set of about 60 manuscripts that included state-of-the-science reports containing opinions about scientific reproducibility of mammalian in vivo studies (not necessarily toxicology) and background on potential sources of variability that were used to inform our discussion.

From this body of literature, we selected toxicologically relevant study evaluations, specifically summaries of studies following human health-relevant TGs (from OECD and EPA Series 870 Health Effects). These guideline-like studies adhered to a set of criteria that were either aligned with existing regulatory guidelines or followed a standardized protocol used in regulatory assessment. We prioritized evaluations of data from mammalian studies, including analyses of variability from in vivo studies not specific to toxicology. The final number of publications yielding quantitative variability analyses that were useful for understanding replicability of relevant toxicological studies (e.g., TG or guideline-like) was 27. This subset was identified by excluding studies such as Ames assays that did not directly measure in vivo outcomes and endpoints such as metabolomics and vaccine evaluation that were considered too complex to evaluate for replicability.

Within our literature set, some reports evaluated variability in studies that were generally similar but did have minor differences (e.g., used different dosing vehicles). For those, we evaluated replicability among all studies, regardless of minor study differences. This was done for consistency, as not all analyses made such distinctions to account for study design. In studies that included multiple analyses, the most broadly representative summary metric was retrieved. For example, when respective analyses were conducted for males only, for females only, and for all animals, we considered only the evaluation that included all animals. Thus, our data compilation contains disparate data formats not intended to draw comparisons across study types; the intention of this variability review was to compile data and present reference values that can serve as a resource for better understanding in vivo toxicological studies.

Summaries of reported replicability were compiled for both categorical and quantitative endpoints. Variability analyses that considered categorical classification schemes were limited to those using the EPA and the United Nations Globally Harmonized System of Classification and Labeling of Chemicals (GHS) schemes. The replicability of European-specific classification, labeling, and packaging (CLP) criteria was not included in our review. Categorical replicability was generally reported in the literature set in the form of conditional probabilities. A conditional probability represents the probability of a chemical being assigned to a category given its prior categorization, with consideration to the number of studies with which a chemical was categorized. Calculations were performed as previously described (Karmaus et al., 2022; Luechtefeld et al., 2016).

Replicability of continuous quantitative endpoint values was reported as described by the primary literature source. We considered statistical metrics that conveyed some aspect of variance in replicate studies or the variance in replicate studies explained by study metadata (e.g., standard deviation [SD], coefficient of variation [CV], coefficient of determination [R ²], and root mean squared error [RMSE]; see Table 2).

Categorical replicability analyses focus on study types with a categorical interpretation, which can be either a binary outcome of positive or negative or assignment to a hazard classification. Study types with binary classification endpoints included in our review are endocrine assays (Hershberger and uterotrophic), genotoxicity studies, carcinogenicity studies, and DNT studies (Table 3). The Hershberger and uterotrophic assays gave similar degrees of replicability: 72% and 74%, respectively (Kleinstreuer et al., 2016; Browne et al., 2015). For carcinogenicity, the two prior variability evaluations yielded different outcomes: 65%–86% replicability when comparing between sexes of a single species, or 36%–74% replicability between different species (rat and mouse) (Huff et al., 1991; Haseman and Lockhart, 1993).

More complex categorical analyses were conducted for studies using multicategory hazard classification schema to assign chemicals categories based on quantitative or qualitative study results. As noted above, our evaluation of this aspect of replicability was limited to studies using the GHS (the most common classification scheme for chemical hazard categorization) and the EPA categorization scheme (specific to endpoints of interest for the EPA). These systems have different category cutoffs and varying numbers of categories.

Multicategorical replicability was evaluated using conditional probabilities where possible, gathered from retrospective variability analyses or calculated based on available study information. As a point of reference, assignment of categories by random chance would equate to 50% replicability for a binary categorization scheme or 33% and 25% for systems of three and four categories, respectively. Endpoints with such data included ocular irritation/corrosion, dermal sensitization, dermal irritation/corrosion, rat acute oral lethality, and rat acute inhalation lethality (Table 3). Replicability was generally higher for categorization systems with fewer categories. For genotoxicity studies, the replicability of three categories (positive, negative, or ambiguous outcomes) varied widely, ranging between 23% and 78%, depending on the TG and its protocol variants (Raitano et al., 2026). It was rare to observe replicability above 75% across many study and endpoint types, especially for complex or targeted endpoints such as organ-specific effects. Depending on how replicate studies were aggregated across study type and species, the concordance of any target organ effects for subchronic and chronic repeated dose studies ranged from 38.5% to 90% per organ and the frequency of positive findings (Hoffmann et al., 2005).

For dermal and ocular study types, increased replicability was observed for categories representing the lowest and highest toxicities, suggesting robust results when toxicity is either absent or overt. This is most notable in the Draize rabbit eye irritation test for which the GHS Not Categorized classification was replicated as much as 100%, while the GHS Category 2B classification, representing mild irritation, had a conditional probability of only 16% replicability (Luechtefeld et al., 2016; Barroso et al., 2017). Given the low replicability for such mid-categories (e.g., GHS categories representing hazard of mild to moderate ocular irritation), subsequent studies were conducted to evaluate the impact of combining these categories to determine whether the replication improved with grouping. When authors collapsed GHS Categories 2A and 2B for ocular irritation, replicability was improved to as high as 71% or 88% in two independently conducted analyses (Barroso et al., 2017; Adriaens et al., 2014). Two evaluations of dermal sensitization data conflicted with the increased replicability seen for fewer categories (Dumont et al., 2016; Hoffmann et al., 2005). One study that considered reproducibility of three GHS classifications found Not Classified outcomes to have 52% replicability, while another study that considered five classifications found these outcomes to have 80% replicability. The discrepancy may be explained based on the analysis approach used: the study finding 52% replicability was based on any chemical for which two or more studies were available, and all categories were assigned in a normalized approach with equal study weights. The study finding 80% replicability, in comparison, was found for chemicals with three or more studies and a single categorization determined based on the majority outcome. These findings underscore the importance of methods and review assumptions as well as curation protocols for retrospective analyses.

The replicability of continuous numeric endpoints was reported using different quantitative approaches. Retrospective consideration of all of these disparate reported metrics is particularly important because most of these studies were not subject to replicability analysis for a range of chemicals prior to acceptance of the relevant TGs. Variability among quantitative endpoints was evaluated for ocular irritation, dermal sensitization, acute oral lethality, subchronic/chronic repeated dose toxicity, and carcinogenicity studies (Table 4). For acute lethality, variability analyses were available for both rat and mouse studies. Studies using either species showed similar degrees of variability when evaluated by the same authors (Hoffmann et al., 2010). Where possible, CVs were provided, but these were not available for all study outcomes due to a lack of reported data. Other quantitative metrics describing dispersion of the data included SD and a margin of uncertainty based on the median absolute deviation (MAD). Some variability analyses constructed models using study metadata to quantify the variance in replicate studies; these studies reported RMSE as a measure of spread of the predicted values from the true values, and R ² as the amount of variance in the data explained by the model. When the model mean prediction is the same as the data mean, the RMSE is equal to the SD of the residuals. We have reported SD, margin of uncertainty, and RMSE in the same units as the data, whereas R ² is expressed as a proportion of the variance explained by the model or variable (Table 4). Importantly, maximal R ² attainable by using curated study metadata to explain variance is limited by unexplained variance due to undocumented parameters (i.e., experimental factors that were not collected with the study or not curated consistently) or inherent biological variance (Ly Pham et al., 2020; Kleinstreuer et al., 2016; Dumont et al., 2016; Paul Friedman et al., 2023; Lubet et al., 2018; Ashby, 2002; Crofton et al., 1991).

Although these different metrics measure different statistical observations of the data or models of the data, examining the set of statistical metrics revealed multiple high-level findings. Inter- and intralaboratory CV values were available for ocular irritation with comparable CV values ranging between 40% and 60% (Weil and Scala, 1971; Earl et al., 1997; Cormier et al., 1996; Blein et al., 1991). Single-dose acute studies demonstrated less dispersion of effect-level values, likely the product of study design (e.g., limit tests at 2000 mg/kg), with the margin of uncertainty equal to 0.25 log10-mg/kg in one evaluation (Karmaus et al., 2022) and the SD falling below 0.42 log10-mg/kg for most studies in another evaluation (Hoffmann et al., 2010). Repeat-dose studies demonstrated greater dispersion of replicate values that typically approached ±0.5 log10-mg/kg/day, depending on how the dispersion was quantified. The linear correlation between 50% tumorigenic doses between two separate sources of carcinogenicity studies showed an R ² of 0.63 (Cormier et al., 1996), suggesting major differences in study conduct, curation, and/or biological observations. In the aggregate, estimates of variance suggested large amounts of spread in replicate study data for quantitative endpoints. Linear correlation of values related to carcinogenicity (Gottmann et al., 2001) and the amount of variance in replicate oral repeat-dose study toxicity values explained by multilinear regression modeling (Ly Pham et al., 2020; Paul Friedman et al., 2023) suggest some upper bound exists in the amount of variability in replicate toxicity values that can be explained by study metadata, which likely approaches 60%–70%.

One evaluation of replicability for behavioral endpoints assessed as part of neurotoxicity studies indicated good replicability for motor activity studies, with a lowest-observed-effect level maximum-to-minimum ratio range of one to six for nine positive control compounds tested within six laboratories (Crofton et al., 1991). Replicability of DNT motor activity was reported for negative controls showing CVs ranging from 20% to 140% (Moser et al., 2016). Despite the broad variability, these are notable data as negative controls are rarely reported. No other studies were available that could provide replicability estimates for this or other behavioral methods. Several abstracts presented at Society of Toxicology Annual Meetings between 2001 and 2005 (which were not peer-reviewed) indicated incomplete reporting and incomplete positive control data in evaluations of startle response data (Sette et al., 2004) and learning and memory tests (Raffaele et al., 2004), such that the considerable within- and between-lab variability could not be adequately analyzed for these endpoints. One of these, a 2004 evaluation of motor activity measurements, indicated that, “Further consideration of how to reduce variability … is warranted” (Raffaele et al., 2004). A qualitative retrospective analysis of DNT studies reported that few laboratories (3/16) provided usable positive control data (Crofton et al., 2004), with studies lacking such data thus being uninformative for replicability estimates. However, negative control CV values for parameters of motor activity or startle response may provide some indication of the replicability of study results. Typically, CV values for such readouts range between 20% and more than 100%, depending on the laboratory, test conditions, and animal age (Moser et al., 2016). These high and variable CV values indicate that significant differences between control and dose groups may be recognized in some labs and test conditions but not in others. In contrast, as reported in a 2001 Society of Toxicology Annual Meeting presentation, CVs among brain weights and brain morphometrics were reported to be significantly lower, ranging from 4% to 12% and 5%–30%, respectively (Crofton, 2001).

It is important to distinguish between quantified variability, sources of variability, and uncertainty about variability. For evaluating traditional in vivo toxicological assays, we must consider the effects of study conduct on both derivation of quantitative points-of-departure (hazard characterization) and interpretation frameworks (hazard identification, e.g., classification) for replicability. It is also important to note that different group size and statistical power between studies may explain limited replicability of study outcomes. Previous investigations have revealed that variability can stem from numerous sources related to either the assay protocol or study design variations, including elements such as animal strain, diet, and vehicle used (Lubet et al., 2018; Ashby, 2002). For example, a review focusing on variability in genotoxicity studies conducted a multivariate analysis to identify drivers of variability for both OECD TGs 474 and 475 across 31 chemicals with replicate data identifying strain and species as having the greatest contribution to variability (Raitano et al., 2026). Administration via injection as compared to the oral route has also been shown to increase the likelihood of a positive response in uterotrophic assays (Kleinstreuer et al., 2016), and lower variability is observed among studies that use the same vehicle (Dumont et al., 2016). Study parameters that can be customized such as species, dose spacing, and substance purity were found to contribute to more than half the total variance in organ-level lowest-effect levels (LELs) (Paul Friedman et al., 2023). This is particularly well-characterized for underlying physiological and metabolic differences across species, strain, and sex. For example, tumor incidence in the rat reproductive tract (You et al., 2002; Buelke-Sam et al., 1998) or the presence of thyroid tumors in male rats vs. female rats (Coperchini et al., 2025). In addition, one report indicated that endpoint selection in the Draize rabbit eye irritation test had a significant impact on the degree of variability: higher variability was seen when a GHS Category 2 classification was made based on conjunctiva effects without corneal involvement as compared to classification based on other drivers such as iritis (Adriaens et al., 2014). Protocol timepoints, such as the age of the animal, the timepoint following exposure, or timing of the response measurement, can also significantly impact study outcome (Lubet et al., 2018; Ashby, 2002). We must acknowledge that TGs are not strict standard operating procedures, recognizing that some flexibility in the study design is acceptable and within the scope of “guideline-like” study conduct. However, even where the source of the variability may be explained, the data variability may remain an uncertainty since it is difficult to understand which protocol variant is most relevant for the human population.

Addressing inherent biological variability is even more complex, but can be considered to be derived from four general factors: physiological, genetic, ontogenetic, and exposomic (Kreutz et al., 2024). These sources of variability have been quantified to some extent but require further characterization. For example, with regard to chemical toxicokinetics, studies have found that variability estimates usually, but not always, fall within the acceptable 10X uncertainty factor for some chemicals and lifestages, with many uncertainties remaining (Quignot et al., 2021; Dorne et al., 2001; Kasteel et al., 2020; Darney et al., 2020; Di Consiglio et al., 2021; Wetmore et al., 2015; Strikwold et al., 2017). There are also uncertainties regarding the generalizability of animal studies to human and ecosystem outcomes due to differences such as species, exposures, and endpoints. Both animal models and in vitro NAMs present challenges for generalizability of model outputs to humans, with neither paradigm fully representing the toxicokinetic and toxicodynamic processes present in humans. Rodent-based studies must be extrapolated to human equivalent doses for human health risk assessment using a set of assumptions regarding dosimetry, toxicokinetics, and applicability of targets in rodents for targets in humans. Human-based NAMs are typically isolated cell-based or protein-based assays that require extrapolation to the human body, including toxicokinetic and metabolic considerations, such as tissue-specific metabolism.

Notable data gaps in our retrospective analysis include reproductive and developmental toxicity study types and toxicokinetic studies as well as variability among controls for these studies, which are rarely reported. We also note that the different analyses summarized did not all represent unique data sets, as some of the larger replicability evaluations mined the same source databases, resulting in overlapping representation of primary data. However, as our goal was not to directly compare each variability study, we felt that some redundancy was acceptable as we sought to more comprehensively characterize variability. More specifically, even variability estimates reported from the same datasets may differ due to the filtering steps and statistical approaches applied. One such case is for the vehicle used: while some analyses provided separate estimates of variability for the same vehicle (Adriaens et al., 2014), most did not consider vehicle as a factor in their analyses.

Results summarized here are likely not unique for mammalian toxicology and are expected to be consistent across ecotoxicology studies as well. For example, 50% lethality or moribundity concentrations (EC50s) in acute fish toxicity studies performed according to OECD TG 203 may vary over several orders of magnitude, with variation only partly accounted for by the use of the 11 different fish species allowed in the TG. Similar levels of variability are observed in EC50 data from acute daphnid studies conducted according to TG 202 (Schür et al., 2025). Extending this review to other such endpoints could prove useful to better characterize current toxicological test methods. However, it is necessary to acknowledge that performing literature searches and compiling comprehensive, harmonized, and robust datasets required for retrospective analyses is a laborious task that will never capture all existing data and important metadata.

To more readily access individual or collated study data for conducting retrospective evaluations, databases of in vivo results have been developed and are routinely being updated. Some of these from the United States include the EPA’s ToxRefDB (Feshuk et al., 2023a) and ToxValDB (Wall et al., 2025), the National Toxicology Program’s Chemical Effects in Biological Systems database (CEBS), and the National Institutes of Health’s Integrated Chemical Environment (https://ice.ntp.niehs.nih.gov/). Additional resources from Europe include the European Chemical Agency’s (ECHA) IUCLID database (https://iuclid6.echa.europa.eu/) and the European Food Safety Authority’s (EFSA) Open FoodTox (https://www.efsa.europa.eu/en/microstrategy/openfoodtox). While each of these databases include summary endpoint metrics per chemical (e.g., LEL, NOAEL, LD50, etc.), a few of the resources (namely, ToxRefDB and CEBS) also have prioritized supporting detailed concentration-response data (e.g., response values per testing concentration) which can further support more granular retrospective evaluations. Of course, each of these databases has varying numbers of chemicals and studies, and compilation of a dataset for robust retrospective evaluation requires careful characterization and curation. Future retrospectives using data from such databases should be performed to gain further insights into reproducibility of study types or routes of administration that we were unable to consider here, e.g., reproductive and developmental, inhalation, or dermal repeat-dose toxicity studies. Investigation of whether reproducibility has improved over time, e.g., due to more stringent GLP requirements and given TG revisions, could also be considered. The increasing requirements for findability, accessibility, interoperability, and reusability of data (so-called “FAIR” principles) should help to facilitate future evaluations of study variability.

Both traditional animal models and human-based NAMs can be used to derive protective points-of-departure for human health risk assessment. In recent work examining pharmaceuticals, the difference between preclinical animal and NAM-based points-of-departure in predicting doses at which human toxicity was observed were compared (Weitekamp et al., 2025). NAM-based values were consistently lower than rodent-based values. For values from rodent studies converted to a human equivalent dose to be protective of human adverse effect levels for at least 95% of pharmaceuticals in the dataset they needed to be divided by a composite factor of at least 100. However, for human NAM-based values converted to human equivalent doses to provide a similar level of protectivity, they needed to be divided by a factor of only 10. Further work is required to integrate uncertainties and variability to account for biological factors that are critical for understanding how NAM-based values can inform toxicity values when compared to traditional methods.

It is possible that study variability is accounted for in the conservatism of uncertainty factors applied when deriving toxicity values, such as for extrapolation from subchronic to chronic values or database uncertainty. That said, it should be emphasized that study variability, and the specific uncertainty introduced by study variability, is not typically directly recognized. In recent work to rapidly derive database-calibrated oral toxicity values, study variability is included in quantification of uncertainty in deriving these toxicity values (Aurisano et al., 2023; Harrill et al., 2026). In many regulatory paradigms, regulators must develop a weight-of-evidence strategy for decision-making, and this can include review of databases with heterogenous data and grouping or read-across approaches for the reuse of data from similar compounds. Therefore, the gross estimates for in vivo data variability provided in this review may be useful when considering current regulatory practice. Next-generation risk assessments focus on establishing scientific confidence through a weight-of-evidence approach that considers reproducibility and technical characterization in addition to biological relevance, mechanistic understanding, and fitness for purpose (van der Zalm et al., 2022; ICCVAM, 2024). Such considerations support innovation using modern science and facilitate regulatory acceptance of NAMs in safety assessments that ensure the protection of human health and the environment. However, building confidence in new test systems requires demonstration of robust, consistent, and interpretable outcomes. One suggested first step, which is already frequently being performed, would be to confirm assay performance in terms of replicability and minimal variability, relative to animal TG data variability. For example, variability can be more tightly controlled through rigorous and well-defined assay development frameworks for NAMs (ICCVAM, 2024). In many cases, studies involving NAMs include much higher replicate counts and internal positive controls. In addition, reporting of variability measures is becoming a common practice. Additional databases of in vitro data including PubChem (Kim et al., 2025) and the EPA’s invitrodb (Feshuk et al., 2023b) contain some performance metrics and individual replicate data that can be leveraged for further retrospective evaluations of NAM variability. Such analyses of both biological and technical replicability can of course be included among the metrics for consideration of a NAM as “better than” traditional in vivo approaches. Building additional validation approaches beyond those that rely solely on direct comparisons to legacy animal tests and qualifying expectations of human-based NAMs to recapitulate rodent findings reflects a growing recognition that fundamentally different principles should be considered when evaluating NAMs.