The sample size can influence precision: a sample size that is too small can underpower the study and negatively affect the validity and reproducibility of the findings, while an oversized study uses more resources than necessary, and exposes more subjects to a potentially harmful experience (98).

In microbiome studies, additional considerations might be applicable to how we estimate the sample size because microbiome datasets have intrinsic features that do not apply to traditional studies. A key challenge lies in defining biologically meaningful effect sizes for the microbial community changes or individual taxa differences we aim to detect, which complicates power analysis and makes it a main barrier to robust study design in microbiome research (99–101). For ocular surface microbiome research, dealing with the low abundance microbiome necessitates exceeding the calculated estimates to account for individual differences, technical variability, and subgroup analyses. Furthermore, the low biomass of the ocular surface can lead to a higher proportion of samples with insufficient DNA yield for downstream analyses. This “sample loss” needs to be accounted for when determining the initial sample size for a study.

Sample size calculations, or the lack thereof, were explicitly reported in only 17 (19.1%) of the articles reviewed. Among the 89 papers, 13 (14.6%) described their sample size calculation approach. For instance, four studies utilized Micropower, an R package developed by Kelly et al. (100), to estimate power and pairwise sample sizes based on permutational multivariate analysis of variance (PERMANOVA). These studies determined that low-abundance 16S rRNA datasets required a minimum sample size of 30 to achieve a discriminant power of 0.8 with a significance level of 0.05 for detecting statistically significant differences in alpha or beta diversity of the microbiome composition between groups (39, 64, 80, 92). Other tools used for sample size calculation included StateMate Software version 2.0 (GraphPad Inc., San Diego, CA) (15), Easy R (EZR, version 1.41) (35), and G*Power software (version 3.1.9.7) (70) which calculated a required sample size of 42 patients and 64 controls, based on an effect size of 0.5 (with no reference to a specific metric), alpha error of 0.05, and power of 0.80. Additionally, two studies employed species accumulation curves from prior ocular microbiome studies, determining that 20–45 samples were sufficient to detect >80-85% of microbial species (85, 95). Population proportion methods were also used, with one study deriving a sample size of 21 based on a 6% margin of error and 90% confidence interval to compare abundance of pathogenic bacteria among three groups, while another study, which compared the microbial composition among four groups, calculated a sample size of 20 eyes per group using a formula comparing two independent means and standard deviations from a previous study (55, 75).

Reasons for omitting sample size calculations included the pilot nature of the study, which aimed to provide preliminary data for future research, or the descriptive nature of the study, which lacked a formal null hypothesis (41, 43, 71, 83). Notably, 73 articles (82.0%) did not perform sample size calculations or provide a justification for their chosen sample size. Nonetheless, many of these studies acknowledged their small sample size as a limitation, highlighting this as a recurring challenge in the field (12, 28, 35–37, 43, 46, 47, 50, 57, 61, 65, 78, 84, 89, 90, 93).

The total number of participants enrolled in each study ranged between 4 and 260 human subjects (median = 48), and the sample size for groups (e.g. in case-control studies) ranged between 4 and 137 (median = 22). Figure 2 illustrates the sample size variability across these 89 articles where the average group size was used for multigroup studies. Of the 71 (79.8%) studies that enrolled two or more groups of participants, 67 enrolled healthy normal individuals as the control group. The case and control groups were the same size in 15 of these studies. In 26 studies, the sample size of the control group was 2% to 339% larger than that of the cases. In the remaining 26 studies, the sample size of the normal control group was 3% to 76% smaller than that of the cases. Nonetheless, comparison of the sample sizes between case and control groups showed no statistically significant difference (paired t-test p = 0.268).

While the choice of left or right eye for sampling might seem inconsequential, the decision to sample unilaterally or bilaterally has significant implications for study design and analysis. This distinction is crucial because most statistical methods assume that samples are independent. If these assumptions are violated, such as when analyzing data from both eyes of the same individual, the calculated statistics can be misleading, potentially leading to inflated Type I error rates (false positives) or decreased statistical power (102, 103).

In this review, the choice of unilateral or bilateral sampling approach was either explicitly stated or could be deduced in 70 (78.7%) of the articles (Figure 3). Of these, 36 studies performed sampling from only one eye; this was either a randomly-chosen eye (n = 17), the right eye (n =6), the left eye (n = 1), the eye receiving treatment (n = 5), or the selection was not specified (n = 7). In 34 studies, samples were obtained from both eyes of each participating individual; these were used for performing fellow eye comparisons (n = 12), counted as independent samples to increase the overall sample size (n = 11), pooled as one sample (n =8), collected separately to perform downstream analysis with the one generating a higher DNA yield (n = 1), or it was unclear whether they were treated independently or pooled (n = 2). In the remaining 19 articles (16.9%), it was not clear whether sampling was done unilaterally or bilaterally.

For the gut (104, 105), skin (106), and respiratory system (107) microbiome, studies have observed significant seasonal variations influenced by factors such as diet, lifestyle and behavior changes (e.g. stress, sleep patterns, and indoor/outdoor physical activity), and environmental conditions. Similarly, the ocular surface microbiome is a dynamic ecosystem which can be influenced by internal and external factors over time. Diurnal variations, such as eyelid closure during sleep, tear film dynamics, and exposure to environmental factors (e.g., pollutants, allergens) may significantly impact the microbiome throughout the day. Seasonal changes in environmental factors (e.g., humidity, temperature, pollen levels) can also affect the ocular microbiome. Furthermore, host circadian rhythms can influence tear production, immune function, and potentially the composition of the ocular microbiome. Thus, sampling time is an important factor in ocular microbiome research, and its clear reporting is essential for analyzing how the microbiome composition and function change over different time scales, controlling for the potential influence of time variations, and making valid comparisons between studies.

The study time frame was specified as the start and end month in 45 articles (50.6%), 3 articles reported the duration, and 3 reported the years in which the study was done. In the 48 studies that provided information about the study duration, the sampling periods ranged from 2 weeks to 28 months (median = 4.5 months) (Figure 4). Although very few of the studies were longitudinal, 10 studies had sampling periods between 3 and 6 months, 11 were conducted over 6 to 12 months, and 10 had a time frame longer than 12 months. The remaining 41 studies (46.1%) did not provide information about the study duration, suggesting a cross-sectional study design at an undefined point in time. Moreover, the sampling time of the day was specified in only four articles (4.5%), all of which stated that samples were collected in the morning.

Recruitment and enrollment criteria are foundational to study design and analysis. They define the study population, and they are essential to control bias, enhance validity, and ensure generalizability. Common inclusion criteria, like age ranges, separate distinct groups (e.g., adults vs. children) with potentially significant differences in the study variable. When recruiting patients with a disease condition of interest, such as diabetes or dry eye, the inclusion criteria are often based on established diagnostic definitions to focus the research on a well-defined population. For example, the Ocular Surface Disease Index is used to include patients with dry eye disease (31), or to rule out the condition from the normal controls (19, 43). Exclusion criteria, on the other hand, serve to remove individuals with factors that could distort the results. This might involve excluding participants with pre-existing conditions that could interact with the studied intervention or those taking medications that might bias the findings.

Among the reviewed studies, 63 (70.8%) applied convenience sampling and recruited their cases from clinics and hospitals. Specifically, participants were recruited from those visiting clinics and hospitals for an eye exam (n = 52), those undergoing surgery (n = 9), hospitalized patients (n = 1), and medical staff (n = 1). Additionally, some studies recruited participants from the general population, either through community cohorts (n = 4) or community screening programs (n = 1), as well as through internet surveys and advertisements (n = 2). A few studies used a combination of recruitment methods, such as recruiting from both inpatients and outpatient, medical school students, and retirement communities (n = 3). The recruitment pool was not mentioned or unclear in the remaining 16 articles.

Exclusion criteria were provided in 82 (92.1%) of the reviewed articles. The most common exclusion criteria were use of systemic and/or topical antibiotics (n = 54), contact lens wear (n = 48), ocular (or ocular surface) disease and/or infection (n = 47), systemic disease (n = 40), use of ocular medications (n = 37), history of ocular surgery (n = 43) or trauma (n = 23), pregnancy (n = 17) and/or lactating (n = 11), use of corticosteroids or immunosuppressants (n = 20). Some articles also included skin disease and/or infection (n = 4), tobacco use (n = 9), ocular allergies (n = 3), environmental considerations such as being a healthcare worker or having spent a significant time in a hospital (n = 2), mental illness (n = 1), intravenous drug injection (n = 1), and certain dietary habits (vegetarian or vegan) (n = 1).

In microbiome research, adhering to aseptic techniques is crucial to minimize contamination, especially when dealing with low microbial biomass, such as the ocular surface microbiome (108). Sampling should ideally be conducted in a clean or sterile environment, to reduce the introduction of environmental contaminants. The inclusion of negative controls - such as unused sampling tools exposed to the same collection environment - help identify and account for potential contamination introduced during sampling, or during manufacturing, packaging, or handling.

The sample collection environment could be determined in only 25 (28.1%) of the articles. The environments described included ophthalmic treatment rooms sterilized with ultraviolet light (n = 8), clean rooms (n = 7), sterile operating rooms (n = 2), a clean prior fumigated room (n = 1), and rooms with appropriate lighting, temperature, and humidity (n = 2). Additionally, two studies performed sample collection in the operating room during surgery, one in a lab or ophthalmology office, one in the field, and one where participants collected their own samples at home. In the remaining 64 (71.2%) articles, the sample collection environment was not mentioned.

The inclusion of unused collection tools (e.g. swabs) as negative environmental controls was mentioned in only 42 studies (47.2%), including 17 of the 25 studies that had a description of their sample collection environment. However, reports of the results with these negative controls were found in only 10 articles. Some carried out the analyses only up to the DNA extraction stage (n = 3). An example is the study by Ham et al. (24) who reported that their unused dry cotton swabs yielded no significant amount of DNA (below 0.05 ng DNA μL-1) and thus, the samples were not amplified. Similarly, Liang et al. (50) found no bacterial DNA in the negative control swabs, although it is not clear how DNA was quantified. In some cases, the researchers ruled out contamination by checking PCR amplification results in their negative controls (n = 5). One example is the study by Graham et al. (10) who confirmed that the conventional broad-range 16S rDNA PCR products of their negative controls produced no visible bands on gel electrophoresis. In the remaining two studies, full analysis was conducted on the negative controls, and the operational taxonomic units (OTUs) or amplicon sequence variants (ASVs) found in these controls were removed from the samples for contaminant filtering. On the other end of the spectrum, Zhou et al. (14) did their sample collection in the field, and although their methods section does not mention collecting negative controls during sampling, their results indicate running some through the complete set of analyses and finding Ralstonia to be the major taxon in their negative controls, and thus, they were not able to confirm whether its presence in 80% of the participant samples was due to contamination or part of the ocular flora.

Sample collection from the ocular surface can be a challenge because the eye is a very sensitive organ and shows strong reflexes such as blinking and tear secretion in response to external stimuli such as dust and light. With a nerve density of 300–400 times greater than that of the skin, the cornea is one of the most sensitive tissues in the human body (109). As such, instilling proparacaine hydrochloride 0.5% or tetracaine hydrochloride 0.5% ophthalmic drops is a common technique for achieving topical anesthesia before performing ophthalmic procedures, even non-surgical ones (110–112). However, the antibacterial effects of proparacaine have long been known using culture techniques (113, 114). Using culture-independent techniques, Shin et al. (18) observed that using proparacaine before sampling leads to lower diversity levels, while Delbeke et al. (65) who conducted a contralateral control study, found that it specifically effects Cutibacterium.

As illustrated in Figure 5, information regarding the use of topical anesthetics prior to sampling was available in 60 (67.4%) studies; in 39 studies anesthetics were used, while in 21 studies, the authors explicitly stated that no anesthetics were administered. Proparacaine was the most frequently administered topical anesthetic (n=13), followed by tetracaine (n=10), oxybuprocaine (n=9), and Alcaine (n = 2). In the remaining five studies, sample collection was done after instilling anesthetics, but the actual preparation was not stated. In the study by Cavuoto et al. (22), sampling for the children was performed under general anesthesia, although the use of anesthetic agent is not mentioned for their adult participants. For the remaining 28 studies (31.4%), it was not clear whether anesthetics were administered before ocular surface sampling.

Research on the effect of collection tools and sampling approaches in ocular surface microbiome is sparse. Herzog et al. (115) compared standard cotton versus flocked nylon swabs and observed that the choice can influence results. Büttner et al. (116) reported that pre-moistened swabs increase the rate of Gram-positive bacterial detection in their study of feline conjunctival microbiota. Ozkan et al. (117) suggest there can be significant differences in bacterial richness, diversity, and composition between tissue samples and swab samples. Comparing 12 different ocular regions using culture-based methods, Fahmy et al. (118) found that Staphylococcus albus and corynebacteria were less likely to be recovered from the bulbar conjunctiva. In terms of the pressure applied while swabbing, Dong et al. (11) suggested that deep (more pressure) sampling leads to a higher abundance of Proteobacteria, while soft sampling was associated with Firmicutes and Actinobacteria.

In the reviewed articles, a variety of tools were employed for sample collection. The most common tool was sterile swabs (n = 70) which varied by material (cotton, nylon, or flocked), tip fiber arrangement (spun or flocked), and preparation (dry, pre-moistened, or soaked) (Table 1). Less commonly used collection tools included paper (n = 5), membrane (n = 5), pipette with 1.0 ml of saline rinse (n = 1), and capillary tube (n = 2). One of the latter cases is the study by Pal et al. (72) who suggested that tear samples collected with capillary tubes can produce results comparable to conjunctival swabs. Three studies compared various tools, including swabs. For example, Katzka et al. (47) compared conjunctival swabbing using three different materials: calcium alginate, cotton, and Weck-Cel cellulose. They found that calcium alginate swabs yielded results most similar to corneal epithelial biopsies, while Weck-Cel cellulose sponges differed the most. Chen et al. (80) reported differences in alpha diversity and microbial composition between pre-moistened cotton swabs and tear test paper strips. In the remaining articles, tissue was collected during surgery (n = 1), saline eye wash was collected in a tube (n = 1), or the tissue extraction tool/method was not described (n = 1).

Of the 73 studies that used swabs, 58 provided descriptions of their various sampling techniques. The swabbed conjunctival regions included the general conjunctival sac surfaces (n = 19), or explicitly the palpebral conjunctiva (n = 2), palpebral conjunctiva and fornix (n = 9), the bulbar conjunctiva and fornix (n = 5), just the fornix (n = 15), or just the bulbar conjunctiva (n = 4); both the superior and inferior conjunctival surfaces (n = 22) or just the inferior conjunctiva (n = 40), and the choice to include the caruncle (n = 8) or the lid margin (n = 2). Descriptions also included the number of times the swab was swept across the conjunctival surface (n = 19), the level of pressure exerted during swabbing (n = 7), the direction of swabbing (n = 12), and precautions to avoid contamination or contact with other regions (n = 4).

In human fecal microbiome, the choice of the DNA extraction methods has been found to have a large impact on the results of metagenomic analyses, due to variations in cell lysis efficiency and the introduction of biases (123–126). These factors, along with DNA contamination and the presence of inhibitors, are important considerations in any microbiome study (127).

However, ocular surface microbiome research presents unique challenges. The inherently low bacterial biomass of ocular samples often results in low DNA yields, exacerbating the problems caused by inefficient lysis. Furthermore, ocular samples may contain a high proportion of host DNA, and the prevalence of Gram-positive bacteria with their robust cell walls necessitates effective lysis strategies. Another consideration is components of the tear film or other ocular secretions that may act as PCR inhibitors. Given these challenges, DNA extraction methods for ocular samples require optimization to maximize bacterial DNA yield, minimize host DNA contamination, and effectively lyse both Gram-negative and Gram-positive bacteria while removing potential inhibitors, ensuring accurate microbiome characterization.

Various DNA extraction methods were utilized in the reviewed papers. In 78 studies (87.6%), a commercial kit (Table 2); of these, 52 stated the extraction was done in accordance with the manufacturer’s instructions, and 11 applied modifications such as adding a lysis step (n = 7), RNA digestion step (n = 2), or optimization for high throughput (n = 1). Five studies described their custom/homemade protocols, and in the remaining 6 articles, the DNA extraction method was not mentioned or not clear.

In microbiome research, the quantity and quality of DNA is typically checked at certain key points: 1) after DNA extraction to ensure that the DNA is suitable for downstream applications like PCR and sequencing; 2) during library construction to ensure that the correct amount of DNA is used for creating the libraries; and 3) after library preparation to verify the concentration of the final libraries and prevent issues related to overloading or underloading the sequencer, which can affect the overall data quality. While a successful study may not require very high yields, the extracted DNA must be of sufficient quality and quantity for downstream analysis to be informative.

The consensus in ocular surface microbiome research is that samples are low biomass, and transparent reporting of DNA yield and quality metrics is crucial for evaluating the success of DNA extraction and interpreting microbiome analysis results. This information can also help develop threshold criteria for sample exclusion based on DNA yield and quality. Additionally, the methods used for quantification, such as UV-Vis spectrophotometry and fluorescence-based assays, can influence the perceived quality and quantity of DNA. UV-Vis spectrophotometry, while useful for assessing purity, can overestimate DNA concentration due to the presence of contaminants. Fluorescence-based methods, such as Qubit, are more sensitive and specific for double-stranded DNA, providing more accurate quantification (128, 129).

The methods in 48 papers (53.9%) mentioned checking the quantity and/or quality of the analysis material at some point. The most frequently used devices were different versions of the Qubit fluorometers (Invitrogen, Thermo Fisher Scientific) (n = 23) and the NanoDrop spectrophotometers (Thermo Fisher Scientific) (n = 14). Other devices included the Agilent TapeStation 4150 system (n = 5) and the Bioanalyzer 2100 (n = 5) (Agilent Technologies), sometimes alongside Qubit (n = 4). Twelve studies checked the extracted DNA with 0.8% or 1.0% gel electrophoresis alone or in combination with Nanodrop (n = 6) or Qubit (n = 2). Other methods included qPCR (n = 2) and the Promega QuantiFluor Fluorescence Quantification System (n = 1). In the remaining 41 papers (46.1%), quantification or quantification was not mentioned.

While over 50% of the papers mentioned quantification in their methods, quantities were mentioned in only 27 (30.3%) of the reviewed papers which are summarized in Table 3. As illustrated in this table, 21 of these 27 papers had described their quantification method, and the other 6 had not. Three studies explicitly stated that quantification was done “to ensure that they meet minimum concentration and mass of DNA”, but they did not specify what the threshold was.

Different sequencing methodologies, such as shotgun sequencing and amplicon sequencing, can contribute to variability in results. Shotgun sequencing, while offering species-level resolution and functional insights, is constrained by the high DNA input required and the dominance of host DNA in low-biomass samples, which can inflate costs and reduce usable microbial reads. Amplicon sequencing, which targets specific regions of the 16S rRNA gene, is widely used due to its cost-effectiveness and ability to provide detailed taxonomic information. However, the choice of amplified region can significantly impact microbiome characterization as different hyper-variable regions of the 16S rRNA gene, exhibit varying resolutions and specificities for different bacterial taxa. Consequently, the choice of target region can influence the captured microbial diversity and composition (21, 23, 28). As there is currently no evidence on which 16S rRNA region is the most suitable for sequencing of the ocular surface microbiome, further investigation is required (8).

In 77 (86.5%) of the reviewed studies, amplicon sequencing of the 16S rRNA gene was performed, and the remaining five studies conducted shotgun sequencing (n = 11) or RNA-seq (n = 1). In the former group, the target regions were V1-V2 (n = 3), V1-V3 (n = 4), V3 (n = 2), V3-V4 (n = 44), V4 (n = 11), V4-V5 (n = 4), V5-V6 (n =1), or the full or almost full length of the gene (n =5); the 16S region was not specified in the other 3 studies. Among the 44 studies that targeted the V3-V4 region, some provided the primer name (n = 25) (Table 4), primer sequences (n = 29), or both name and sequence (n = 21); primers were not mentioned or specified in the other 11 articles.