We developed a custom pipeline for scraping and annotating insect images from online databases centered around three major components.

BugNet follows a four part data pipeline centered around a modular localization and hierarchical classification system. Insects are cropped from input images using a YOLO-V5-M (Jocher et al., 2020) localization model trained on 250,000 manually annotated insect images. This model is used for insect localization only before passing cropped insects to a separate model for classification. Although there are more recently developed models available in the YOLO family of architectures, we consistently find YOLO-V5 outperforms more recent models in localization only tasks. The localization model can be run on images of any resolution, but expects insects to be between approximately 5x5 pixels and 800x800 pixels in size. As such, images where insects fall outside of this range may have to be rescaled before being processed. As insects attracted to light and chemical baits can typically vary in size over three orders of magnitude (Chown and Gaston, 2010), the localization model is run twice on each input image: once at a resolution defined by the user, and once with the image resolution decreased by a factor of eight. Potential duplicate detections are filtered out by applying a size filter to each bounding box, ensuring that the user-defined resolution captures only small insects and the reduced resolution pass captures only large insects. An adjustable confidence threshold is applied to remove low-probability detections from each image, and the remaining cropped insect images are scaled to 128x128 pixels before being passed to an EfficientNet-B5 classification model.

To incorporate information across multiple taxonomic levels, the classification model was modified to incorporate hierarchical outputs, allowing the simultaneous output of confidence scores for taxonomic order, family, genus, and species (Supplementary Figure S1). Incorporation of hierarchical data into models has shown promise in systems where objects can be grouped taxonomically (Redmon and Farhadi, 2017), and more recently in insect systems (Badirli et al., 2023; Bjerge et al., 2023). The BugNet hierarchical classifier differs from these models by using a simplified architecture which only modifies the final layer of a pre-existing model, and by using a modified loss function which allows the model to incorporate images which lack annotations at certain taxonomic levels (Supplementary Figure S2), i.e. the training data can contain images for which the species, genus, family, or order are unknown. Complete details on how this architecture was developed and the loss functions used to incorporate hierarchical information into the models can be found in the supplement. This approach allows the model to learn to directly discriminate known and unknown taxa by using missing labels as data to inform model training, without using post-hoc out-of-distribution tests (Liu et al., 2020; Chen et al., 2025) or anomaly detection (Bjerge et al., 2023; Supplementary Figures S3-S4).

Classified images are run through a binary filter based on an EfficientNet-B0 model trained on 56,000 field images to remove additional false positives. As cameras with fixed positions produce series of images which may capture the same insect multiple times, the cropped insects are clustered across frames using a networking approach. For each detected insect, the intersection over union (IOU; See Table 1 for term definitions) between the insect’s bounding box and all other predicted bounding boxes from the previous frame is calculated, and boxes with indices greater than 60% are connected within a single sub-network. This process is repeated iteratively across all input images to build a network representing all groups of bounding boxes which have a high probability of representing the same insect over multiple frames, giving each a unique ID. Cropped insect images can be saved with their associated ID and predicted taxon for later validation or statistical processing. All software is made available under the GNU Affero General Public License v3.0.

The YOLO-V5-m localization model used for both cropping of GBIF image data and field image inference was trained at a resolution of 832x832 pixels on a dataset of 250,000 manually annotated insect images (1–108 insects per image, ~260,000 total) with 70% of these images used for training and 30% for validation. Insect images used for training this model were downloaded from GBIF and represent temperate and tropical species from 11 orders. To improve the model’s capability to detect insects over large body size ranges, a random scale augmentation was applied to each image batch, where images were randomly scaled on the range of 50%-150% of the input size. The standard PyTorch RandAugment image augmentation suite was applied on top of these augmentations. Model overfitting was monitored by comparing performance on the training and validation imageset in each epoch, and model performance during training was determined from Precision, Recall, and F-score metrics.

To understand model robustness to image distortions that might typically be encountered in the field, we tested the performance of the localization and classification models on augmented validation images. For each image, we tested fractions of salt and pepper noise ranging from 5% to 60% of pixels in an image being modified to all white or all black, exposures of 0.5x and 1.5x normal image brightness, and Gaussian blurs with 5x5, 7x7, and 11x11 kernels. We note that the highest levels of image noise and blur tested here fell well outside the range we would typically expect from field images, but this allowed us to capture the range of image distortions the models can still function under. Salt and pepper noise was applied to images for a given noise fraction such that a) exactly that fraction of pixels would be modified rounded up to the nearest pixel, and b) if there were an odd number of pixels to be modified there would be exactly one extra white pixel. Brightness was modified linearly using the adjust_brightness() function from the torchvision transforms.functional library. Gaussian blur was applied using the GaussianBlur() function from torchvision transforms.v2. with a standard deviation of 1, 2, and 5 pixels respectively for each level of blur. Augmentations were applied to images at their standard dimensions (ranging from 5 x 5 px to 6,000 x 4,000 px, mode resolution = 1024 x 1024 px) before being scaled to the size required by each model. Images for localization were scaled to 832 x 832 px and images for filtering and classification were scaled to 128 x 128 px. Model performance was evaluated on balanced accuracy, precision, recall, and F-score metrics across images with annotations at each taxonomic level. All computer vision models were trained on an NVIDIA GeForce RTX 3060-Ti GPU for 100 epochs.

For testing the model training and inference pipeline at larger scales, we trained an EfficientNet-B5 model under the Wide architecture and coding unlabeled images as zeros (See supplemental methods) on approximately 1.12 million images of neotropical nocturnal insects sampled automatically from GBIF. We focus on tropical insects for two major reasons. First, while tropical forests contain the majority of the world’s biodiversity, they are underrepresented in existing classification datasets (but see Jain et al., 2024 and Gonçalves-Souza et al., 2025 as recent examples in these systems). Second, the high species richness of these ecosystems in combination with the large number of undiscovered species they contain make tropical systems ideal for testing the performance of classification models on known and unknown species. In the full dataset, 100% of images were labeled to the level of Order, 99.44% to Family, 78.57% to Genus, and 2.64% to Species. The training dataset contained 13 orders, 230 families, 915 genera, and 34 species. Due to the presence of families with no label at the genus level, and genera with no species level labels, this represented a total of 1127 unique taxa. The model was trained following methodologies established above for 450 epochs.

To understand model behavior in field conditions, we ran the full model pipeline on images of nocturnal insects collected using camera traps in Costa Rica and Ecuador. Images were collected using 16MP Arducam board cameras at light traps at La Selva Biological Station, Costa Rica; Iyarina Research Station, Ecuador; and Yanayacu Research Station, Ecuador for four to eight nights at each site (Supplementary Table S1). A subset of 5,000 images containing insects from these sites were selected for analysis, but were not otherwise curated before testing. Images were localized at a resolution of 4656 x 3496 px. Classified images were validated by a trained entomologist to the lowest taxonomic level that could reasonably be applied for any given image. If features necessary for identification were ambiguous at a certain taxonomic level, that level was left unannotated and a classification was applied at the next highest level. Model performance was evaluated on balanced accuracy, precision, recall, and F-score metrics on images with annotations at each taxonomic level. We additionally report these metrics on images which could not be confidently annotated. While these do not necessarily represent misidentifications, as a model ID may be correct even when an image cannot be confidently classified by a human, they do represent the model operation on images which received an unverifiable ID. All models were trained and tested using PyTorch with CUDA v11.7.

Performance for the localization model was calculated by 1) merging predicted bounding boxes with an intersection over union (IOU; da F. Costa, 2021) greater than 0.25, 2) choosing the bounding box with the highest IOU over 0.25 as the prediction for a ground truth box, and 3) using standard binary methodologies for calculating balanced accuracy, precision, recall, F-score, and the area under the Receiver Operating Characteristic (ROC) curve (Table 1). Performance for the hierarchical classifier was calculated using standard microaveraged techniques (i.e. by averaging across all samples without weighting by taxon) for balanced accuracy, precision, recall, and F-score. These metrics were calculated after applying a 50% confidence threshold to estimate the ability of models to ignore unlabeled images. Multi-class area under the ROC curve was calculated following the method of Hand and Till (2001).

Accuracy, precision, and recall were typically calculated based only on known taxa within any given taxonomic level, as a classification cannot reasonably be considered correct or incorrect when there is no ground truth to validate against. As classification models are frequently applied with confidence thresholds which remove low confidence classifications, the inclusion of “unknowns” in model outputs should not strongly affect how these data are interpreted compared to traditional classification models, but the presence of unknowns in the training and test data will affect the interpretation of model performance on datasets where different proportions of the data are labeled. As such, these performance metrics may appear overinflated in this study compared to studies where all images are labeled, and direct comparisons of taxonomic levels become difficult as each taxonomic level in the data we present here has different proportions of labeled and unlabeled images. To help correct for this, model performance on the field dataset was calculated based on the entire dataset as well as the subset of the data which could be confidently labeled by including performance metrics where “unknowns” are considered a separate class in each taxonomic level. The proportion of field images which received any classification was additionally calculated for the full dataset, to allow comparison to recall of known taxa. Unless stated otherwise, accuracy, precision, and recall should be interpreted as model performance only when taxa are known.

Model performance for each suite of augmentations was compared to baseline performance using beta regressions with a logit link function. As the quality of cropped insect images varied dramatically with insect size and resolution within each field image, we additionally fit generalized linear models with binomial distributions for each taxonomic level comparing model precision to image resolution for all known cropped field insects. All analyses were conducted in R version 4.2.1 (R: The R Project for Statistical Computing, 2025).

Localization and binary filter model performance were high on unaugmented images (Supplementary Tables S4, S5). Area under the ROC was 97.85 for the localization model and 99.15 for the filter model on unaugmented images. Localization model accuracy was reduced by addition of noise (pseudo‐R² = 0.79, P < 0.001) and blur (pseudo‐R² = 0.78, P < 0.001), but not by changes in brightness (pseudo‐R² = 0.32, P = 0.255). However, model accuracy only significantly differed from baseline performance when > 40% of pixels were replaced with noise (pseudo‐R² = 0.24, P < 0.001) and at the highest level of blur (pseudo‐R² = 0.04, P = 0.003; Supplementary Table S4). The binary filter model was more dramatically impacted by noise and blur, with significantly reduced accuracy with >10% of pixels replaced by noise (pseudo‐R² = 0.69, P < 0.02) and at the two highest levels of blur (pseudo‐R² = 0.83, P < 0.001; Supplementary Table S5). As the binary filter acts to remove low quality insect crops before classification, this decrease in accuracy represents insects which would not be passed to the hierarchical classifier due to noise or blur.

The dataset scraped from GBIF using the BugNet pipeline consisted of 1.12 million images representing 13 orders, 230 families, 915 genera, and 34 species of insect for a total of 1127 unique taxa.

The number of training images available for each taxon was typically highest at the order level and lowest at the species level, with a median of 10,543 images in each order, 1,263 in each family, 520 in each genus, and 431 in each species. However, sample sizes were highly imbalanced across taxa, with the least represented order, Dermaptera, containing only 1,050 images and the most represented order, Lepidoptera, containing 263,071 images. Similarly, samples sizes ranged from 225 to 51,451 at the family level, 219 to 4,903 at the genus level, and 240 to 1,689 at the species level (Supplementary Figure S5A).

Classification model performance was typically highest at the order level, followed by family, species, and genus. Model balanced accuracy was 98.6%, 94.6%, 85.7%, and 84.3% at the order through species level, respectively (Table 2). The area under the ROC was 93.50%, 88.39%, 82.27%, and 90.33%, for each taxonomic level, respectively (Supplementary Table S6). Model accuracy was typically highest for taxa in the order Lepidoptera, which comprised the majority of images in the training data, and the only orders with accuracy below 90% were the four least represented orders, Orthoptera, Blattodea, Mantodea, and Neuroptera, which together comprised less than 4% of the training database. Errors when classifying these orders were primarily misidentifications as Lepidoptera, likely to due bias introduced by imbalanced training data. Outside of these orders, errors tended to cluster within taxonomic groups. For example, while family level accuracy in the order Ephemeroptera was only 63.2%, 83.9% of these misclassified images were still classified as the correct order (Figure 3). Model accuracy was significantly reduced by the addition of noise (pseudo‐R² = 0.25, P < 0.001) and image blur (pseudo‐R² = 0.23, P = 0.011), primarily driven by reduced accuracy at the highest level of noise (pseudo‐R² = 0.48, P = 0.002) and blur (pseudo‐R² = 0.42, P = 0.007). There was no effect of image brightness on model accuracy (pseudo‐R² < 0.001, P = 0.91). Model performance did not significantly differ from baseline performance on any augmentation within standard levels expected in the field for precision (Supplementary Table S7), recall (Supplementary Table S8), or balanced accuracy (Table 2).

The localization model was able to detect a total of 13,739 insects in the 5,000 field images tested. From these images, a total of 12 orders, 56 families, 35 genera, and 3 species were manually identified. There was considerable variation in the number of images identified for each taxon in each taxonomic level (Supplementary Figure S5B), with the least represented orders, families, genera, and species consisting of only one image, while the most represented order, Lepidoptera (Supplementary Figures S6A-D), was identified from 6,743 images. The most represented family, Erebidae (Supplementary Figures S6B-D), was identified from 3,356 images, and the most represented genus, Amastus (Supplementary Figure S6D) was identified from 719 images. Identification of insect taxa and especially species was limited in part by cryptic taxa which cannot be identified without microscopy or dissection, but was also affected by low image quality (Supplementary Figure S6) and the abundance of extremely small insects. For example, the smallest insects which could consistently identified below order were two to four millimeters in length (e.g. Supplementary Figure S6J), but approximately 21% of insects cropped from field images were smaller than this length (Figure 4).

On images where taxa could be manually identified, model balanced accuracy was 98.3%, 96.2%, 93.3%, and 79.2% at the order through species level, respectively. Model recall was 97.0%, 92.7%, 86.8%, and 58.5% at the order through species level (Table 3). Species classifications could not confidently be applied in 99.52% of images, but model attempts to classify these images were minimal. Species predictions were applied to unknown insects in 0.67% of cases, and for images where genus could be confidently identified but species were unknown, species predictions were applied 2.3% of the time. Model precision was highly affected by the resolution of detected insects at the level of species (χ^21,129 = 6.29, P = 0.033), genus (χ^21,3272 = 979.7, P < 0.001), and family (χ^21,8092 = 17.2, P < 0.001), but not order (χ^21,13186 = 2.5, P = 0.13). Model precision was low below the median insect size observed in field images, but precision approached 100% for insects larger than 500x500 px at the genus level, approximately 3.25x3.25 cm at the typical field of view the cameras were set at (Figure 4). This increase in precision closely mirrored the increase in the proportion of taxa that could be confidently identified by eye as insect size increased. Although species level precision was low across observed insect sizes (Table 3), we note that only two of the species present in the training set, Samea ecclesialis and Pantherodes conglomerata were observed in field images. Precision on these taxa was 75% and 85.4% respectively.

Additional results, including an investigation of how the models handle “unknown” taxa that were not part of the training dataset can be found in the supplement.