To contextualize and understand the progress and challenges in UAV-based flower detection and classification, it is important to examine how flower detection methods have evolved over time. Initially, these methods were based on classic computer vision techniques that focused on analyzing and extracting image features such as color and texture using simple mathematical approaches. For example, Adamsen et al. (2000) developed a fully automated system for non-destructive flower counting using digital camera images and the Euler method, which resulted in a time savings of 92% per image compared to manual counting. Also threshold analysis emerged as an effective technique for separating peach (Horton et al., 2017) or canola (Zhang et al., 2021) flowers from the background. In a different approach, Hsu et al. (2011) developed an interactive flower recognition system that allows the user to define an area of interest in which the flower is located. The area is segmented and the extracted features - such as color and shape - are statistically analyzed and compared to a database of flower images. The flower class is identified based on the smallest Euclidean distance between its features and those of the input image. As the field progressed, machine learning algorithms were introduced to further enhance the process of classification. For instance, Nilsback and Zisserman (2008) improved flower recognition by combining multiple feature extraction techniques with a support vector machine (SVM), leading to a significant boost in accuracy. They explored the importance of different features for distinguishing between several similar classes of flowers and achieved a 12.7% increase in accuracy by combining four distinct features ⁷ compared to using a single one. Siraj et al. (2010) compared the performance of a logistic regression model for feature extraction with that of a neural network. The neural network outperformed the logistic regression, achieving 41.83% higher accuracy in classifying flower images. Building on the foundational work of early neural networks, recent advances in deep learning, particularly through the use of convolutional neural networks (CNNs) (O’Shea and Nash, 2015), have revolutionized image processing (Ye, 2024). Unlike traditional neural networks, CNNs are specifically designed to recognize and process spatial patterns in images. Their architecture of local connections and parameter partitioning not only increases efficiency, but also minimizes computational requirements (O’Shea and Nash, 2015). These models have significantly enhanced the capabilities of flower recognition, delivering very good results. For example, Xu et al. (2018) reported an accuracy of more than 97% when applied to cotton flowers and the best CNN model from Sarkar and Kelley (2023) achieved an accuracy of 94%. Further extending CNN capabilities, models like Faster R-CNN have been adapted for real-time applications in various environments. Patel (2023) employed this model with a ResNet50 backbone to accurately recognize the flowering stages of marigolds in real-time field scenarios. Additionally, Abbas et al. (2022) compared different backbones for Faster R-CNN against SSD for flower detection and classification using digital camera images. Their findings revealed that the Inception V2 backbone demonstrated the best performance with a mean Average Precision (mAP) of 91.3%. Beyond these applications, John et al. (2024) explored YOLO, RetinaNet, and Mask Region-Based Convolutional Neural Network (Mask R-CNN) to determine the biodiversity of mountain meadows. Furthermore, Basavegowda et al. (2024) used EfficientDet, trained on data from greenhouses and grasslands, to detect High Nature Value (HNV) indicator plants in semi-natural grasslands. This model was specifically trained with nadir perspective data to facilitate future research, including studies using UAVs.

The advances in flower detection and classification have greatly improved the accuracy of these methods. To apply them to larger and difficult-to-access areas, innovative monitoring and data collection approaches, such as UAVs, are necessary (Pettorelli, 2013; Wan et al., 2018). Capture devices for flower detection range from smartphones (Wu et al., 2020; Shang et al., 2023; John et al., 2024) to digital cameras on ground vehicles (Ozcan et al., 2020). For large-scale analysis, remote sensing technologies such as satellite imagery (Landmann et al., 2015), manned vehicles (Barnsley et al., 2022), and UAVs (Wan et al., 2018; Xu et al., 2018; Zhang et al., 2021) are becoming increasingly important. In particular, UAVs enable efficient and large-scale data collection that is faster (Xu et al., 2018) and less resource-intensive than traditional, error-prone methods (MacLeod et al., 2010; Pettorelli, 2013). Furthermore, UAVs offer a standardized setup for automatic data collection, allowing for repeated measurements at the same location with minimal effort (Duro et al., 2007). Through their physical distance from the ground, UAVs also provide a non-invasive way to collect data (Xu et al., 2018). In addition, they offer a cost-effective alternative for environmental monitoring (Huang et al., 2013; Anderson et al., 2024), helping to alleviate the high costs typically associated with automated flower detection systems - a burden typically shouldered by public institutions (Gaston and O’Neill, 2004). By expanding the capabilities of automated detection and classification applications, UAVs are now being applied to a broad range of targets. These include fruit detection (Chen et al., 2017; Bellocchio et al., 2019; Akiva et al., 2020; Zhao et al., 2024), tree flower detection and quantification (Carl et al., 2017; Horton et al., 2017; Zhang et al., 2023; Shang et al., 2023), as well as flower identification, monitoring, and counting in grasslands (Petrich et al., 2020; Gallmann et al., 2022). Xu et al. (2018) were the first to introduce flower counting using UAV imagery. Despite its advantages, using UAVs for flower counting presents challenges, particularly in detecting smaller or obscured flowers due to the lower resolution compared to ground-based imagery. Even with manual counting in drone imagery, these issues persist, although the risk of human error, such as overlooking or double-counting (MacLeod et al., 2010) flowers, is reduced (Gallmann et al., 2022).

Overall, the cited studies show that deep learning models, especially when combined with UAV technology, are powerful and reliable tools for efficient evaluation of flower resources. Given the growing adoption of these methods for flower detection and classification, the Gallmann Phenotator Toolbox has been updated and extended by integrating two additional deep learning models and performing a comparative analysis of their accuracy on drone imagery. The extension allows expanding compatibility of the Gallmann Phenotator Toolbox with a wider range of models of the TF model zoo.

Given the proven effectiveness of drones for large-scale, non-invasive data collection, a drone was used to collect additional test data. As mentioned above, UAVs are particularly advantageous in areas that are difficult or inaccessible to humans (Feng et al., 2021). The use of the drone in the test field proved to be extremely useful, as the tall grass poses a challenge for ground-based flower counting methods - treading on the vegetation flattens it, which can affect the accuracy of the survey. The drone allowed us to collect data without disturbing the natural state of the field. However, it was necessary to maintain a certain altitude, as the wind generated by the drone can cause movement in the grass, resulting in visual noise in the images (Stojnić et al., 2021). For this study, the DJI Air 3 (see Figure 1 ) was chosen as the primary tool for capturing images from the drone. This easily maneuverable rotary-wing UAV allows for efficient flower collection as it does not require a launch or landing site and can take off directly in the field. It also has an automatic return function. The drone can take high-resolution images while hovering over the area at low altitude. However, the short battery life, which requires frequent recharging, limits the efficiency and range of data collection (Shi et al., 2019). Under optimal conditions, the DJI Air 3 offers a maximum flight time of 46 minutes and a maximum flight distance of 32 km. Its internal memory is 8 GB and can be expanded with an SD card. A key reason for choosing this drone was its dual camera system, consisting of a 1/3-inch CMOS 48 MP wide-angle and a medium telephoto camera ⁸ . This combination makes it possible to capture both wide-angle and detailed close-up images of flowering areas for more accurate and efficient automated flower detection. UAVs with RGB sensors provide images that can quickly cover large areas and are easy to operate, making them a cost-effective method for monitoring grasslands (Sweet et al., 2022). At 720 grams, the DJI Air 3 qualifies for the “open” category under EU Regulation 2019/947 ⁹ . This category is designed for low-risk operations, provided the pilot completes online training and passes a basic theory test. This minimal certification process makes drone technology widely accessible and encourages wider adoption for environmental monitoring.

While the UAV facilitated efficient data collection in the field, the training of the three models was performed using a machine learning hardware. It was equipped with two AMD EPYC 7742 central processing units (CPUs, 64 cores each), eight NVIDIA RTX A6000 graphics processing units (GPUs, 48 GB VRAM per GPU) connected via NVLink, and 768 GB of random access memory (RAM). As part of this work, the code base was updated to TF 2, following the official recommendations from the TF developers. Specifically, TF 2.15 was selected to ensure compatibility with CUDA 12.2 and cuDNN 8, as specified in the official TF table ¹⁰ . This combination of software versions was installed on the available server, allowing the integration and use of GPU resources for training and running the models ¹¹ .

The Gallmann Phenotator Toolbox is based on TF 1, a powerful framework for machine learning. It was designed to enable scalable and efficient computations in heterogeneous environments by optimally allocating computational resources and managing the state of variables (Abadi et al., 2016). However, with the evolution of machine learning technologies, TF 2 has introduced several enhancements offering a more intuitive, adaptable, and robust framework. The improvements over its predecessor include the following ¹² :

To take advantage of these new features and optimizations and to ensure compatibility with the latest software packages, the migration of the Gallmann Phenotator Toolbox code base to the newer TF version was required. TF 2 eases migration by providing comprehensive guides, documentation, and tools specifically designed to help developers transition from TF 1 to TF 2 ¹⁸ . The migration involved identifying incompatible code sections, particularly deprecated functions and outdated APIs, and updating them to their TF 2 counterparts to ensure full compatibility with the new framework. In addition to updating the Gallmann Phenotator Toolbox with newer software, the code was extended to support not only Faster R-CNN, but also EfficientDet and SSD models. Their differences and advantages are further outlined in Section 2.6 (Model Training).

After successfully updating and extending the Gallmann Phenotator Toolbox for better and extended performance and compatibility with modern hardware, the source code was made available in a GitHub repository ¹⁹ . The next step in the application was to compile a comprehensive dataset. The datasets used serve as the basis for training and evaluation of the models. In the following section, the datasets are presented in detail to provide a basis for the following analyses.

For this study, two datasets were utilized. To train and evaluate the models, the dataset from Gallmann et al. (2022) was selected. It consists of drone images of 1 m² test squares captured from a height of 19 meters. A different drone and camera were used compared to the setup of this study. The dataset contains several flower classes (see Table 1 ), with some classes combined due to their visual similarity. It is thoroughly annotated with these flower classes.

To obtain additional test data, the Hohenheim Dataset, an experiment was conducted. The data collection took place at the Hohenheim Gardens of the University of Hohenheim in Baden Wuerttemberg, Germany (coordinates: 48°42’29.6”N, 9°12’50.1”E) (see Figure 2 ) on July 16, 2024, between 13:35 p.m. and 16:05 p.m. The weather was sunny to cloudy with an average temperature of 31.6°C, an average global radiation of 800W/m² and an average wind speed of 12 km/h, which is within the endurance range of the drone. The grass of the test site was tall and dry. The light conditions varied slightly due to intermittent cloud cover. Care was taken to ensure that the drone’s shadow is not visible within the test field. The test site had a biodiversity of 17 different flower classes, nine of which match the flower classes of the Gallmann Dataset (see Table 1 ). The experiment was designed as follows. In an area of 18.400 m² (1.84 ha), 50 smaller plots of 1 m² each were marked out. Due to the use of the wide angle camera for some images, where no flowers could be identified, only 36 plots remained for analysis (see Figure 2 ). Each plot was marked using wooden stakes to ensure minimal environmental impact. Ground truth data were collected manually by observers at each plot, where all flower species were identified and counted. This step allowed us to verify the accuracy of the drone data with field observations, using standardized data sheets to ensure consistency. In addition to the number of instances of each flower class counted, the number of insects flying to the flowers were counted. Results show that the collected flower classes are attractive to pollinators. Subsequently, drone images of each test plot were captured. To maximize detection accuracy, it is crucial that the image quality is high enough to clearly identify the flowers. This requires sufficient image resolution to clearly capture the fine features of the flowers (Xu et al., 2018). As described in Gallmann et al. (2022), the ground sampling distance (GSD) for flower recognition should be a maximum of 5 mm/pixel, as up to this value the prediction performance only changes slightly. Ideally, the images used for analysis should maintain a GSD similar to that of the training images to ensure consistent flower sizes. The images of the Gallmann Dataset, including training images, have a GSD of approximately 1.5 mm per pixel, while the images acquired for the Hohenheim Dataset have a GSD of approximately 1 mm per pixel. In addition to the images, videos of the test site were recorded with the goal of creating an orthomosaic. For this purpose, a predefined route was set in the included DJI Fly app. The drone flew over the site at the same altitude used for image acquisition, maintaining a speed of 1.9 m/s, as specified in the study by Li et al. (2023). The wide angle camera was used for video recording to cover as much of the area as possible. As a result, the resolution is lower, at 3840 x 2160 pixels. An orthomosaic was created using Web Open Drone Map (WebODM) ²¹ .

To prepare the images for model training, the Gallmann Dataset was deterministically divided into 70% training data, 10% validation data, and 20% test data. To ensure sufficient representation of each class in the training set, any class with fewer than 50 instances was excluded, following the approach of Gallmann et al. (2022). Additionally, in this study the image preprocessing script was modified to exclude any class with zero instances in the training set, ensuring that only classes present in the training data were used. This resulted in 14 classes left for training (see Table 1 ). Gallmann et al. (2022) originally resized the input images into 450 x 450 pixel tiles, which were then upscaled to 900 x 900 pixels. This method is consistent with the recommendations of Hu and Ramanan (2017), who advocates dividing images into smaller sections and upscaling them when detecting small objects. In contrast, this study applied a resizing strategy using larger dimensions. For example, Liu et al. (2016) emphasize the importance of input size for small object detection, showing that larger input sizes improve detection performance compared to smaller ones. Their experiments showed that increasing the input size from 300 x 300 pixels to 512 x 512 pixels improved mAP by 2.5%. Based on these results, an input size of 512 x 512 pixels was chosen. To further increase the resolution and detail of the images, especially for recognizing very small flowers, for this study, the input images were scaled to 1024 x 1024 pixels before training. This additional scaling aims to improve the detection of finer details.

All three models relevant for model comparison were trained on the annotated data of the Gallmann Dataset. The Faster R-CNN model from the TF 1 Model Zoo used a manual learning rate by default, which was adopted by Gallmann et al. (2022). However, a cosine decay learning rate of 0.08 was chosen for this work. This learning rate adapts automatically during the training process, allowing for dynamic adaptation. In general, the default configurations of the models were used, with a few adjustments made to improve performance. Specifically, the maximum number of detections per class and the total number of detections were both increased to 300, and gradient clipping by norm was applied with a value of 10.0 to stabilize training. The models were trained with a batch size of 4 for at least 200,000 steps ²² . Performance was evaluated every 2,500 steps. EfficientDet required an additional 50,000 steps due to the learning rate not converging earlier. The model that achieved the highest F1-Score was saved separately in a designated folder for further analysis. The results of the analysis are detailed further in Section 3 (Results). The TF 2 Detection Model Zoo is a comprehensive collection of pre-trained object detection models based on the Common Objects in Context (COCO) 2017 dataset (Lin et al., 2014). Each model is available with different backbones, making them versatile tools not only for inference on new data, but also as starting points for training on new datasets. The model zoo contains a variety of models, including the aforementioned Faster R-CNN, SSD, EfficientDet, CenterNet (Duan et al., 2019) and Mask R-CNN (He et al., 2018), each of which offers different trade-offs between accuracy and speed ²³ . The first model analyzed is Faster R-CNN, which was also used in the study of Gallmann et al. (2022). The model is a two-stage detector that performs object detection in two steps. Its region proposal network (RPN) scans the input images of any size to generate region proposals, which are areas in the image where potential objects might be located. In a second step, the model classifies the objects within these proposed regions (Ren et al., 2016). The Faster R-CNN available in the TF 2 Model Zoo differs slightly from the TF 1 configuration used by Gallmann et al. (2022). The following data augmentation techniques were used in this study:

Liu et al. (2016) emphasizes the importance of data augmentation in detecting small objects especially for SSD models. Compared to the Faster R-CNN, a SSD combines object localization and classification into a single step. Instead of using a region proposal network that adjusts anchor boxes to suggest possible object regions, the SSD model uses predefined default boxes with different aspect ratios and sizes that are placed on the feature map ²⁴ . For each of these default boxes, the model computes probability scores for all object classes, indicating how likely it is that an object of a given class is located within the box. These default boxes are then refined by applying calculated offsets to generate the final bounding boxes that more accurately enclose the detected objects (Liu et al., 2016). SSD outperformed Faster R-CNN on the PASCAL VOC 2007 dataset (Everingham et al., 2010), delivering better accuracy and faster processing times (Liu et al., 2016). The third model in the comparison, EfficientDet, is also a single-stage detector designed for high efficiency and accuracy. Its key innovation, the Weighted Bi-directional Feature Pyramid Network (BiFPN), dynamically adjusts the importance of different features during training, allowing the model to focus on the most relevant information. By tightening connections and eliminating unnecessary paths, EfficientDet optimizes both computational power and accuracy. This makes it ideal for real-time analysis and resource-constrained applications (Tan et al., 2020).

Once the models are trained, the next step is to ensure their effective use in the field. Hodgson and Koh (2016) highlight the need for clear, actionable guidelines for UAV use in research. Building on the documentation in the Gallmann Phenotator Toolbox, step-by-step instructions are provided to help users accurately identify flower classes and numbers in meadows, ensuring ease of use in field studies.

In this section, the key steps for using UAVs in flower detection are outlined, including pre-flight setup, flight execution, and post-flight data processing. These guidelines ensure transparency, reproducibility and enable integration with machine learning models for flower classification in Germany. Before conducting drone operations, certain resources are required. These include a drone, such as the DJI Air 3, equipped with the necessary cameras, as well as a server for data storage, model training and evaluation. Additionally, operators must comply with the above mentioned EU Regulation 2019/947, which outlines requirements such as registration and certification for drone pilots. In addition, national regulations, including Germany’s Air Traffic Regulations (LuftVO) ²⁵ and the German Air Traffic Act (LuftVG) ²⁶ , mandate requirements such as liability insurance. Drones in the ‘open’ category require operator registration and a pilot’s license. In addition, operators should check geographic restrictions such as no-fly zones or altitude restrictions ²⁷ . Once all legal requirements have been met and a flight area has been selected, safety must be ensured by avoiding bystanders and strictly following regulations. During pre-flight preparations, camera settings must be configured to capture essential metadata, including GPS coordinates, altitude, speed, and timestamps for each image or video frame, the subtitle feature should be enabled in the drone’s camera settings, generating an SRT file during video recording (DJI, 2023). This metadata is essential for subsequent georeferencing and accurate orthomosaic creation. To enhance the accuracy of the georeferencing process, the use of ground control points (GCPs) should be considered (Gallmann et al., 2022; Sweet et al., 2022; Zhang et al., 2023; Torresani et al., 2024). Flight height should be carefully determined based on the size of the objects to be detected, the acceptable wind generated by the UAV, and the area to be covered (Stojnić et al., 2021). Minimizing environmental disturbance is also critical to reduce the impact on wildlife and the surrounding ecosystem, as recommended by Hodgson and Koh (2016). For flower imaging with the DJI Air 3, a height of 15 meters has proven effective for capturing images. Since video footage has a lower resolution than still images, a lower altitude is recommended. To increase efficiency and conserve battery power, it is advisable to use waypoints. These can be planned before the flight, enabling the drone to automatically follow a predefined route, or set at specific positions during the flight (DJI, 2023). Other tools, such as DroneDeploy ²⁸ , offer waypoint planning with integrated orthomosaic generation. Finally, automatic settings for altitude, gimbal angle, and speed should be used to ensure consistent image quality. These parameters can be configured in the DJI Fly app before the flight, with the gimbal set to - 90° to provide a vertical view of the terrain. After completing pre-flight preparations, the drone will autonomously follow the predetermined flight path. It is important to make sure that the battery is fully charged and the weather conditions are stable before taking off (DJI, 2023). During flight, the drone’s status is monitored by the controller, allowing intervention if unexpected situations arise. Once the flight is complete, the captured data - including images, video, and metadata - should be transferred from the SD card to secure storage. It is recommended to back up the data in multiple locations to avoid potential loss. While the creation of an orthomosaic is optional, it can greatly assist in post-flight analysis for large areas. Tools like WebODM, a free and open-source solution, offer reliable alternatives to expensive commercial software for photogrammetry processing (Vacca, 2020). Creating orthomosaics is a standard practice in many UAV-based studies, with several researchers using commercial software solutions such as Agisoft Metashape (Gallmann et al., 2022; Zhang et al., 2023; Torresani et al., 2024), Pix4D (Zhang et al., 2021; Anderson et al., 2024), and DroneDeploy (Horton et al., 2017). In contrast, this study uses WebODM, which offers photogrammetric processing capabilities comparable to the commercial options, providing reliable photogrammetric processing on low-cost drone imagery (Vacca, 2020). In order to identify flowers on the created orthomosaic, it may be necessary to divide the large image into smaller sections if the maximum number of 178,956,970 pixels to be processed by the Phenotator Toolbox TF2 is exceeded. For analysis, regions of interest (ROIs) can be selected using the Gallmann Phenotator Toolbox. If adequate resources are available, custom models can be trained; otherwise, pre-trained models can be used. Ideally, when custom models are trained, they should be tailored to the specific flower species present in the meadow being analyzed. Predictions can be made on the orthomosaic or on selected ROIs (Gallmann et al., 2022). The results can then be visualized in a dashboard, which is introduced as a new feature in the Phenotator Toolbox TF2, offering a clear overview of detected flower classes, facilitating interpretation of results and supporting decision making in ecological studies.