A total of 56 canine/handler teams from law enforcement, government, and private companies participated in this study; however, not all teams participated in all areas of the study (See Supplementary information). Prior to the trial, participating canine teams were asked to provide information about their canine (breed, age, number of years the dog has been in service) and themselves (number of years the handler has been in service), and the most recent year of successful certification (see Supplementary information for these details). To maintain anonymity, canine/handler teams were assigned team numbers that cannot be traced back to the original team or their organization. The participating canines were all operational dogs that were trained, housed, and handled by their handler or agency.

The six explosives used in the study were those dictated as “required” by Standard 092. For security reasons, these will be referred to as Explosives 1 through 6. The explosives were purchased from OMNI explosives. According to Standard 092, the minimum amount of each explosive for the certification assessments should be no less than ¼ lbs. (113.5 g) or 8 ft. in length of 50 gr/ft. (9 g/m), and thus this quantity was utilized in all searches related to the certification assessment. For non-operational odor recognition assessments, such as the odor recognition test (ORT), a maximum of ¼ lbs. (113.5 g) or 8 ft. (metric) in length of 50 gr/ft. (9 g/m) of explosive material was used. These quantity specifications were used in all certification trials. For the “real-world” scenarios, the amounts of explosives used are listed below in Tables 2, 3.

Materials were handled in a specific way to minimize cross-contamination of odors. Explosives were weighed out one at a time into anti-static bags, placed into separate metal paint cans based on explosive type, and properly sealed. The area’s surface was decontaminated using an alcohol wipe and allowed to dry before the next explosive was prepared. Target odors were prepared a minimum of 18 h in advance of the study. Olfactory distractors for the trials were chosen from a variety of commonly used household items, non-target items used in the experiment, and other items thought to cause false alerts (refer to Tables 2, 3). The chosen distractors ranged from having low to minimal odor to being highly odorous. Distractors and blanks were housed separately from the explosives. Explosives and distractors were placed in 8 oz. Training Aid Delivery Devices (TADDs; SciK9) on all occasions, unless otherwise noted. A Mixed Odor Delivery Device (MODD) was used in the scenarios to safely deliver the odor of targets that would traditionally be detected as a mixture (21). For the ORT, all target odors, blanks, and distractor odors were presented in 4 × 4 × 6-inch or 6 × 6 × 6-inch boxes that were purchased from Uline.

The study consisted of three trials conducted within different regions of the United States (see Table 1). Testing sites used for each location were chosen based on availability and the standard-dictated space requirements. The SW trial took place in an office building, the SE trial at a university, and the W trials in a prison. Search areas varied across trial locations but were kept as consistent as logistically possible. The organization of when these searches took place remained consistent across all three trials. Each trial took place over 2 days and included searches in compliance with Standard 092 as well as more operationally realistic searches, referred to as “real-world scenarios.” Standard 092 included searches of rooms, parcels, vehicle exteriors, and luggage, as well as odor recognition tests. The ORT is a test of the canine’s olfactory ability to alert to target odor(s) in a controlled manner where the odor is readily available, but still visibly concealed from the canine/handler team. The targets odors are placed in a in line-up surrounded by distractor odors and blanks 3 ft. apart from each other. Scenarios were created to mimic searches canine/handler teams may encounter in real search operational contexts. Tables 2, 3 list all searches for each day of the trial, as well as the Standard 092 requirements for each search, the type of containment used, and the explosive and distractor odors presented. Table 4 provides the timing each team was permitted for each search, as well as the Standard 092 requirements for each search. A more detailed set-up and explanation of each search can be found in the Supplementary material.

All rooms utilized in each location had the minimum required space dimensions (Table 4) and included extra furniture and other items such as desks, cabinets, office supplies, etc. Target odors were placed within search areas a minimum of 30 min in advance of the first search for both the morning and afternoon sessions to allow for the odor to penetrate into the room (soak). Targets were left in place and were not moved in between canines. The searches conducted were single-blind, meaning the evaluators knew the placement and number of the targets, but the handlers did not. The search order of canine/handler teams was randomized prior to the start of each trial. Canine teams were not permitted to view the assessment area beforehand, nor were they permitted to watch any other canine team perform the assessments. Two evaluators were present for every search and provided instructions to the canine/handler teams about the search area and time limits before each search (Table 4). Handlers were instructed to provide a verbal indication if their canine alerted and to specify where the alert occurred. Evaluators then gave a verbal indication of whether the team was correct. The team continued to search until the target was found, the room was cleared, or the time limit was reached. All handlers were given the opportunity to allow their canines to search on or off leash.

Assessment sheets from each evaluator were collected and compared, and the canine responses were coded onto an Excel spreadsheet. The percent positive alert rates were calculated by the total number of positive alerts throughout the entirety of the trial divided by the total number of times the target odorant was present throughout the trial. The false alert rates were the number of total false alerts throughout the trial divided by the total amount of blanks and distractors present at each portion of the trial. Statistical analysis was completed using the Chi-square test (Microsoft Excel 365) to compare performance between trials as well as to compare the performance between “real-world” scenarios and Standard 092. The results were considered statistically significant if p ≤ 0.05. Additional multi-level logistic regression analyses are available in the Supplementary material that account for the variation attributable to canine/handler team and trial location, but these show the same effects as the more parsimonious Chi Square analyses. As a result, we have chosen to present the simpler analyses in the main body of the paper.

A total of 56 teams participated throughout the course of the study. Twenty (20) teams participated in Trial 1 (SW), 17 teams participated in Trial 2 (SE), and 19 teams participated in Trial 3 (W). Figure 1 provides the summarized results from the real-world scenarios and Standard 092 certification assessments for each trial. Canine/handler teams performed significantly better in Trial 2 than in either Trial 1 (χ² [1, N = 27] = 33.16, p < 0.001) or Trial 3 (χ² [1, N = 26] = 24.10, p < 0.001). Further comparison between scenarios and standards revealed that teams in Trial 3 performed significantly better on Standard 092 than in the scenario (χ² [1, N = 19] = 5.64, p = 0.018) while in Trials 1 and 2, the teams depicted no significant difference in performance between Std 092 and the scenarios.

Figure 2 compares the positive and false alert rates for both the scenarios and standards. The ORTs were removed from the standard calculations to simplify the data analysis. Across all three trials, Trial 2 had the highest false alert rate for both real-world scenarios and Standard 092 searches at 15 and 13%, respectively, while also having the highest correct response rates. Figure 3 further delineates the types of false responses that occurred during the trials. The false alerts were categorized by blanks, such as if the canine alerted to empty TADDs and/or empty boxes, and distractors, such as those listed in Tables 1, 2. Canines alerted to distractors more frequently in both the scenarios (14%) and standard (8%) than blanks at 8 and 7%, respectively. Canines had the highest false alerts on Sharpies and anti-static bags. There were a few additional false alerts that were neither on distractors nor blanks; these were classified as unknown false alerts.

Analyzing the individual canine/handler team responses (Supplementary information) revealed that no team would have passed the OSAC certification. Two teams were excluded from the data set as they had only completed the morning session of day one of the trial. Standard 092 requires a 90% correct alert rate with a less than 10% false alert rate for a successful certification. Five teams achieved a 70% or greater positive alert rate and a less than 10% false alert rate. An additional three teams achieved 70% or greater positive alert rate, but a false alert rate greater than 10%. The average percent alert rate of this high-achieving group was 79% ± 6% on Standard 092 and 86% ± 16% for the scenarios with 4 out of the 8 teams achieving a 100% positive alert rate on the scenarios (Figure 4), indicating high success on the Standard 092 assessment was correlated to high success on the real-world scenarios. Low-achieving groups were also examined. Twenty-one (21) teams fell into the range of 36 and 50% performance rate, and 15 teams performed less than 35% on the Standard 092 assessments (Figure 4). While the performance of the teams in the 36–50% group on the scenarios was more variable, overall, the performance of the low-achieving groups on the Standard 092 also appeared correlated to low success on the real-world scenarios.

Six different explosives were used throughout the study. Figure 5 provides information on canine/handler team performance based on explosives used in the Standard 092 portion of the trial. Overall, Explosive 2 had the highest positive response across the three trials at 87, 74, and 64%. In addition to Explosive 2, teams from Trial 1 performed above their overall percent positive alert rate on Standard 092 (shown by the red line in Figure 5) on Explosive 4, teams from Trial 2 on Explosives 1, 3, and 4, and teams from Trial 3 on Explosive 3. Explosive 6 had the lowest detection rates across all three trials, and Explosive 5 had low detection rates on Trials 1 and 3.

Figure 6 presents a comparison of performance between the first and second day of the trial, categorized by explosive type. Generally, a higher positive alert rate was observed on the second day of the trial for most of the explosives in Trials 1 and either an increase or consistent response in Trial 2; however, Trial 3 rates were more varied. The most notable improvement of detection was on Explosive 4 and 5 in Trial 1 with an increase of 28 to 74% alert rate and 8 to 50% alert rate, respectively, and on Explosive 5 in Trial 2 with an increase of 25 to 74% alert rate, implying that exposure to the target improved later detection.

Interestingly, the opposite trend was observed in Trial 3, where overall detection performance either declined or remained unchanged across explosive types, most notably for Explosive 2, which showed a decrease in alert rate from 92% on the first day to 35% on the second.

The goal of this proof-of-concept study the performance of working canine/handler teams on certification tests developed in alignment with the OSAC explosive detection Standard 092, as well as their performance on real-world scenarios designed to mimic operational conditions, utilizing 56 teams from three locations in the U.S. Canine/handler team performance varied greatly between the three trials. Trial 2 had the overall highest rate of true positives for both Standard 092 and the real-world scenarios. Trials 1 and 3 yielded similar, but significantly lower, true positive rates than seen in Trial 2 for Standard 092 assessments. Canine/handler teams who participated in Trial 3 seemed to struggle with the real-world scenarios more than the Standard 092 assessments, in contrast to observations in Trials 1 and 2. True positive rates also varied based on which of six different explosives was presented, suggesting that teams may not have had access to some of the explosives as often during training. Many canine/handler teams showed better performance on Day 2 of the trial, with teams in Trial 1 showing the greatest improvement from Day 1 to Day 2. In contrast, Trial 3 teams showed either no improvement or worse detection rates on Day 2 compared to Day 1.

When evaluating performance at the individual team level, none of the 54 teams that participated in at least one full day of a trial met the performance threshold required for certification under Standard 092. The standard requires a correct response rate of 90% with a false alert rate of less than 10%. Team 9 from Trial 2 was the highest scoring team with an 88% correct alert rate, with a 6% false alert rate on Standard 092 assessments. Seven other teams from across the three trials achieved a correct alert rate greater than 70%, while the remaining 46 teams fell below this passing threshold.

Several factors may account for the failure to meet certification requirements. One possibility is that the OSAC standard represents a particularly stringent and demanding certification process. Even teams accustomed to passing other certification tests may have struggled under the higher demands of the OSAC protocol. In comparison, other certification protocols have been cited as less rigorous than the OSAC standards for excluding the ORT (22), having smaller search areas for rooms and vehicles (22, 23), and having ill-defined requirements for the certification, leaving it up to the interpretation of the evaluator (24). Due to the length of testing, canines may have experienced fatigue or frustration which may have contributed to poor performance. This could have been especially true during prolonged or difficult searches such as the ORT or the vehicle search.

Second, four of the six explosives used within the trial, and required by Standard 092, can be difficult or expensive to obtain. Teams that showed reduced efficacy with these particular explosive materials likely had limited exposure to them during routine training. Indeed, four of the six explosives used in these trials can be very challenging to access. Thus, teams that performed fairly well overall but struggled on trials with particular explosives likely had less exposure to these materials. In many cases, these teams may only encounter these explosives once or twice a year when completing their certification hosting by other agencies, such as the North American Police Working Dog Association (NAPWDA) (25) and the National Police Canine Association (NPCA) (26). In order to increase proficiency, a team should train with multiple exemplars of target odors to improve generalization. If teams are only exposed to one source of a target odor, it could lead to discrimination, leading to no alert on a target of a similar make-up. To improve generalization, it is recommended that EDD teams be exposed and trained to a variety of training aids and training samples to promote generalization (2, 27).

Finally, teams may have experienced performance anxiety or felt unfamiliar with the trial as a whole, leading to less than optimal performance, especially on the first day. This interpretation is supported by the pattern of improved detection rates on Day 2 (see Figure 6), as this may reflect increased comfort with the flow of the trial and reduced anxiety. Additionally, performance differences could have also resulted from positive contingency and reinforcement history.

We also found differences in false alert rates across the three trials. Teams in Trial 1 achieved a low false alert rate on the Standard 092 tests (6%) and on the real-world scenarios (7%). In contrast, Trial 2 yielded the highest false alert rates for Standard 092 tests (13%) and Real-World Scenarios (15%), higher than the maximum false alert rate required to pass the OSAC certification (10%). Other certification programs, such as those offered by NAPWDA, require a comparable level of performance, mandating a minimum accuracy rate of 91.6% and permitting only a single miss over the course of the certification evaluation (25, 28).

Given that Trial 2 also yielded the highest overall correct alert rate, these patterns may reflect a response bias rather than a difference in discriminability. Trial 1 teams may be more conservative in calling alerts, resulting in fewer false alerts in addition to fewer correct alerts, while canine/handler teams in Trial 2 tended to be more liberal in their alert calls. Thus, Trial 2 yields higher false alert and correct alert rates. Teams from Trial 3 seemed to struggle with discriminating between target odors and distractors, though, because their correct and false alert rates were more similar than seen in the other trials.

In the context of explosives detection, these response tendencies may reflect strategic real-world tradeoffs. Being more willing to call an alert could be preferable, even if it means there will be more false alarms. A team that is too conservative with calling alerts may risk clearing an area when an explosive is merely very well hidden or unusual in some way.

Another objective of this study was to determine if the observed performance rate on Standard 092 could predict how well canines would perform in operational contexts. Standard 092 recommends incorporating searches reflecting circumstances that canine/handler teams may face operationally into both training and certification. A key limitation of standardized search assessment scenarios is that, while they offer a controlled and consistent environment conducive to fair and reproducible certification, they lack the complexity and unpredictability of real-world operational settings. Real operational searches are typically more challenging (due to various distractions and attempts to mask or conceal the target) and unpredictable.

Overall, if teams performed well on Standard 092, they also tended to perform similarly on the real-world scenarios (Figure 1). The individual team data reinforced this finding as the teams who performed best on the Standard 092 also performed well when completing the Real-World Scenarios. Likewise, teams that did not perform well on Standard 092 also did not perform well on the real-world scenarios (Supplementary Tables 4, 5). There were only two teams that were outliers, achieving a correct alert rate of 36 to 50% on Standard 092 tests, but 100 and 80% on the Real-World Scenarios. This suggests that meeting the Standard 092 criteria may predict success in operational contexts too.

Distractors (non-target, intentionally placed items) made up the highest percentage of false alerts across all three trials (9%). The highest false alert rate was on Sharpie markers. Other distractors odors that had high false alert, included anti-static bags, dryer sheets, isopropanol wipes, rubber bands, Tang, latex gloves, Play-Doh, banana peels, motor oil, and taco seasoning. Many of these are commonly used when training canines. Sharpie markers are typically used to indicate the contents inside a container or training aid, while anti-static bags are used to store explosives to protect against static electricity. Dryer sheets are reportedly used to decrease the static on canines before searching, and isopropanol wipes are used for cleaning and disinfecting items or an area before and after placing odors to minimize the chance of cross-contamination. Empty TADDs yielded the highest false alert rate of the blank items. This could have been a visual cue, as sometimes the canines were able to dislodge hidden distractors. The TADDs may also have an associated odor to which the canines tend to respond if these devices are regularly used during training. False alert rates of this type can be mitigated through regular training and certifications using distractors and matched blanks.

One assessment that was particularly challenging for canine/handler teams was the parcel searches, where teams had difficulty locating the two explosive hides in a ten-box line-up, as delineated in the Standard 092 assessments. In Trial 1, parcels were filled with packing material and office supplies and taped closed (a single line of packing tape across the middle seam on top and bottom). Many handlers attributed the difficulty to the fact that they did not normally train or certify with “closed” boxes. In Trial 2, the boxes contained only the targets or distractors, with no additional packing material added, and were closed in a crosswise fashion with no tape involved. Figure 7 provides the results of Days 1 and 2 of the parcel searches for Explosive 4, as this explosive was used in both of the parcel searches. On the first day of Trial 1, teams correctly identified the box containing Explosive 4 only 5% of the time; however, the detection rate greatly improved on Day 2 to 67% detection rate. Comparatively, Trial 2 similarly had a detection rate of 70% on Day and 100% on Day 2.

As one would expect, the canines had a more challenging time finding the target odor when the boxes were taped closed than when they had just been crossed over. This shows an underlying problem with the way some teams train for parcel searches. It is unlikely that an explosive being concealed within a parcel would remain unsealed, as the perpetrator would want to hide the energetic material. Studies have shown that packaging materials, such as cardboard, limit the amount of vapor that escapes into the open environment (29), making it harder to detect. However, other studies have shown that trained detection canines are highly efficient and effective at locating contained targets with proper training (30). The data here also suggests that the canine/handler teams were capable of completing this task after some practice (see Day 2 data in Figure 7). This reinforces the idea that materials used in training and certification should be prepared in ways that prepare canines for operational contexts.

The paradigm provided by the OSAC standard was challenging to carry out, both in regards to time and space. The certification took place over two full days starting around 8:30 am and ending around 5 pm. With the real-world scenarios removed off the itinerary, the days would shorten, but not by much. Finding a facility to accommodate each individual search where no overlapping of rooms was difficult. In the first trial, multiple rooms needed to be combined to meet the size requirements of the room searches and the same rooms needed to be reused on the second day just due to lack of extra space within the facility. For agencies that either have or can work with an organization that has such large facilities, the certification may be feasible to achieve, but for smaller organizations that do not have access to a facility it may be more challenging. Additionally, since the study was not double-blinded, it was possible that assessors provided unintentional clues to handler about the location of targets. This would lead to the handler providing unintentional clues cues that influenced canine performance, potentially skewing the results.

Although our study did not directly measure cognitive or biological state variables, several factors could have plausibly contributed to detection failures. Mostly commonly, training issues, such as inferior training materilas or limited training opportunities for either the handler or canine, or sufficient searching patterns [31], can be pinpointed for detioration in team performance. However other complications, may have been at play. For example, the temperature was high in most days of all three trials (Day 2 of the SE trial was the exception). Several searches were outdoors, which, in addition to the walking from the vehicle kenneling location to the indoor searchs, could cause excessive exertion, though the canines were given the opportunity to rest whenever needed. It has been shown that increased physical exertion can degrade olfactory sensitivity, where dogs on a treadmill exhibited a drop in accuracy rate from approximately 87% to below 45% for weak odor concentrations after moderate-intensity exercise over time [32]. Similarly, intrinsic characteristics such as arousal and motivation—often described as ‘hunt drive’ or ‘search arousal’—have been linked to detection success across operational contexts [33, 34]; however, while high arousal may initially elevate engagement, excessive arousal, such as marked excitability, can impair detection accuracy, potentially via panting interfering with olfactory sampling [35, 36]. Non-cognitive variables like handler stress due to unknown or testing scenarios can also undermine performance, regardless of a dog’s detection capability [31]. Although these factors were not directly measured in our study, considering their potential influence provides a richer interpretation of the observed protocol differences and highlights valuable directions for future investigation.