To appraise the methodological quality of each study, three JBI quality appraisal tools were used according to the study design: (1) RCTs (13 items), (2) quasi-experimental studies (9 items), and (3) cohort studies (11 items) (46). Each item was answered in one of the four following ways: yes, no, unclear, or not applicable. Quasi-experimental designs were defined as those with exogenous explanatory variables (treatment or exposure) that the investigator does not control (27). This includes before- and after-design studies without randomization, including quality improvement. No studies were excluded based on quality. Two independent reviewers (IMD and VdG) appraised all the studies. Disagreements were resolved through discussion and consensus, and no additional reviewer was required.

No tool exists for the appraisal of the methodological quality of algorithms. We modified the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument (47) to create the aPpRaisal OF algorIthm quaLity instrumEnt (PROFILE). Modifications included: a) replacing the word CPG with the word algorithm; b) eliminating domain 6: editorial independence as this is not applicable for locally developed algorithms; and c) adding eight items related to content and development based on a literature review of clinical pathway appraisal tools (48–51). This review was done because Govender identified this as missing from the AGREE II instrument when using it to appraise algorithms (52). The PROFILE contains 24 items across five domains: (1) scope and purpose; (2) stakeholder involvement; (3) rigor of development; (4) content and process; and (5) implementation. The domains are further categorized across three processes: (a) development (domains 1–3), (b) content (domain 4), and (c) implementation (domain 5). Each item was scored as either “1 = yes” or “0 = no”. No algorithm was excluded based on quality. The PROFILE has three types of scores: (1) an overall quality score that uses all items; (2) domain scores, five in total, one for each domain; and (3) process scores, three in total. It was pre-tested using one algorithm by all three reviewers (IMD, EF and MM), following which a consensus meeting was used to finalize the items and create a user manual with details for each item (as with the AGREE II instrument). An overview of the PROFILE is available in Supplementary Table S4. One appraiser (IMD) independently evaluated each algorithm, and four reviewers (EF, MM, SA and A-SR) cross-checked all data. Discrepancies were resolved through consensus discussion. The inter-rater reliability was calculated using kappa statistics (53). The scores were represented as a percentage by totaling the number of each present item (1 = yes) and dividing by the total number of items.

Meta-analyses were conducted using STATA version 17 software (54). Random-effects (Hedges' g) models using the Sidik-Jonkman method were used to measure effectiveness of algorithms across continuous outcomes of interest using the standardized mean difference (SMD) and its 95% confidence interval (CI) (55). Random-effects models using the Sidik-Jonkman method was used for dichotomous outcomes of interest using pooled odds ratios (OR) and 95% CI (55).

When the median and interquartile range (IQR) were reported, they were transformed using Wan's method and Excel tool (56).

The I² test was used to assess statistical heterogeneity, which was considered low if <40%, moderate 30%–60%, substantial 50%–90% and considerable 75%–100% (57). When heterogeneity was ≥40%, a sensitivity analysis of influencers was performed by removing one study at a time to assess the impact of each study on the overall effect size (57). To determine other sources of heterogeneity, sensitivity analyses were performed by stratifying studies based on the type of setting (PICU or NICU) when two or more studies were available. Risk of bias was not assessed due to the limited number of moderate quality studies. Additionally, study design was not assessed as pooling bodies of evidence has a mainly concordant direction of effect (58).

A subgroup analysis to estimate the treatment effect was performed on the type of algorithm, as recommended in a systematic review of clinical pathways (28). Type of algorithm was determined by the embedded measurement instrument(s) and medications used.

When more than one time point was measured post-implementation, the first point was used because fidelity to implementation was considered the highest. When studies on IWS/weaning used two different algorithms based on different medications, both groups were included in the meta-analysis.

For each outcome of interest a forest plot displaying the meta-analysis was created, additionally subgroup and sensitivity analyses figures were created.

To assess publication bias, funnel plots were generated, and the Egger's test was used to indicate the likelihood of publication bias.

When a meta-analysis could not be performed, the results were presented narratively.

To assess the certainty of the findings, a summary of findings (SoF) table was created using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach (59). The GRADE approach assesses all studies together for each outcome of interest and rates the level of uncertainty for the risk of bias, indirectness, consistency, imprecision, and publication bias (57). The main outcomes included in the SoF table were: (1) LOS intensive care (2) length of MV; (3) duration of analgesics; (4) duration of sedatives; (5) cumulative dose analgesics; (6) cumulative dose sedatives; and (7) incidents of IWS, as seven is the maximum suggestion (39). The selection of these main outcomes was based on the literature and its relevance to the clinical setting (60).

All corresponding authors were sent a personalized email to clarify unclear risk of bias items or missing details of algorithm content, development, and implementation. Six authors provided additional information included in the analysis (61–66).

The three PROFILE scores (as described in section 2.4.2) were categorized based on a three-step quality threshold as determined in a systematic review of AGREE II instrument usage (67). A score was classified as “high” if >60%, “medium” if between 59% and 30%, and “low” if <30% (67). The results are presented as a heat map.

To analyze the process of establishing the algorithm in practice, the strategies in each study were categorized according to the EPOC subcategories of interventions targeted at healthcare workers (36). One item called “case-based and scenario evaluation” was added because this could not be categorized in the existing EPOC taxonomy. Results are presented in tabular format.

To analyze the determinants (barriers and facilitators) of algorithm implementation, narrative descriptions were extracted. and categorized as intervention, professional or organizational using Lau's framework (68). Each determinant was further categorized as a barrier or facilitator, and the level of implementation was categorized (pre-implementation, implementation, or post-implementation). The method used to obtain the determinant (measured vs. mentioned but not empirically verified) was recorded. Results are presented in tabular format.

To analyze algorithm fidelity, rates were extracted, reported as percentages and ranges, and presented in a table. Fidelity is defined as whether an intervention has been implemented as intended with two subcomponents: adherence and dose (69). Adherence refers to whether the intervention is delivered as intended, and dose refers to the number of intervention components delivered (69).

Staff and family satisfaction was reported in a table.

The characteristics of the 32 included studies are summarized in Supplementary Table S6, and the details of the algorithms are provided in Supplementary Table S7. Twenty-three studies used a quasi-experimental design (73%) (61, 63, 64, 66, 70–74, 76–83, 85–88, 93, 95), seven were cohort studies (21%) (65, 75, 84, 90–92, 94) and two were RCTs (6%) (62, 89). The studies were conducted across 11 countries: 15 in the United States (47%) (62, 64, 65, 70–72, 74, 75, 86–88, 90–93), 11 in six European countries (34.3%) (61, 63, 66, 73, 76–80, 94, 95), three in Australia (9.4%) (83–85), and three in three Asian countries (9.4%) (81, 82, 89). All but two studies (62, 83) were single-centers (94%) (61, 63–66, 70–82, 84–95). Two thirds of the studies were published in the last five years (66%) (63–66, 71–74, 77, 79–81, 85–93).

The sample comprised 9,289 children (55% in the algorithm/intervention group). Seventy-five percent of the studies (n = 24) had a sample with a mean or median age of less than 4 years (61–63, 65, 66, 70–75, 77, 78, 80, 82–85, 87–89, 91, 92, 94, 95). The majority of the settings were PICUs (72%) (61–66, 70, 72, 73, 75–77, 79–83, 85, 89–93), 16% were neonatal ICU (86–88, 94, 95), and 22% were pediatric cardiac ICUs (71, 74, 78, 84, 85).

The distribution of the 28 algorithms by type of condition represented is shown in Figure 2. Nine (32%) of the 28 algorithms focused on one condition alone (65, 70–72, 81, 87, 89–92), one on pain (87), one on sedation (81), and seven on IWS/weaning (65, 70–72, 89–92). Of the remaining 19 algorithms (68%), all but one (88) combined sedation with at least one other condition, which were distributed as follows: 12 “pain-sedation” algorithms (61, 63, 64, 73, 75–77, 84–86, 93–95), four “pain-sedation-IWS” algorithms (62, 74, 78–80, 83), one “pain-sedation-delirium-IWS” algorithm (66), and one “sedation-IWS” algorithm (82). The other combination was “pain-IWS” (88).

Nurses were responsible for managing the algorithm (88.9%) (61–66, 70–81, 83–86, 88, 89, 91–93, 95), one algorithm was managed by a pharmacist (3.7%) (82), another by a pharmacist, a critical care physician, and a nurse (3.7%) (90), and one by a pharmacist and nurse (3.7%) (65). One study did not report the HCP responsible for management (87).

Fifteen studies included information on the process and documentation of algorithms. Of these, ten (67%) integrated the measurement instruments into the electronic health record (61, 65, 66, 74, 81, 86, 88, 92–95), two (13%) included a portion as an order set in the electronic health record (70, 85), and three were paper-based at the bedside (63, 64, 71).

The primary analgesic agent used by algorithms was morphine (77%) (61–63, 65, 73–75, 77–80, 83–89, 91, 92, 94, 95) and the primary sedative agent was midazolam (77%) (61–63, 71, 73, 74, 76, 78–80, 82, 83, 85, 86, 89, 92, 94, 95).

The measurement instruments used by algorithm, condition, and measurement frequency are summarized in Supplementary Table S8. Of the 14 algorithms that included IWS (62, 65, 66, 70–72, 74, 78–80, 82, 83, 88–92), the most commonly used measurement instrument was the WAT-1 (69%) (62, 65, 66, 71, 74, 89–92), and the monitoring frequency varied from 4 to 12 h (62, 65, 66, 71, 89–92). Nineteen algorithms included pain measurement instruments (61–64, 66, 73–80, 83–88, 93–95), of which seven (37%) assessed pain without using a combined pain and sedation measurement instrument (62, 64, 74, 83, 87, 88, 93), the most commonly used was the FLACC (57%) (62, 64, 74, 93), and was monitored every 4 h (62, 74, 83, 93). Nineteen algorithms included sedation measurement instruments (61–64, 66, 73–86, 93–95), eight studies assessed sedation alone (62, 64, 74, 75, 81–83, 93), and the remaining 11 used a combined measurement instrument (one which combines the assessment of pain and sedation) (61, 63, 66, 73, 76–80, 84–86, 94, 95). When only sedation was assessed, the most often used measurement instrument was the SBS (62, 64, 74, 83, 93), and the most often used monitoring frequency was every 4 h (75%). One algorithm used the COMFORT-B but assessed only sedation (81), another used the COMFORT and assessed only sedation (82), and another used a non-validated measurement instrument, the Seattle PICU Comfort Tool (75). The 11 algorithms that used a combined pain and sedation instrument, 45% used the COMFORT-B (73, 76, 77, 84, 85), 27% used the COMFORT-B + NISS (30%) (61, 66, 78, 80), one used the COMFORT-B + NRS (9%) (63), and 18% used the NPASS (86, 95). The monitoring timeframe varied between 3 and 8 h. One algorithm assessed delirium using the CAPD (66).

Twenty-five of the 29 studies with data on ICU LOS were included, with Vipond et al. having two different medication groups (62–66, 70–80, 82, 87–95). The algorithm group showed a small decrease in ICU LOS (SMD = −0.13; 95% CI = −0.22 to −0.05; p = 0.01; Figure 3A) compared to the usual care group. Substantial heterogeneity was observed (I² = 60%).

Four studies were not included in the meta-analysis because of missing data, three of them showed no significant differences between the two groups (78, 83, 84), while Larson et al. found a significant increase in PICU LOS, but this outcome was measured in hours instead of days (85).

All ten studies with outcome data on hospital LOS were included, and the algorithm group showed a small decrease in hospital LOS (SMD = −0.28; 95% CI = −0.45 to −0.10; p = 0.001; Figure 3B) compared to the usual care group (62, 65, 70–74, 77, 90, 93). Substantial heterogeneity was observed (I² = 77%).

Twenty-six studies included duration of MV (62–64, 66, 70–80, 82–85, 87, 89–92, 94, 95), of which two were excluded due to missing data (83, 84). Four studies were analyzed separately because duration of MV was reported in hours (63, 74, 80, 85). Of twenty studies, the algorithms group showed a small decrease in time on MV (SMD = −0.14; 95% CI = −0.27 to −0.01; p = 0.03; Figure 4) (62, 64, 66, 70–73, 75–79, 82, 87, 89–92, 94, 95) compared to usual care. Substantial heterogeneity was observed (I² = 81%).

The four studies that reported MV in hours showed no difference in the length of MV between the algorithm and usual care groups (SMD = −0.11; 95% CI = −0.49 to 0.27; p = 0.58) (63, 74, 80, 85). Substantial heterogeneity was observed (I² = 81). Sensitivity analysis showed that Larson et al. (85) was an outlier, and when removed I² was reduced to 0. A small statistically significant effect in hours of MV was observed (SMD = −0.30; 95% CI = −0.46 to −0.15; p = 0.0001; Figure not shown).

One study with missing data and showed no significant difference between the algorithm and the usual care groups (84).

Nineteen studies included cumulative dose of analgesic medications (62, 64, 66, 70–73, 78–80, 82, 84, 87, 90, 92–95). Four studies were excluded due to measuring outcomes differently: per visit (74), per patient (72) instead of over time, cumulative dose for the first 12 h, instead of the entire admission period (84), and Yang et al. included data in box plots (93). Of the 15 studies; the algorithm group showed a decrease in the cumulative dose of analgesic medications (SMD = −0.26; 95% CI = −0.43 to −0.08; p = 0.0001; Figure 5A) (62, 64, 66, 70, 71, 73, 78–80, 82, 87, 90, 92, 94, 95) compared to usual care. Considerable heterogeneity was observed (I² = 89).

Of the four studies not included in the meta-analysis, one study showed a statistically significant decrease in the cumulative morphine dose (mg/kg/visit) in the algorithm group (74), while the other three studies showed no differences between the algorithm and usual care groups (72, 84, 93).

Eighteen studies included the outcome of cumulative dose of sedative medications (62, 64, 66, 70–74, 78–80, 82, 84, 87, 89, 92–95). Three studies were excluded because the outcomes were measured differently: per visit (74), per patient (72), per hour instead of days (84). Yang et al. included data in box plots, so was not included (93). Of 14 studies; the algorithm group showed a decrease in the cumulative dose of sedative medications (SMD = −0.20; 95% CI = −0.33 to −0.08; p = 0.0001; Figure 5B) (62, 64, 66, 70, 71, 73, 78–80, 82, 87, 92, 94, 95) compared to usual care. Substantial heterogeneity was observed (I² = 73).

Three studies not included in the meta-analysis showed statistically significant decreases in the algorithm group: one in cumulative lorazepam dose (mg/kg/visit), another in total dose of midazolam (displayed as box plots) (93), and the last in cumulative dose of midazolam per hour (74). The study by Sanchez-Pinto et al. showed no statistically significant difference between the algorithm and the usual care groups (72).

Twelve studies evaluated the duration of use of analgesic medications (62, 65, 70, 71, 73, 75–77, 83, 91, 92, 95). Three studies were excluded, one due to missing data (83), and two for measuring duration in hours instead of days (73, 76). Of nine studies; the algorithm group showed a decrease in the number of days of analgesic administration (SMD = −0.19; 95% CI = −0.37 to −0.02; p = 0.03; Figure 5C) (62, 65, 70, 71, 75, 77, 91, 92, 95) compared to usual care. Substantial heterogeneity was observed (I² = 79).

Three studies not included in the meta-analysis showed no statistically significant differences between the algorithm and the usual care groups (73, 76, 83).

Fifteen studies included the duration of sedative medications (62, 71–73, 75–77, 81–83, 86, 89, 90, 92, 95). Three studies were excluded because of missing data (81, 83, 86). Three additional studies were excluded, two for measuring duration in hours instead of days (73, 76), and one for measuring duration of sedative medications per patient (72). Of nine studies; the algorithm group showed a decrease in the number of days sedatives were administered (SMD = −0.32, 95% CI = −0.55 to −0.09; p = 0.01; Figure 5D) compared to usual care (62, 71, 72, 75, 77, 82, 90, 92, 95). Heterogeneity was substantial (I² = 88).

Three studies not included in the meta-analysis showed no statistically significant difference between the algorithm and usual care groups when measuring duration in hours instead of days (73, 76, 83). Hawzani et al. included run charts and showed a significant decrease in the hours of sedative use over the course of algorithm implementation (81). Puthoff et al. did not include the data for the control group but noted a statistically significant decrease in days on benzodiazepines (86).

Thirteen algorithms reported the percentage of patients with IWS symptoms (62, 64–66, 71–73, 76–79, 82, 89). Ten studies included the event of developing IWS for both groups, and the analysis showed that the odds of experiencing IWS were reduced by 35% for children in the algorithm group (OR 0.65; CI = 0.49 to 0.87; p < .0001; I² = 47; Figure 6A) (62, 64–66, 73, 76, 78, 79, 82, 89) compared to usual care.

Two studies not included in the meta-analysis showed a decrease in the rates of IWS in the algorithm compared to the usual care group (71, 82), and the remaining five showed no significant difference between the groups (72, 73, 76, 77, 92).

One study reported on the treatment of delirium with antipsychotics and there was no significant difference between the two groups (p = 0.09) (64).

Seven studies reported on inadequate sedation management (61, 62, 76–78, 80, 85). One study was excluded because it did not report total numbers (76); two reported only on the post-implementation period (78, 80); and another did not include data (77). Of three studies, no difference in under-sedation was observed between the two groups (OR 1.23; CI = 0.63–2.40; p = 0.54; I² = 82; Figure 6B) (61, 62, 85).

Sensitivity and subgroup analyses were not performed because there were only three studies.

Two studies reported on over-sedation and were not statistically significant but had opposite directions of effect, one with more incidents of over-sedation in the algorithm group (85) and the other with more incidents in the usual care group (61). Dreyfus et al. found a decrease in the mean number of over-sedation levels in the algorithm group (76). Gaillard et al. stated no significant difference was observed between groups (77).

Three studies reported on inadequate pain management (62, 76, 95), but a meta-analysis could not be performed due to the difference in data. One study reported only post-implementation results (95). One study reported no difference between the two groups (62), while another showed an increase in the number of adequate mean pain levels documented (76).

Six studies reported on the duration of weaning, and all were included in the analysis (65, 71, 72, 89, 90, 92). The algorithm group had fewer days of weaning from medications (SMD = −0.63, 95% CI = −0.88 to −0.39; p = 0.0001; Figure 6C) than the usual care group. Heterogeneity was moderate (I² = 52).

Four studies reported the duration of methadone exposure and were all included in the analysis (65, 70, 89, 91). The algorithm group had fewer days of methadone exposure (SMD = −0.76, 95% CI = −1.78 to −0.05; p = 0.04; Figure 6D) than the usual care group. Heterogeneity was substantial (I² = 84).

Two studies reported on pain scores (62, 74). Curley et al. reported that the intervention group had a greater percentage of days with any report of a pain score of 4 or higher compared to the control group (p < .001); however, there was no difference between the two groups for modal pain scores of less than 4 (62). Lincoln et al. reported no significant differences in pain scores between the groups (74).

Nine studies reported on sedation scores (61, 64, 73, 74, 76, 77, 81, 85, 93). Three studies reported median COMFORT-B scores per patient, and showed no statistically significant difference between the algorithm (SMD = 0.09, 95% CI = −0.32 to 0.49; p = 0.68; I² = 87; Figure 6E) and the usual care groups (61, 73, 76).

Among the studies not included in the meta-analysis. One study reported the percentage of COMFORT-B scores per patient increased significantly from 40% in the usual care group to 75%-85% in the algorithm group (81). Three studies that implemented a “pain + sedation” algorithm reported a significant increase in the number of COMFORT-B assessments completed daily (73, 76, 85). Two studies reported no difference between the number of assessments per day, one using the COMFORT-B (77) and another using the SBS (74). Three studies using the SBS reported no statistically significant difference between SBS scores before and after implementation (64, 74, 93).

Two studies reported on withdrawal scores (62, 74); both showed no difference between groups, one in peak WAT-1 scores (62), and the other in mean WAT-1 scores (74).

The subgroup analyses by algorithm type indicated no statistically significant subgroup effects for cumulative dose of sedative medications, duration of analgesic medications and methadone exposure (Supplementary Figures S1–S8), suggesting that algorithm type does not modify the effect of algorithms compared to usual care. However, the small number of studies and participants in some subgroups may indicate that the analysis may have been unable to detect subgroup differences.

There were some differences observed by algorithm subgroup types: •

IWS algorithms showed a decrease in length of hospital stay (SMD = −0.34; 95% CI = −0.64 to −0.03; p = 0.08; I²= 60); a decrease in the cumulative dose of analgesic medications (SMD = −0.54; 95% CI = −0.93 to −0.15; p = 0.07; I² = 69%) (70, 71, 90, 92); and a decrease in the number of days analgesic medications were administered (SMD = −0.41; 95% CI = −0.82 to −0.01; I² = 65; p = 0.09) (71, 72, 90, 92).

The summary of findings table for the seven key outcomes, using GRADE, is presented in Table 1.

No publication bias was observed for any of the outcomes with more than 10 studies (Supplementary Figure S9).

The sensitivity analyses of removing one study at a time showed no significant reduction in heterogeneity for all but three outcomes of interest. Firstly, for length of MV, three studies individually decreased the heterogeneity by 2%–7% (66, 78, 91) but removed together, I² decreased by 30% with the same small-sized effect (SMD = −0.14; 95% CI = −0.23 to −0.05; p = 0.001; I²= 51). Secondly, for duration of analgesic medications, two studies were outliers (71, 77) with no significant decrease in heterogeneity when individually removed, but when removed together, I² decreased by 24% (SMD = −0.18; 95% CI = −0.31 to −0.05; p = 0.01; I² = 55). Lastly, for incidence of withdrawal where one study (82) when removed decreased the I² by 28% with a slight effect size change (OR 0.69; CI = 0.56 to 0.86; p < .0001; I² = 19) (Supplementary Figures S10–S20).

Sensitivity analysis by type of setting showed a decrease in LOS ICU for PICUs (SMD = −0.12; 95% CI = −0.22 to −0.02; p = 0.34; I2 = 53%) but not for NICUs, and a decrease in the cumulative dose of analgesic medications for NICUs (SMD = −0.33; 95% CI = −0.52 to −0.15; p = 0.03; I² = 57%) but not for PICUs. The test of group difference was not significant for either analysis (results not shown).

Sensitivity analyses by setting could not be performed for the outcomes of hospital LOS, duration of analgesic medications, duration of sedative medications, and incidents of withdrawal because there were no NICUs or >2 studies.

All sensitivity analyses by study design showed that quasi-experimental studies favored algorithms (results not shown).

Implementation strategies were reported in 26 of 32 studies (61–63, 65, 66, 70–78, 81, 83–88, 91–93, 95). Of the 17 EPOC categories, 13 were used (76.5%), the details are presented in Supplementary Table S13. Apart from one study (91), all other studies applied multiple strategies. The most frequently applied strategies were educational meetings (n = 24, 92%) (61–63, 70–78, 81, 83–88, 91–93, 95), the provision of materials (n = 24, 92%) (61–63, 65, 66, 70–78, 81, 83, 85–88, 92, 93, 95), followed by out-reach visits (n = 13, 50%) (62, 63, 66, 70, 71, 75, 77, 78, 81, 83, 84, 93, 95). The most frequently used non-educational strategy was continuous quality improvement (n = 12, 46%) (63, 70–75, 81, 86, 88, 93, 95). The least frequently used implementation strategies were audit and feedback (n = 2, 8%) (62, 81), tailored interventions (n = 2, 8%) (70, 81), use of local champions (n = 1, 4%) (62), managerial supervision (n = 1, 4%) (63), and monitoring performance (n = 1, 4%) (65). The mean number of implementation strategies used across all studies was 4.2 (median = 3.5, IQR 2). High-quality algorithms used a mean of 5.4 (median = 4.5, IQR 5) implementation strategies, compared to medium- and low-quality algorithms, with a mean of 3.4 (median = 3.5, IQR 2.5).

Fifty-five determinants (barriers or facilitators) were reported across ten studies (61, 66, 70, 73, 81, 83, 88, 90, 95) and grouped into 48 unique determinants. Five studies measured determinants (66, 73, 81, 83, 90), three measured determinants quantitatively using surveys (66, 83, 90), and two measured determinants qualitatively (73, 81). Supplementary Table S14 presents the organizational, professional, and interventional determinants.

The major organizational barriers were lack of leadership support, lack of planning for training, and competing priorities. The organizational facilitators were team buy-in, support, and involvement.

The major professional barriers relate to the complexity of applying algorithms to patients due to stability and age, and nurses' lack of knowledge of the algorithm. Professional facilitators included education and a positive attitude.

The major intervention facilitators were the structure of the algorithm, automatic alerts for assessment, and support for decision-making, resulting in ease and efficiency. No intervention barriers were reported.

Fidelity to the algorithm was reported in 19 studies (68%) (61–63, 66, 70–73, 77–79, 81, 83, 85, 87, 90, 92, 93, 95). Five studies reported both adherence to the algorithm and the dose of algorithm components (26%) (62, 66, 70, 83, 87). Eight studies reported on adherence (42%) (71–73, 78, 79, 81, 90, 93) and six reported on the dose of delivery of components (32%) (61, 63, 77, 85, 92, 95). The dose category had two overarching sub-categories: (1) dose related to the use of the measurement instrument, reported by eight studies (63, 66, 71, 73, 77, 83, 85, 95), and (2) dose related to medication delivery reported by four studies (61, 63, 87, 92). Overall adherence varied considerably, ranging from 36% to 100%. The dose associated with using the measurement instrument across all conditions was high, ranging from 65% for IWS (62) to 95.9% for pain and sedation (77). One study reported adherence to medication at 61% (63), and two studies reported loading dose delivery ranging from 8.2 to 86% (63, 92). One study reported both bolus prescription and medication prescription delivery with a 90.6% adherence rate (63). Supplementary Table S15 presents details of fidelity.

Six studies reported on staff satisfaction levels (63, 66, 74, 83, 90, 95) and one on parental satisfaction (87) (Supplementary Table S16). All studies with pre- and post-implementation statistics supported improvements; however, physicians showed greater satisfaction, as compared to nurses. Lincoln did not include a percentage score, instead indicated nurses' satisfaction with patient sedation management post-implementation was 8.5 (74).