Introduction
The World Health Organization defines patient safety as the reduction of risk of unnecessary harm associated with healthcare to an acceptable minimum. An adverse event (AE) is defined as an event that results in harm to the patient, while a no-harm incident is defined as an event that reaches the patient but results in no discernible harm (1). Patient safety extends beyond individual incidents; it encompasses a broader perspective within healthcare systems. This perspective considers patient safety as a discipline within healthcare systems, and as an attribute of the systems that minimises the incidence and impact of AEs and maximises recovery from such events (2). Learning from mistakes and enhancing visibility play crucial roles in patient safety efforts (2, 3).
When identifying healthcare-related complications, often referred to as AEs, a structured retrospective medical record review can serve as a method for extracting data (4, 5). Among the established review methods, the Global Trigger Tool (GTT) developed by the Institute for Healthcare Improvement, is one of the most commonly used (6). The methodology has been found to identify more AEs when compared to other methods (7, 8). The GTT includes a two-step review process. First, two primary reviewers independently review the medical record for the presence of triggers—specific terms or events that could indicate AEs. They then determine whether an AE has occurred, assess its severity, and compare their findings to reach consensus. In the second step, a physician authenticates the reviewerś findings but does not conduct an additional record review (6). The GTT is designed to be used within somatic inpatient care for adults but has also been used without adaptations for evaluating paediatric care (9).
Adaptations from GTT, so-called trigger tools, have been developed for various healthcare settings and focus areas worldwide (4, 10), including paediatric inpatient care (11–16), prehospital emergency care (17) and in- and outpatient oncology care (18). Additionally, a medication trigger tool has been designed to explore adverse drug events in paediatric haematology and oncology patients (19). The method has also been adapted to identify no-harm incidents in adult patients in different settings (20–22) and in paediatric prehospital emergency care (17). Previous work on trigger tool development has highlighted that adaptations are often required to reflect the characteristics of the specific clinical environment (12, 15, 18). In line with this, paediatric oncology presents characteristics that warrant consideration.
Paediatric oncology is a complex and continually evolving clinical field. New treatments continue to emerge, and paediatric oncology care has increasingly shifted from inpatient admissions to day care and home healthcare settings, where care is collaboratively provided by patients, parents and healthcare professionals (23). Remarkable progress has been made, leading to increased survival rates for paediatric oncology patients over the years (24–26). However, children with oncological diseases still face extensive care needs due to their underlying conditions and the treatments they receive, which can result in serious treatment-related complications and patient safety risks (27–31). As paediatric oncology care increasingly extends beyond inpatient care into day care and home healthcare, safety risks arise not only from treatment-related toxicity, which itself is not confined to inpatient care, but also from care transitions, communication failures and complex medication management at home (32). Furthermore, distinctive characteristics of paediatric care, such as pharmacological and physiological differences (33, 34), and childreńs dependence on parents (35–37), create safety challenges that require dedicated consideration with a distinct form in paediatric oncology due to the intensity and complexity of oncological treatment (27). Traditionally, patient safety efforts in paediatric oncology have centred on treatment-related toxicity (30). Knowledge about different types of AEs and no-harm incidents along the broader continuum of care remains limited. Safety knowledge therefore needs to extend beyond treatment-related toxicity to support the development of safety strategies that align with the evolving field (24, 27, 30).
As the GTT was originally developed for adult inpatient somatic care it does not capture the full range of AEs and no-harm incidents occurring in paediatric oncology. Existing adapted trigger tools likewise do not fully address the specific needs of this context, and few are designed to capture no-harm incidents, which are included within the scope of this work. In response to these contextual challenges, the national multicentre study, Patient Safety in Paediatric Oncology (PaSPO), was initiated to increase knowledge about patient safety in paediatric oncology through identifying AEs and no-harm incidents throughout the patient journey from specialised hospital care to home healthcare. As part of PaSPO, the aim was therefore to develop a Paediatric Oncology Trigger Tool (POTT) to facilitate the detection of AEs and no-harm incidents across the continuum of care in paediatric oncology.
Materials and methods
Study design
The design of the study to develop a context-adapted trigger tool was based on a literature search, a three-phase modified Delphi process, and experiences captured from a manual record review process. The reporting of the study was guided by the proposed steps of the Accurate Consensus Reporting Document guidelines (ACCORD) (38, 39). The PaSPO multicentre study was led by a research group acting as the steering committee for this study.
Setting
Paediatric oncology care in Sweden is provided by six geographically dispersed paediatric oncology centres at university hospitals in collaboration with paediatric departments at county hospitals. The patients receive inpatient care, outpatient care, and home healthcare. The latter is sometimes provided by municipality nurses. Four out of six paediatric oncology centres and several county hospitals, whose catchment area corresponds to 64% of the population of Sweden covering both urban and rural areas, participated in this multicentre study. All six paediatric oncology centres were represented in the Delphi process.
Collation and solicitation of triggers
The development of the POTT was initially inspired by previous work on trigger tools for home healthcare and paediatric hospital care (15, 22). The last author (MU), who was involved in these earlier studies, contributed methodological expertise to ensure consistency and comparability with previously used approaches for trigger tool development. These studies provided a conceptual and methodological foundation for defining and categorising triggers and associated AEs and no-harm incidents.
Building on this foundation, a literature search was conducted in PubMed to identify trigger tools and studies on AEs across a broad range of care contexts, with particular attention to oncology, paediatrics and paediatric oncology across different settings, including both hospital care and home healthcare. The purpose of the search was to identify additional potential triggers, ensure alignment with existing knowledge of AEs within the clinical field, and support the adaptation of the triggers to the specific context of paediatric oncology care. Each potential trigger was linked to associated AEs and no-harm incidents. The findings from the literature search constituted the basis for the first preliminary version of the POTT, which was created in an iterative process by the steering committee. This preliminary version, with triggers, trigger definitions and decision support information, formed the basis of the subsequent Delphi process.
Prioritisation and refinement of triggers
A modified Delphi process was used to gather the insights of a multidisciplinary group of experts in a structured way (40). Delphi was originally developed in the 1950s and involves a structured iterative process aiming at gaining consensus among experts (41). The modified Delphi method is a common approach in the development of trigger tools in various settings (12, 15, 16, 22, 42). Flexibility exists regarding the design and format of the method (43). In line with several other studies using modified Delphi methods, adjustments were made from the original method regarding the view of anonymity, consensus and the use of virtual meetings in the study (44). The Delphi process included three rounds, including two virtual meetings and a web-based survey.
The Delphi panel
Potential expert participants in the Delphi panel were identified by the steering committee through existing networks. The professional experts were purposefully selected with the aim of representing relevant stakeholders and capturing practical and theoretical expertise in patient safety and trigger tool methodology, as well as experiences of paediatric oncology in various geographical and healthcare settings in Sweden. The invitation, with introductory information, was sent to 60 potential participants. One reminder email was sent. Those who responded that they could not participate in the virtual meeting in the first Delphi round were offered the opportunity to contribute in writing. To incorporate the perspectives of patients and parents early in the development of the POTT, four parents with experience in paediatric oncology care were invited to participate in writing in the first Delphi round. The experts contributing to the virtual meeting and/or in writing in the first Delphi round constituted the Delphi panel. To minimise potential bias, none of the members of the steering committee participated in the Delphi panel; however, they acted as facilitators in the meetings.
First Delphi round—virtual meeting
Before the virtual meeting, the experts received information, including an overview of the POTT, and information about the study and Delphi methodology. The experts were strategically assigned into four equal-sized groups to achieve representativeness in all groups based on competence and context. The respective group discussions were led by a facilitator from the steering committee; they were recorded, and notes were taken. The triggers were discussed in terms of clinical relevance, comprehensibility and usefulness in the four groups. The experts were also asked to rate the relevance of the triggers and trigger definitions using a four-point Likert scale from “not at all relevant” to “very relevant” and they were given the opportunity to suggest improvements. After the meeting, the experts could add assessments of triggers, other than those discussed in their own group, in writing through ratings and freely worded answers. The parents and those experts who did not participate in the virtual meeting answered in writing in the same way. The results of the ratings, the level of consensus and the free-text comments were analysed. The analysis and recommendations from the Delphi panel were compiled and discussed by the steering committee and revisions of the POTT were made.
Second Delphi round—web-based survey
A web-based survey was used, including the full revised POTT. The questions were aimed at capturing the relevance of the triggers and the comprehensibility of the trigger definitions. Ratings were made using a four-point Likert scale. The experts were given the option to answer “no opinion” and to contribute freely worded answers. In addition, the expertś self-reported experiences of paediatric oncology, trigger tool methodology, and patient safety, as well as their experiences of participating in the Delphi process, were rated. The survey was internally pilot-tested by the steering committee for content, layout and wording. The Delphi survey was distributed electronically using the web-based system “esMaker®”. One reminder email was sent. The “esMaker®” system facilitates anonymised feedback from each expert, which was utilised in this study. The results of the ratings and the free-text comments were compiled, analysed and discussed by the steering committee. A refined POTT was produced and used in the record review process.
Capturing experiences from the use of triggers
In this phase, the medical record reviewers' experiences with the POTT in PaSPO were captured. The reviewers were identified by the steering committee through existing networks. Before the main record-review phase, a pilot test was conducted in which one reviewer examined 29 medical records representing patients diagnosed at one paediatric oncology centre. The purpose of the pilot was to assess the feasibility of the preliminary POTT in a real-world setting. Subsequently, the reviewers and the steering committee convened in a virtual meeting to discuss the clinical relevance, comprehensibility and usefulness of the triggers based on the pilot test results and the revieweŕs gained experience. The reviewers were invited to provide additional improvement suggestions in writing after the meeting. The discussions and written comments were compiled, analysed and discussed by the steering committee, resulting in a refined POTT. In the third step, the revised POTT was used in the main record review process in PaSPO. Medical records representing patients of different ages, diagnoses and types of treatment were reviewed across various healthcare settings, including paediatric oncology centres at university hospitals, paediatric departments at county hospitals and home healthcare. No specific time limit was imposed on the reviewers for completing the assessment of each medical record.In the fourth step, a focus group interview and a web-based survey were conducted to capture the reviewerś experiences. At the time of the survey, most medical records in PaSPO had been reviewed. The survey was internally pilot-tested in the same way as in Delphi round two. The reviewers rated the relevance and usefulness of the triggers, as well as the comprehensibility of each trigger definition and decision support information, using a four-point Likert scale. They also had the opportunity to submit free-text comments for each trigger. The survey was distributed electronically using the web-based system “esMaker®”, with one reminder email sent. The reviewerśexperiences with the POTT are elaborated further in a separate qualitative study (45). The findings from the virtual meeting discussions, the web-based survey, and input from the focus group interview were compiled, analysed and discussed by the steering committee. A refined POTT was then produced for the third Delphi round.
Final refinement of triggers
Third Delphi round—virtual meeting
In the third round, a smaller group of experts from the original Delphi panel was selected through purposeful sampling. The aim was to discuss, refine and anchor the POTT with stakeholders connected to national networks and organisations from the first two rounds. Prior to the virtual meeting, the experts received information including the refined POTT, details about the study and the Delphi methodology. During the virtual meeting, the triggers were discussed in terms of clinical relevance and usefulness. Those unable to attend the virtual meeting were invited to provide assessments of triggers and improvement suggestions through freely worded comments. The discussions and the written comments were compiled, analysed and discussed by the steering committee and revisions of the POTT were made, resulting in the final POTT.
Data analysis
Data from the web-based surveys were recorded in the survey tool software “esMaker®”. Ratings on Likert scales were converted into numerical values for analysis and presented as median values and min-max. A dichotomisation process was used to assess the consensus of the ratings, merging ratings of “one” and “two” as “low ranking” and “three” and “four” as “high ranking”. Consensus level was defined as the proportion of respondents giving the same dichotomised rating, ranging from 0.5 to 1, indicating either low or high ranking. Triggers with a median relevance rating below three, or those not reaching a consensus level of 0.80, were classified as “triggers to remove”. These could also be classified as “ambiguous triggers” if deemed valuable to retain based on steering committee discussions about clinical relevance in patient safety work. The remaining triggers were classified as “triggers to retain”. Data were analysed using IBM SPSS Statistics (RRID:SCR_016479).
Ethical approval
The study was conducted in accordance with international research standards and was approved by the Swedish Ethical Review Authority (2020-00116, 2021-03512).
Discussion
The POTT was developed using a rigorous and systematic multi-step approach, including literature reviews, a three-phase modified Delphi process, and insights gained from practical use. This process resulted in a tool with 22 triggers, corresponding definitions and decision support information designed to facilitate the detection of AEs and no-harm incidents in the patient process and enhance understanding of patient safety in the complex field of paediatric oncology.
The need for a context-specific trigger tool aligns with existing literature (9, 12, 15, 18). Our decision to develop a specific POTT reflects the unique challenges in paediatric oncology care and is strengthened by the fact that a significant burden of AEs has been identified using a context-specific trigger tool in adult oncology care (46).
It is strongly recommended to include paediatric high-risk populations and high-alert medications in medication safety research, consistent with our study (34). Adapting adult-focused trigger tools for paediatric use without modifications is not advisable, as noted over a decade ago (9). While several research groups have developed trigger tools for paediatric inpatient setting (11–16), our study is, to the best of our knowledge, the first to create a trigger tool for a medical record review in paediatric oncology. Our tool is designed for use across the entire care continuum, not just inpatient care; this is necessary to reflect the shift towards day care and home healthcare settings in paediatric oncology (23, 47). This comprehensive approach aligns with the development of a trigger tool for adults with oncological diseases and the call for research including smaller hospitals, as existing evidence primarily comes from tertiary university medical centres (18, 34).
The final version of POTT reflects the contextual requirements of paediatric oncology through its triggers, definitions and, in particular, its decision support information. Throughout the development process, the content was informed by considerations important to paediatric oncology. These contextual influences are especially evident in the decision support descriptions, where examples and guidance were shaped to capture AEs and no-harm incidents relevant across inpatient, day care and home healthcare settings in paediatric oncology. Our development process is distinguished by its robustness and systematic nature. We describe this process transparently, following recently published guidelines (38). By adhering to the ACCORD guidelines, we provide insights for others developing trigger tools. Previous descriptions of paediatric trigger tool development have varied in detail. Some authors mention modifying existing tools without explaining how or why (48–51). Others describe adaptations but not the rationale (14, 52). Some mention using a modified Delphi method but lack transparency about the process (11, 53, 54). These inconsistencies might detract from the value of the developed tools (44). Our development process, and the transparent description of it, builds on the robust methodologies used in developing the Global Assessment of the Pediatric Patient Safety Tool (13), the Pediatric All Cause Harm Measurement Tool (12), the Canadian Paediatric Trigger Tool (16) and the Swedish Paediatric Trigger Tool (15), and benefits from recently published guidelines (38, 39).
The initial literature search grounded our preliminary POTT used in the Delphi process in existing research. We searched not only for trigger tools, but also for AEs of relevance for paediatric oncology. This work, combined with the clinical experience of the steering committee, ensured the tooĺs relevance and is consistent with the preliminary trigger tool remaining relatively stable through the Delphi rounds. We did not conduct a systematic literature search, as relevant compilations were already available from previous studies by the last author (15, 22).
The relative stability of the preliminary trigger tool across the Delphi rounds appears to reflect the strong foundation provided by the initial literature search and the high level of agreement among the Delphi experts. Consequently, discussions mainly focused on clarifying definitions and ensuring clinical applicability rather than questioning the inclusion or relevance of the triggers. Discrepancies did arise for triggers requiring contextual interpretation across care settings, such as “fall” and “drug that requires follow-up” where relevance varied between inpatient, outpatient and home healthcare settings.
The Delphi approach is a robust methodology for achieving consensus among experts with diverse skills. The Delphi methodology has been adapted in various forms, leading to debates about certain principles, such as expert selection, anonymity, and the number of rounds (43, 44). We describe our Delphi process thoroughly, in alignment with reporting guidelines (38). This is often lacking in other studies, which might hinder the interpretation of the results (44). Similar to other developers of trigger tools (12, 15, 16, 42), we found the Delphi method useful.
There is no precise guidance on the size or definition of an expert panel (43, 44), although 20–30 experts in a panel are common (38). Heterogeneity in a panel may lead to better performance than homogeneity, and all relevant stakeholders should be represented (44). Our number of experts and their diverse backgrounds made it possible to obtain a wide range of perspectives relevant to the development of the POTT, while still allowing the practical implementation of the Delphi rounds. The steering committee selected the potential experts starting from the criteria that the experts would represent relevant stakeholders and possess relevant expertise. The steering committeés strong connections within relevant stakeholder organisations ensured good knowledge of potential experts. The expertś self-assessment of their experience in paediatric oncology, trigger tool methodology and patient safety indicates that the Delphi panel consisted of experienced experts. Only Swedish experts were included, which can be seen as a limitation. However, paediatric oncology is characterised by international collaborations regarding research, treatment regimens, working methods and professional training, suggesting that generalisation of the POTT from an international perspective and context is possible. This suggestion is further strengthened by the fact that the literature search that formed the basis for the tool included literature from various countries.
Depending on the studýs objective, including patients or patient representatives in the expert panel should be considered (44). Patients and parents are integral to paediatric oncology care, especially with the shift towards home-based care (3, 23, 55). To ensure a comprehensive perspective, parents were invited to participate in writing in the first Delphi round. Their perspectives complemented those of the professional experts by underscoring aspects of safety that may be more apparent to parents than to healthcare professionals, including psychological distress, symptoms associated with medication withdrawal and safety risks arising from gaps in care coordination. However, they were not included in the virtual meeting to avoid inhibiting open discussion of patient safety risks. This decision means some perspectives may be missing. In hindsight, patients and parents would have been involved to a greater extent, reflecting increased readiness and openness to fully involve patient representatives in research (56). Increased patient and parent involvement could further strengthen the understanding of patient safety risks and needs in paediatric oncology, particularly in home and transitional care context.
There is no precise guidance in the literature on the number of Delphi rounds. Ending the process too early risks obtaining invalid or meaningless results (43, 44). We used a model where the same experts, except for the parents, were invited to the first two rounds. The medical record reviewerś feedback about the POTT weighed heavily in the development process, therefore, the third Delphi round was primarily aimed at discussing, refining and anchoring the POTT with stakeholders connected to national networks and organisations. Accordingly, a purposefully selected smaller number of experts from the first rounds, representing these stakeholders, were invited to the final round. The thoughtful selection of invited experts ensured that the final round's purpose was achieved, although the low number of experts can be seen as a limitation. The smaller number of experts in the third round also mitigated the risk of decreased engagement, which can occur if the Delphi process is overly complex or time-consuming (43, 44). Ultimately, each expert was provided with the final version of the POTT.
There is no precise guidance in the literature on how to assess, manage and relate to consensus. A lack of clarity regarding consensus can impair the value of any study (43, 44, 57). Our transparency in assessing and managing how the level of consensus informed the trigger tool development is a strength. The level of consensus reached was not entirely decisive for whether triggers were retained or removed. Ultimately, these decisions were made by the steering committee, which might be perceived as a limitation. However, this modification of the original Delphi method is common and can improve the validity of the developed tool (43, 58). The steering committeés decisions were guided by clinical relevance, patient safety considerations and methodological coherence. The committee reviewed quantitative results together with qualitative comments, and professional judgement was applied to ensure that the final set of triggers was grounded in the study findings, informed by previous research, and supported by collective expertise within paediatric oncology and patient safety. The role of the steering committee also informed our decision to only give feedback on the latest version of the POTT before round two, not including the expert group´s and individualś ratings, which might be seen as a limitation. However, feedback was given in the last round.
The combination of virtual meetings and web-based surveys allowed us to introduce the study to the experts and helped them become familiar with it, which probably increased their commitment, participation and knowledge regarding the study (43, 44). Preservation of the expertś anonymity is a basic requirement in the original Delphi method, aiming to reduce the risk of excessively dominant individuals influencing the result, but adding meetings is a common modification (44). The facilitators were aware of the risk of a few experts dominating the discussions at the expense of others and actively counteracted such a situation by ensuring that everyone was involved in the group discussions. The feedback on the Delphi activities did not indicate any substantial concerns regarding the virtual meeting format.
Despite the technological development enabling virtual meetings and their explosive increase, there is a lack of scientific guidance on how virtual consensus meetings are best conducted (59). The use of a digital platform made it possible for busy and geographically dispersed experts to participate in the same meeting. The lack of scientific guidance was compensated by the steering committeés extensive experience in conducting virtual meetings in various settings.
A Delphi process alone may be insufficient to ensure the validity and reliability of the developed tool and may need to be supplemented with pilot tests or focus groups (43). In this study, involving actual users in the tool's development yielded invaluable insights into their priorities, thereby ensuring that the tool remains responsive to their needs, while relevant information and guidance was offered during the review process. The reviewerś experiences of participating in the development and use of the POTT are further elaborated in a complementary qualitative study (45), which provides a broader perspective on the development process described here.
Conclusion
Implications for practice
This study provides a novel POTT, complete with triggers, definitions, and decision support information. We anticipate that this context-adapted patient safety tool can be utilised in local patient safety initiatives to detect AEs and no-harm incidents in paediatric oncology. Its use in practice may be supported by local strategies such as opportunities for reflection within existing patient safety structures. By gaining new insights into risk areas for children with cancer, the groundwork can be laid for developing safety strategies aimed at enhancing patient safety in paediatric oncology, supported by mechanisms that allow findings to inform both ongoing and emerging safety improvement work, which represent important considerations for the tooĺs use in clinical practice.
Additionally, this paper provides a transparent description of a systematic development process and offers insights for further research.
The paediatric oncology settings in parts of the world where treatment is given according to strict international treatment protocols have much in common. Therefore, we anticipate that the POTT can be utilised internationally, even though the tool was developed in Sweden.
Further research
In subsequent publications, we will present the performance and the validation process of the POTT, along with the identified AEs and no-harm incidents and their clinical implications. These forthcoming publications align with the aim of the multicentre study PaSPO, of which this study forms a part.
The interest in developing and using semi- or fully automated AE detection methods is increasing, since less time and personnel resources are required and real-time feedback as a basis for rapid interventions can take place (60). We anticipate that the POTT could be the basis for research and development of automated methods for the detection of AEs in a collaboration between clinicians, patient safety experts and experts in natural language processing.