As defined in the study protocol of ImplementIT (31), the implementation study follows a mixed-methods design with quantitative (surveys, reporting data) and qualitative approaches (focus groups, open questions). The stepwise implementation determining the availability of the digital interventions across all federal states in Germany is illustrated in Figure 1. According to the Conceptual Model of Implementation Phases (42), the study can be divided into the phases “exploration”, “preparation”, “implementation” and “maintenance”. The evaluation is based on the established Consolidated Framework for Implementation Research (CFIR) (43), which has already been applied in a similar setting (44). The CFIR (43) unifies constructs from various implementation theories and offers a pragmatic structure to assess complex and interacting states in the implementation across five domains consisting of four to eight constructs. The first domain is related to characteristics of the implemented intervention and includes constructs such as “evidence strength and quality”, “relative advantage”, and “adaptability”. The second domain is the outer setting and includes constructs from the cultural, political, and structural context (e.g., “patient needs and resources”, “external policies and incentives”) that influences the implementation. The third domain includes aspects with regard to the inner setting (e.g., “networks and communications”, “culture”, “readiness for implementation”). The fourth domain is related to characteristics of the individuals involved in the intervention/implementation (e.g., “knowledge and beliefs about the intervention”, “individual stage of change”, “other personal attributes”). The fifth domain is the implementation process and includes constructs such as “planning”, “engaging”, and “reflecting and evaluating”. The study is described alongside Standards for Reporting Implementation Studies; see the StaR checklist (45) as supplementary.

The SVLFG advertises its health services through public relation (PR) activities. In addition, insured persons are also advised on the health services by field workers and in-house staff. During consultations at the SVLFG call center, insured people who meet the eligibility criteria of SVLFG receive detailed information on what prevention services they can access. They can choose between digital interventions (e.g., IBIs, TC) or group preventive services delivered onsite. The call center agents carry out the registration process. The digital mental health interventions that are provided free of charge for the insured are described in detail in the study protocol (31) and summarized below.

Employees from the following fields at the SVLFG were included in the study: •

Staff involved in the referral process who provide advice on and referral to health services including digital and onsite offers, such as field workers, in-house staff, and call center agents. Since the advice on health offers had to be newly introduced (field workers, in-house staff) or the previous advice had to be adapted or extended to include the digital offers (call center agents), these employees experienced changes in their daily work.

Employees were recruited from March 2018 to June 2021 for study participation while engaging in one of six kickoff events in the respective rollout area (e.g., for field workers, in-house staff as well as call center agents) at which the digital offerings and the accompanying implementation study were presented. Since the training of the pilot rollout employees has already taken place in 2017/2018, they were invited to the study via their supervisors (by e-mail and at internal meetings) in spring 2019. In total, 365 employees involved in the referral process have been invited for study participation and received an informed consent. The study team sent about two reminder e-mails to the respective group of employees to increase the study enrollment and response rate. In total, 86 employees involved in the referral process (86/365, 23.6%; n = 75 field workers and in-house staff; n = 11 call center agents) and seven implementation team members (7/7, 100%) signed the informed consent.

An overview of employee surveys and response rates from May 2019 until May 2022 per occupational group and rollout area is illustrated in Table 1. The time of the survey varied depending on the employee group and rollout area (consisting of 2–7 federal states of Germany). Data collection occurred at regular 6-month intervals from May 2019 to May 2022. Employees in the pilot area (Bavaria, Schleswig-Holstein) and Rollout Area A (Bremen, Lower Saxony, Saxony-Anhalt) were surveyed from May 2019 to May 2021, and employees from Rollout Area B (Hesse, North Rhine-Westphalia, Rhineland-Palatinate, Saarland) from November 2019 to May 2021. Employees from Rollout Area C (Baden-Württemberg, Berlin, Brandenburg, Mecklenburg-Western Pomerania, Saxony, and Thuringia) received delayed training due to the COVID-19 pandemic and were therefore surveyed only from May 2021 to May 2022. Therefore, the data collection period was extended from 2 years to 3 and a half years. Across all measurement time points and occupational groups, the response rates varied between 79% and 100% with a mean of 91.1% (SD = 18.8). In addition to the quantitative surveys, the implementation team was interviewed as part of a focus group on the current status of service implementation, their perception on the effectiveness of implementation activities, as well as their perceived barriers and facilitating factors. For the aim of this study, only the answers on barriers and facilitating factors were included. In total, seven focus groups were conducted and recorded by the study coordinator (JF) every 6 months during the implementation study. The duration varied between 42 and 105 min (M = 81.29, SD = 20.21).

Table 2 summarizes the 30 of 37 (81.1%) selected CFIR constructs across all five CFIR dimensions, the evaluation methods (surveys, focus group, or reporting data) used for the analysis and the source of items. The CFIR construct selection was based on the research team’s decision as to which constructs were relevant to the agricultural setting. Data from all the participants groups were included for most of the constructs. The surveys consisted of open and close questions from validated questionnaires used in implementation research, adapted items from the CFIR guide (48) as well as from other implementation studies based on the CFIR (44, 49, 53, 57–60) (see Table 2). “Readiness for implementation” among the implementation team as well as staff involved in the referral process was measured by using a German version of the Organizational Readiness for Implementing Change (ORIC; α = 0.88–0.92; 12 items; scale 1–5) questionnaire (54). “Leadership engagement” was captured from the staff-perspective with the Implementation Leadership Scale (ILS; α = 0.93–0.97; 12 items; scale 1–5) (55, 61) with regard to the implementation team as well as supervisors. The perception of the treatment credibility was assessed among staff involved in the referral process by using items from the Credibility Expectancy Questionnaire (CEQ; α = 0.84–0.85, 6 items; scale 1–7) (56) adapted to the interventions and the employees’ perspective within the construct “knowledge and beliefs about the intervention”. Single items from the Discriminant Content Validation (DCV; 32 items; scale 1–7) questionnaire based on the Theoretical Domains Framework Questionnaire (52) were used to assess “access to knowledge and information” (DCV domains: knowledge, training), “compatibility” (DCV domain: compatibility with current working activities), (DCV domain: “self-efficacy” (DCV domain: beliefs about capabilities) and “other personal attributes” (DCV domains: skills, optimism, intentions) among staff involved in the referral process. Single items from a German version of the NoMAD questionnaire (51, 62) were used to assess “available resources” (NoMAD item: contextual integration) and “individual stage of change” (NoMAD items: interactional workability, skill set workability).

Furthermore, we used reporting data on dissemination activities documented by the SVLFG to measure “engaging”. Open questions on barriers and facilitating factors in the implementation (see Supplementary Material) were asked from the second measurement time point (T1) every 6 months in the regular surveys given to staff involved in the referral process as well as to the focus groups with the implementation team whose responses could relate to all five CFIR dimensions.

Descriptive analyses were performed to assess outcomes across the CFIR dimensions. The outcomes within the dimension “intervention characteristics” as well as CEQ and DCV questionnaires and single items of the evaluation survey were evaluated separately for the internet- and tele-based interventions. All other outcomes were analyzed jointly. Items with a 6- or 7-level Likert scale (e.g., DCV items, CEQ items 1–3 and 5, items regarding relative advantage) were transformed into a 5-level Likert scale based on linear transformation (63). As in another CFIR study (44), scores of 4 or above were interpreted as positive aspects in the implementation, while scores below 3 were classified as areas for improvement, and scores in between were considered as neutral. However, this does not apply to the cost and complexity items, where higher agreement suggests higher cost or complexity. The analyses were performed per employee group. All observed data were included according to the intention-to-treat principle. Since the response rates (see Table 1) and data quality was relatively high in this implementation study, missing data was not imputed. The analyses were done with R statistic software (64).

Dissemination activities were categorized by two independent raters (JB, IW) according to the Cochrane Effective Practice and Organization of Care (EPOC) taxonomy (65). Each activity could contain multiple strategies. Inconsistent cases were discussed with the third researcher (JF) until all dissemination activities could be consistently assigned.

Answers to open questions on barriers and facilitating factors in the employee surveys were evaluated by two independent raters using a deductive-inductive approach with MAXQDA (66), a software for qualitative analysis. A code system with main categories based on the CFIR (deductive approach) and identified subthemes from the data (inductive approach) was created by a research assistant (JB) to match the data and developed further in feedback loops with two experts in e-mental health and qualitative research (JF, IT). As a text passage could consist of different aspects, it could be coded with several codes. The data were assigned to the code system and frequencies were calculated in relation to the mentions of the individual themes as well as per occupational group [i.e., (a) field workers and in-house staff or (b) call center agents]. The agreement rate between the two independent raters (JB, SM) can be characterized as high (k = 0.86) (67).

Based on the audio recordings of the focus groups minutes were taken for each focus group with the implementation team. Answers to open questions regarding barriers and facilitating factors were summarized and analyzed per focus group based on a code system following a deductive-inductive approach. The theoretically based superordinate categories formed the basic framework for the inductively identified topics. Frequencies of themes were analyzed at interview level. The analysis was performed by a research assistant (IW) in an iterative process with several feedback loops with input from clinical researchers (IT, JF).

Most of the 93 study participants were involved in the referral process as field workers and in-house staff (n = 75, 80.6%) as well as call center agents (n = 11, 11.8%). Seven employees were part of the implementation team at the insurance company (7.5%). On average, the participants were 49.29 years old (SD = 8.6), mostly male (n = 63, 67.7%) and worked for more than 10 years at SVLFG (n = 61, 65.6%). Most of the field worker and in-house staff had experience with consultations on health offers (N = 60, 69.8%), however only 31.5% (n = 29) of them had experience with consultations on mental health services. Further characteristics of the study participants are shown in Table 3.

In the following, the findings are summarized by the CFIR domain. Each CFIR domain is divided into qualitative and quantitative results. Qualitative findings on barriers (n = 20) and facilitating factors (n = 10) from the perspective of staff involved in the referral process are illustrated in Figure 2 and as Supplementary Table, including a definition and quotation of the identified theme. Barriers (n = 13) and facilitating factors (n = 14) from focus groups with the implementation team are shown in Figure 3. Quantitative results on CFIR items among the implementation team are displayed in Table 4, for employees involved in the referral process in Table 5. Descriptive differences between the two interventions as well as between the different types of employees are pointed out.

Intervention characteristics seem to have the highest positive impact on the implementation of digital interventions from the perspective of both the implementation team and referral staff. The relative advantage of digital interventions, quality of the interventions, and strength of evidence appeared to be facilitating factors in the implementation of the internet- and tele-based interventions in the agricultural setting.

In comparison, the outer setting, inner setting, individual characteristics, and implementation process were less consistently regarded by all staff groups as having facilitated the implementation of digital prevention services. Most of the qualitatively revealed barriers were related to the outer setting, in particular to “patient needs and resources”. The employees reported to have perceived from the insured, among others, inhibitions about using digital interventions and skepticism about offers for mental strain. At the same time, facilitating factors reflecting the acceptance of mental health services and the general interest of the insured were noted by staff involved in the referral process. Likewise, as indicated by the quantitative ratings for “patient needs and resources” as well as “external policy and incentives”, predominantly beneficial aspects for implementation could be achieved (e.g., consideration of needs and preferences of the insured during implementation; facilitating influence of regional, state or national policies, as well as legal requirements on the implementation).

With regard to the inner setting, many barriers referred to “available resources”, and “access to knowledge and information”, including a need for training and information on the interventions and further educational materials. Although several educational meetings were reported as dissemination activities and kickoff events and training sessions were conducted, there seems to be a need for further information and ongoing training for employees. However, many quantitatively assessed constructs of the inner setting (“tension for change”, “compatibility”, “relative priority”) also served to facilitate the service implementation from the perspective of health care workers.

Further barriers to implementation were identified in relation to the characteristics of individuals. Employees involved in the referral process mentioned having little routine in advising on health care services as well as difficulties in recognizing and addressing mental health issues. The results showed a difference in the agreement rates of the occupational groups. While call center agents evaluated most constructs with high agreements (M > 4.0), agreement rates in the group of field workers can be characterized as neutral for “knowledge and beliefs about the intervention” and “other personal attributes”, and as rather hindering for “self-efficacy”.

With regard to the implementation process, facilitating factors were identified in the focus groups with the implementation team, including a clear allocation of tasks and targeted PR activities to destigmatize mental health issues. In the quantitative ratings, facilitating factors related to active implementation support by champions and formally appointed internal implementation leaders and barriers regarding engaging and informing of employees about digital health offerings could be determined by the implementation team. 510 dissemination activities have been carried out so far, most of them in the field of educational meetings, local opinion leaders, and marketing, while others such as strategies like audit and feedback have hardly been used so far.

As there might be differences between the IBI program and TC, some constructs were recorded separately for each intervention. The results for both interventions are generally comparable with neutral to high agreement rates, with slightly higher agreement ratings in favor of the personalized tele-based coaching with regard to “evidence strength and quality”, “relative advantage” and “knowledge and beliefs” about the intervention, particularly evident in the perception of the implementation team and call center agents. However, the current power of the study as well as the unequal number in the groups did not allow for a significance test. As the results of the accompanying randomized controlled trials show (28–30, 34), both interventions effectively reduce depressive symptoms posttreatment.

When interpreting the results, it is important to note that, the COVID-19 pandemic occurred as an unforeseen and uncontrollable event, and the effects are reflected in several constructs. On one hand, kickoff events, exhibition stands, and educational meetings had to be cancelled due to the pandemic, which led to reduced onsite contact with the insured as well as less contact between field workers and the implementation team. On the other hand, digital interventions proved beneficial and important, and these continued to be offered during the pandemic, compared to onsite services. The implementation team indicated that the cancellation of events also eased time pressure, which in turn had a beneficial effect on the planning of new implementation activities. Furthermore, implementation activities could be adapted and planned onsite events were transferred in a digital format. However, the conduction of onsite events from 2022 was also experienced as beneficial by the implementation team.

In the study, different professional groups were interviewed and surveyed. It is particularly evident that there is often an overlap in experiences between professional groups, e.g., both the implementation team and the staff involved in the referral process reported many barriers in relation to the outer and inner setting. In many areas, the results between the professional groups complement each other well (e.g., referral staff who are in daily contact with the insured mentioned even more facilitating factors and barriers regarding “patient needs and resources”). Since the working areas of the employees and therefore the focus, experiences, and tasks in the implementation are different, this contributes to a comprehensive analysis of the implementation process.

Although CFIR is one of the most frequently used frameworks to examine implementation in diverse health settings (68), literature in the area of digital interventions to prevent depression is scarce. Hadjistavropoulos et al. (44) conducted a process evaluation based on the CFIR to understand facilitating factors and barriers impacting the implementation of iCBT within community mental health clinics. In line with their study, intervention characteristics were perceived as mostly facilitating the implementation. Similar to our study, the inner setting was identified as the most significant barrier to implementation due to limited resources for internet-based Cognitive Behavioral Therapy (iCBT) (44). In addition, a greater priority given to face-to-face psychotherapy was reported as a barrier to implementation (44), which was not found in our study. Another reported barrier in the implementation of iCBT was the need for even greater engagement of stakeholders in iCBT (44), which also applies for the prevention setting in this study.

Two studies about the implementation of IBIs for depression prevention in adolescents (37) or for depression treatment in adults (69) showed that primary care physicians or nurses perceive the preventive approach of internet interventions as valuable (37). The pilot study (37) reported that it is challenging to train the individuals to achieve a sufficient level of counseling competence, whereas the qualitative study (69) reported a lack of habit and routine and little knowledge about IBIs that could have hindered the referral process and led to low referral rates in primary care physicians. This is consistent with some of the barriers identified in our study related to staff involved in the referral process (e.g., need for training, difficulties in addressing mental health issues, little routine in advising on health care services).

In line with systematic reviews examining factors influencing the implementation of digital interventions (70, 71), our findings indicate that the implementation of digital interventions for depression prevention are influenced by diverse constructs instead of a single construct explaining implementation. Access to knowledge and information seems to be important for the implementation of e-health interventions (70), which was identified as a hindering factor in our study. While access to ongoing system support had a positive impact on implementation, Ross et al. (70) identified it as a barrier to implementation when it was lacking, which is consistent with identified aspects related to training and educational materials in this study.

Most of the identified hindering themes in our study are related to “patient needs and resources”. These barriers are in accordance with previous studies indicating that help-seeking behavior is lower and stigmatization of mental disorders is higher among farmers compared to other occupational groups (25). Furthermore, many of our identified barriers and facilitating factors are in line with findings of qualitative results of another substudy of ImplementIT (31) from the perspective of users (farmers who participated in digital services to prevent depression), including “location independence” and “anonymity” as positive drivers and a “lack of time” and “inhibitions about using the internet” as negative drivers for the acceptance of and satisfaction with the IMI program (17, 32, 33).

Because of the multifaceted aspects considered in the CFIR, it was a useful framework to explore and map the various contextual factors in the implementation of digital interventions. With the help of the CFIR, adaptations of the digital interventions as well as recommendations for the further implementation and scale-up can be derived. However, we have found an overlap between constructs of the inner setting and individual characteristics as individuals could also be considered part of the inner setting.

Additionally, previous implementation studies on digital interventions for depression identified a lack of methodological rigor and a need for validated questionnaires in the implementation of e-mental health interventions (72). Since no questionnaire exists in which all CFIR constructs are captured, this study provides considerations and initial starting points for data collection and analysis within a mixed-methods approach for other researchers who plan to evaluate the implementation of digital interventions.

Several measures for the further implementation of digital offerings among farmers can be derived from the results. Special attention should be given to addressing barriers regarding “patient needs and resources”. Counteracting the stigmatization of mental stress is a constant and ongoing process, which was also described as beneficial by the implementation team. When implementing technology in the area of prevention, it is often necessary to create an awareness that targeted interventions can increase well-being and significantly reduce the subsequent occurrence of depression. This could be strengthened through targeted press and PR activities, which also refer to the reported positive experiences of participated farmers, and the involvement of local stakeholders. At the level of the inner setting, it could be crucial to conduct (follow-up) trainings for employees in order to provide information about the interventions and current developments and to strengthen the exchange of experiences among each other. In addition to training, other ways of reaching employees at regular intervals should be considered to enhance access to knowledge and information (e.g., newsletter, regular online or onsite meetings). An extension of educational materials (e.g., videos) tailored to the needs of the respective occupational group could be useful to support the consultations. A further strategy according to the CFIR-ERIC Matching Tool (73), which matches barriers to implementation strategies, might be to “involve executive boards” (e.g., supervisors) in the implementation effort to enhance leadership engagement. In order to address barriers with regard to characteristics of the individuals, the trainings should also pay attention to how mental health topics could be addressed or which health offer is useful with the respective insured. Additionally, implementation strategies such as to “conduct ongoing training” for both digital interventions as well as to “make training dynamic” (e.g., by varying the methods of information delivery and making the training interactive to accommodate different learning styles) might strengthen “self-efficacy” as reported in the CFIR-ERIC Matching Tool (73). To enhance “reflecting and evaluating” implementation strategies such as to “develop and implement tools for qualtiy monitoring” as well as to “develop and organize quality monitoring systems” seem to be suitable (73).

For the field of depression prevention, the findings indicate that there are synergies when IBIs and tele-based coachings are implemented as technology-based interventions to complement on-site services. We could identify and address different and common barriers and facilitating factors, while also addressing different preferences and conditions in implementation. Offering different services (individual, group, on-site, tele-based, internet-based) centrally managed by a call center that coordinates inventions from different service providers allows the end-user a lower access threshold with maximum consideration of their needs and could solve the scaling problems observed in research (e.g., low uptakes) (74), where so far usually only one intervention has been implemented and considered. In addition, the findings indicate that it is critical to include, besides end-users, key gatekeepers such as health care workers and their experiences in providing consultations on digital health services as well as implementers at the social insurance company. Further stakeholders in the health care system (e.g., employees/psychologists of the external service providers, political representatives, decision makers of the social insurance) should also be considered, which is partly done in another study by ImplementIT (31) and will be published elsewhere.

Our results should be considered in light of the following limitations. First, a smaller sample size among employees than expected could be reached for participation. While 365 employees involved in the consultation process were trained, only 93 of them chose to participate in the study. Furthermore, selection bias may have occurred in that the participants were likely to be more open and motivated to the implementation of digital interventions (selection bias), and positive responses to satisfy the study team (response bias) may occurred as well, which could have resulted in a more positive assessment of service implementation than would have been the case with a representative sample of employees. Second, as no questionnaire exists that covers all CFIR constructs, a mixture of different questionnaires/items with various scales has been used in the study. Therefore, some of the scales had to be transformed during the evaluation in order to be able to present the results uniformly. Third, barriers and facilitating factors were captured generally and not formulated specifically with respect to each CFIR construct, which would have resulted in a larger output for all CFIR dimensions. However, this can also lead to participants responding more openly and less constrained by predetermined deductive-formulated questions. This can be an advantage because more facets outside fixed constructs are raised. Forth, it is unclear what impact the different assessment time points in the respective rollout areas with different characteristics (e.g., regional conditions, team structures, stage of implementation) might have. Fifth, the qualitative results refer to individual responses from employees and cannot be generalized due to the explorative approach.

Our findings contribute to an understanding of the facilitating and hindering factors in the implementation of digital interventions for depression prevention among farmers from the perspective of health care workers and are part of a complex implementation study design (ImplementIT) (31). The study presents that the use of various mixed-methods for data collection including surveys, open-ended questions, focus groups, and reporting data contribute a comprehensive understanding of the implementation process. While both offerings (guided IBIs, personalized tele-based coaching) are widely accepted by health care workers, the results also point to some barriers and contribute to deriving recommendations for further implementation. For example, to improve access to knowledge and information, as well as beliefs about digital prevention services, implementation strategies focusing on educational meetings for employees and distribution of educational materials tailored to the occupational group within the SVLFG might be helpful.