Introduction
Cognitive Behavior Therapy (CBT) is effective across a wide range of mental disorders, e.g., depression (Cuijpers et al., 2013), psychotic disorders (Bighelli et al., 2018), and anxiety disorders (Cuijpers et al., 2016). With regard to psychosis, CBT has been recommended by several national guidelines [Germany: (German Association for Psychiatry, Psychotherapy and Psychosomatics (DGPPN), 2019); United Kingdom: (National Institute for Health and Clinical Excellence (NICE), 2019)]. However, in most countries, there are still implementation problems [e.g., in Germany (Schlier and Lincoln, 2016)], often due to a lack of therapists’ training (Heibach et al., 2014).
Best practice elements of CBT training have been identified in a review based on 35 randomized-controlled CBT trials (Rakovshik and McManus, 2010). Based on this review, a combination of theoretical instructions with experiential and interactive training elements (reflection on practice cases and role-play) with ongoing regular supervision is considered the “gold standard.” The inclusion of experiential and interactive training elements has been proven effective for delivering theoretical knowledge and improvement of therapeutic attitudes and behavior, which in turn leads to improved CBT outcomes (Beidas and Kendall, 2010).
Postgraduate CBT training in Germany does meet this gold standard and is provided by more than 230 institutes (Psychotherapeutenvereinigung, 2020). Students are both medical and psychological graduates. The mandatory requirement of 600 h of theoretical training courses is delivered by licensed and experienced CBT therapists in small groups (≤18 students) and usually includes lectures and experiential and interactive elements. CBT training is completed with a state examination and is regulated by state law (PsychTHG, 1998).
The German health care system relies heavily on regular enrollment of trained CBT therapists, as every patient diagnosed with a mental disorder according to the International Classification of Diseases (ICD-10; Dilling and Mombour, 2015) qualifies for therapy free of charge (25–80 sessions based on individual need). The ongoing demographic change makes this an even more pressing matter as 80% of the present CBT therapists will have been retired at the end of 2030 (Nübling et al., 2010).
At the beginning of the COVID-19 crisis in March 2020, Germany’s theoretical training courses came to a sudden halt as state laws prohibited gathering in groups. In response, most CBT training institutes and trainers broke new ground by switching to online teaching.
Concerning online learning/e-learning, there is a tremendous variety of different approaches, one distinctive feature of these approaches being the synchronicity of presentation and reception. Synchronous online courses are often virtual classrooms modeled similar to in-person-classrooms: lectures are directly held in web conferences, and training sessions are performed in the virtual classroom. Synchronous courses have some advantages, e.g., they are more familiar for learners, but also disadvantages, e.g., all participants have identical schedules, which might be especially hindering whenever different time-zones are involved. Besides, long web conferences may be somewhat stressful, as they make it harder for participants to perceive additional information such as non-verbal cues. This stress often leads to the currently discussed “Zoom fatigue” (Wiederhold, 2020).
In the purest form of asynchronous training, participants view video-lectures or read texts and train skills independently. Sometimes additional written tasks and quizzes are added to solidify knowledge gain. Participants can learn at an individual learning pace. Still, asynchronous online training often relies too heavily on self-learning and self-monitoring abilities that participants might not always have. Also, participants sometimes feel less connected with their classes and miss the sense of community (Anderson, 2008).
An exciting combination of online and in-person learning is “blended” learning that combines “face-to-face instructions with computer-mediated instructions” (Graham, 2006). One variation of blended learning that includes asynchronous and synchronous elements is the inverted-classroom method (Lage et al., 2000; Handke, 2012): a self-directed learning phase is performed prior to the regular classroom appointment. Students read a lecture or view an instructional screencast. The subsequent classroom appointment is used to answer questions on the content, to reflect on the gained knowledge, to transfer it to practice cases, or to train the newly acquired skills. The most crucial aim of the inverted-classroom method is to enhance the acquisition of more complex skills (analysis, synthesis, and evaluation) during classroom time. In contrast, the self-learning phase is used for lower cognitive processes (acquisition of knowledge) (Tolks et al., 2016).
Thus, this method seems to be a good fit for CBT training, as it involves both synchronous and asynchronous elements. In a recent meta-analysis on studies comparing in-person instructions and inverted-classroom methods in medical education, the inverted-classroom method was superior to traditional methods and associated with more significant academic achievement (examination scores) (Chen et al., 2018). Similar results were obtained in a meta-analysis assessing inverted-classroom methods vs. in-person methods in general academic education (van Alten et al., 2019). Interestingly, while courses presented in inverted-classroom arrangements are superior regarding performance of the enrolled students, students do not rate these courses as more satisfying and acceptable in comparison to courses held in traditional classroom formats (Roehling et al., 2017; van Alten et al., 2019).
With regard to CBT training, there is a consensus that online formats could improve the dissemination of CBT (Shafran et al., 2009; Fairburn and Cooper, 2011) and are an adequate alternative to in-person courses with regard to enhancing participants’ knowledge of the interventions as well as their skills as therapists (Sholomskas et al., 2005; Martino et al., 2011).
Nevertheless, only a small number of studies compared CBT online training courses and in-person courses directly. Stein et al. (2015) compared health care professionals who received an asynchronous self-learning online training (12 h) and an in-person-training (24 h) with respect to the regular application of strategies the professionals learned in both courses (and additional regular supervision). Results revealed that both courses were comparable with regard to the implementation of CBT strategies (notably as rated by therapists’ patients). Similar results were obtained in a direct comparison of synchronous CBT online training that was offered via an avatar on the platform Second Life and in-person training (Mallonee et al., 2018). Though differences between in-person and avatar training with respect to participants’ satisfaction were statistically significant, a vast majority (>90%) of participants were “satisfied “or “very satisfied” in both courses (Mallonee et al., 2018).
In summary, studies suggest that CBT online training courses might be as acceptable and satisfying as in-person trainings. Hence, online-training might not only be a safer alternative in times of a pandemic, but also an effective means of dissemination of CBT techniques whenever there is a scarcity of experts and long distances to overcome.
Still, there is a lack of studies that directly compare similarly designed in-person and asynchronous concepts of online CBT training courses regarding their acceptability and participants’ satisfaction. In addition to this, the trainer’s effects are often not controlled, despite the possibility that they could heavily influence satisfaction ratings (Ghosh et al., 2012).
Thus, the present study aimed to investigate in a quasi-experimental design whether satisfaction with online CBT training courses is non-inferior in comparison to in-person CBT training courses with the same content (CBT for psychosis), duration of training, comparable audiences, and an identical trainer at five CBT postgraduate training institutes.
Discussion
An asynchronous, blended, inverted-classroom online learning environment for CBT training (CBT for psychosis) was developed as an emergency solution to training provision during the COVID-19 pandemic. Our study investigated the hypothesis that participants’ ratings regarding acceptance and satisfaction of online CBT courses were non-inferior in comparison to the former in-person courses. For this purpose, we compared ratings of online courses and in-person courses with similar content, length, and trainer with regard to the participants’ satisfaction ratings, using a non-inferiority test (Lakens et al., 2018). Results revealed that participants’ satisfaction with the online courses was not relevantly lower in the categories information content, conception of content, didactic presentation, satisfaction with the trainer as a suitable role-model, working atmosphere, own commitment, and practical relevance than satisfaction with the in-person training courses. Concerning room for active participation and the course’s professional benefit, we could not exclude relevantly lower satisfaction ratings in the online courses. Nevertheless, a vast majority of online-participants rated these items as very satisfying or satisfying (81.4%, 83.9%).
With regard to satisfaction with online training courses for CBT therapists, to the best of our knowledge, this is the first study that directly assessed whether an asynchronous online training of identical length, trainer and topic is non-inferior to in-person training.
Our results are built on a study that compared various training courses of some evidence-based CBT techniques (e.g., Prolonged Exposure, CBT for depression) as synchronous online and in-person courses (Mallonee et al., 2018) in a large group of various mental health workers (e.g., therapists, social workers, and pastors). They used a more general measure of satisfaction (one item) and found a statistically significant difference (d = 0.35) between the groups. Though being a small-to-moderate effect in the Cohen classification (Cohen, 1992), this difference was not seen as an obstacle for the future use of synchronous online courses by the authors as participants’ ratings were still “satisfied” or “very satisfied” in more than 90% of the cases in both groups.
While synchronous online training cuts geographic connections between trainer and participants, times of learning and teaching remain similar in amount and schedule, posing obstacles to upscaling and dissemination of training e.g., into regions with differing time zones and a lack of suitable experts. Going one step further, we aimed to investigate whether loosening the time-bonds by adding asynchronous elements could still establish equal satisfaction.
Our positive answer for most of the items is in line with a meta-analysis that found no differences in satisfaction rates between in-person teaching in inverted and traditional classrooms (van Alten et al., 2019) in higher education and medical training (Chen et al., 2018).
However, we could not exclude the online training’s inferiority regarding the items room for active participation and professional benefit even though the number and content of exercises were equal in both online and in-person courses. With respect to the active participation of the audience, we assume that the differences are related to the reception of the pre-recorded lectures: participants could not pose their questions spontaneously during the lecture but were required to write them down. Posed later, the context of the question might not have been at the center of attention for the other participants. This might have framed the question as a special issue of the inquirer, leading to less engagement of the rest of the group in spontaneously emerging discussions of practical cases.
In addition, emerging discussions in web conferences are hindered by the fact that social cues such as eye gaze and body gestures cannot be used in an online setting to determine the audience’s degree of interest and adapt speech content towards it. The lack of these implicit gestures also leads to difficulties in signaling turn-taking in conversations (see Rossano, 2012 for more information). Furthermore, joining into the discussion is often hindered by the need to activate microphones. The resulting toll on participants’ concentration has been named “Zoom fatigue” (Wiederhold, 2020). From the trainer’s point of view, this might have impeded her from telling stories from everyday practice, which might explain the reduced professional benefit.
Summing up, the asynchronous, online blended learning solution had some practical advantages at relatively small costs in certain dimensions of satisfaction. Whenever there is a scarcity of shared room, shared time or specialized trainers on the spot, online CBT courses seem to be a feasible and acceptable solution.
Limitations
First, we are not able to report demographic data of the participants, due to the fact that our study was part of the regular CBT training in Germany in small groups (≤18): usually demographic data are not gained to prevent de-anonymization.
In addition, we collected the data via both an online form and pen-and-pencil questionnaires. Several studies suggest that online assessments are as representative as paper-and-pencil evaluations and yield similar results (Spooren et al., 2013). Nevertheless, in the online courses, drop-out rate was more pronounced (41%) in comparison to the in-person courses where we had data from all participants. Although drop-out rates are typically higher in online satisfaction assessments (Ardalan et al., 2007), it is possible that we only included participants who were more satisfied with the online courses. This also led to an unequal sample size in our analysis, however, we remedied this problem by using a robust computation of the confidence intervals, as recommended by Kottas et al. (2014).
Participants were not randomized to the CBT training conditions; thus, we are not able to exclude a cohort effect. Furthermore, both 6-point-Likert scales and 5-point-Likert scales were used and then combined for our analysis, which is not ideal, but did not affect the non-parametric analyses (also, ratings of 6 were not awarded on the 6-point Likert scales).
As our study was part of the regular CBT training for therapists in Germany, no objective measures of the degree of gained knowledge was included. Nevertheless, student’s satisfaction ratings are known to be reliable and valid (Linse, 2017), although the correlation between students’ satisfaction ratings and their learning outcome is small (Abrami, 2001; Eizler, 2002). However, in contrast to students in general, our participants had at least 6 years of previous experience in higher education and can thus be considered experienced in the evaluation of didactic presentations. Since the workshops prepared our participants for an important exam, it is plausible that the amount of gained knowledge influenced their acceptance ratings. Furthermore, no objective evaluation of the skills learned during the training was assessed by the trainer. Also, there were no data on qualitative satisfaction ratings available that would be helpful in evaluating students’ subjective experience.
Because of the satisfaction ratings’ subjective nature, the emotional background of the online courses is worth considering. We delivered the asynchronous online inverted-classroom courses during the COVID-19 pandemic as an emergency solution. Thus, participants were well aware that the trainer had voluntarily increased her effort at short notice to transfer the content into online courses. Satisfaction ratings of the participants might have been influenced by their gratitude for being able to continue their CBT training.
Furthermore, satisfaction ratings might also depend on the courses’ subject (CBT for psychosis) that might be considered more interesting than other topics. Also, some CBT courses might be more easily transferred to an online format in comparison to CBT courses that focus more on training of interventions (e.g., imagination techniques, chair work). Thus, the generalizability on other topics of CBT training is unclear.
Implications for Future Studies
With regard to the special situation of the pandemic discussed above, a first step would be to replicate our study in a non-pandemic situation as a pre-registered randomized-controlled study. It is also important in future studies to assess whether asynchronous online courses are non-inferior in comparison to synchronous online courses regarding satisfaction ratings and to use additional outcome criteria, e.g., to include assessments of gained knowledge, evaluation of gained knowledge by the trainer and to test participants on their newly acquired therapeutic skills.
In order to gain realistic, less pandemic-dependent data on satisfaction ratings, perceived advantages and shortcomings of the online setting can be assessed by the so-called “willingness to pay” paradigm (WTP) in economics. In this setting, scientists or CBT institutes might offer an online-course at the same time as the regular in-person course. Students would have to place a bid for online-participation, which can be either positive (they signal their willingness to pay an additional fee for the perceived advantages), or negative (they signal the amount of reduction they expect for the perceived restricted service). The online course is then sold to the half of the group with the highest bids at the price of the lowest bid in that half of the group (see Breidert et al., 2006 for a review of WTP-assessments).
Implications for CBT Training Post the COVID-19 Pandemic
Asynchronous online courses offer several advantages with respect to convenience and costs. Since candidates are usually employed in a full-time position at different locations and often have caring duties, a more individual schedule and no need for traveling might be appreciated by participants. Without the need for traveling and staying overnight, highly qualified trainers might be easier to find. Of course, this comes at the price of a more individualized learning experience, e.g., no informal meetings at the coffee station and diminished sense of belonging to a community of learners.
There are a number of potential solutions in order to give participants more room for active participation in online courses. One solution is to start the course with short “break out” exercise sessions in smaller groups. Participants often have less problems to engage in these smaller groups, as it is less important to mute their microphone when not talking (to prevent white noise). In these smaller groups, it is also easier to help each other to get used to the software.
In order to encourage participants to ask more questions, the trainer could start each web-conference with a short visual summary of the video lectures the participants were required to watch between the web conferences and could share the slides of her presentation. Also, she could motivate participants to ask questions either verbally or by writing down the question in the group chat.
In addition, participants were often interested in the trainer performing a therapeutic intervention role-play with a student role-playing as a client. These live role-plays in large groups can be improved if all students except the participants of the role-plays deactivate their cameras in order to enhance visibility of the participants. In addition, students can write down additional questions or suggest interventions for the trainer to read during the exercise in the group chat. Obviously, it is beneficial afterward to train the interventions in small groups in break out rooms, supported by the trainer. Also, playful quizzes or tests at the end of the courses could positively influence the effectiveness and attractiveness of online courses (Spanjers et al., 2015).