Modern consumers expect extended service and consumer-centric behaviors from salespeople (Lassk et al., 2012). Businesses focus on keeping the quality of the sales by investing in training (Kodwani and Prashar, 2019). Firms strive to meet consumers’ escalating expectations and manage organizational resources to provide quality service (Shabani et al., 2017). Managers expect the training to result in several returns, such as salesforce motivation and sales competence (Kauffeld and Lehmann-Willenbrock, 2010; Panagopoulos et al., 2020), leading to overall performance enhancement and goal achievement (Salas-Vallina et al., 2020). Sales training is the most frequent and universal intervention to improve sales productivity or customer orientation (Singh and Venugopal, 2015; Kodwani and Prashar, 2019). Effective sales training interventions may enhance sales organizations’ knowledge, skill, and performance (Lichtenthal and Tellefsen, 2001; Singh et al., 2015).

The most common challenges that organizations undergo are (1) constant changes in salespeople’s role and definition, (2) emphasis on accountability and reliability of sales function, and (3) reskilling and upskilling of salesforce facing new technologies (Lassk et al., 2012). However, one of the less explored issues is the (1) understanding of differences in training methodologies and (2) measurement for training evaluation and effectiveness (e.g., Lupton et al., 1999; Attia et al., 2021). This article discusses two primary challenges in the following sections of the literature review, namely, training methodologies and the training evaluation/effectiveness measurements.

Using technology in training must conform to the changing environment (Hartmann and Lussier, 2020). Digital transformation of salesforce pursued by firms has been accelerating (Guenzi and Nijssen, 2021) due to the merits, such as elevated efficiency and effectiveness (Zoltners et al., 2021). Online learning is now replacing traditional, instructor-led, and offline settings, becoming a significant corporate learning and development element to upskill employees based on a lower cost (Lands and Pasha, 2021). Advanced technologies adopted in the learning management system provide a flexible learning environment (Traxler, 2018), and technologies such as gamified learning could affect users’ flow and continuous usage (Kim, 2021).

However, a single method (purely online or purely offline) might be challenging to satisfy all learners. Instead, by combining online and offline, learning satisfaction could be increased (Mantyla, 2001). Integrating technology with traditional methods has raised the interest in “Blended learning” (Hrastinski, 2019). Blended learning combines traditional methods with technology-based learning and avoids the demerits of offline and online learning (e.g., Driscoll, 2002; Graham, 2006). At present, the terms blended learning and hybrid learning are interchangeably used (Watson, 2008; Graham, 2009) and are regarded as one of the most suitable modes for training in the digital era (Liu et al., 2020). Consequently, technology-blended learning could lead to best practices for businesses (Singh and Reed, 2001; Driscoll, 2002) since the learners’ cognitive process and the training/teaching strategy might change (Liu et al., 2020).

This study takes the empirical case from a sales training program conducted by Company A in South Korea. The firm is the regional headquarters of a multinational automotive company. The firm’s overall HRD system is shown in Figure 1. The sales training program, which is the main interest of this study, is named sales consultant training and education program (STEP). The company had difficulties conducting face-to-face sales training during COVID-19. Consequently, the firm decided to implement salesforce training using digital technology. In this respect, two alternatives, namely, structured pure online training and hybrid training that mixed offline and digital, were developed and applied.

Measurement of training outcome contributes to the continuity of training and program improvement (Kirkpatrick and Kirkpatrick, 2006). Training evaluation is an act of evaluating the value or quality of training elements systematically and scientifically. The expectancy theory by Vroom (1964) supports the views of prior researchers that the training’s perceived value or trainees’ beliefs (valence) about the training outcome would be crucial to training success (Colquitt et al., 2000). If the trainees get motivated, their job performance may increase with the feeling of accomplishment, which may also provide the potential for their future growth (e.g., Hanaysha and Tahir, 2016; Ibrahim et al., 2017; Wolor et al., 2020).

One of the most widely accepted performance indicators is the transfer performance of learning (Leach and Liu, 2003). Changes might occur when the employees transfer what was learned to their workplace (i.e., Ployhart and Hale, 2014). Researchers have insisted that training efforts will not bring the desired outcomes if a proper transfer does not follow (i.e., Wilson et al., 2002). Extant training literature has argued that learning transfer is a concern for organizations (Friedman and Ronen, 2015). Learning transferability has been a long-time researched topic in educational psychology (Hung, 2013). Adapting acquired knowledge in changed or modified forms in diverse situations to complete a given task or solve problems based on cognitive processes roughly defines “learning transfer” (Hung, 2013). Training evaluation and training effectiveness have drawn significant recognition in the literature (e.g., Broad and Newstrom, 1992; Kraiger, 2002; Noe and Colquitt, 2002; Holton, 2003). Although two terms, training evaluation and training effectiveness, are interchangeably used, they are independent of each other (Alvarez et al., 2004). Training evaluation means the appropriateness, relevance, and usefulness measured by the reaction to training content and design items. It is more of a “measurement technique” to check if the training programs meet the goals as planned (Alvarez et al., 2004). In contrast, training effectiveness investigates the variables that might affect training outcomes throughout the process (pre/during/post) and could be estimated by transfer measures and post-training attitudes (Alvarez et al., 2004). Several conceptual models (e.g., Tannenbaum et al., 1993; Holton, 1996) were suggested to measure training effectiveness. In summary, training evaluation is a methodological approach for measuring training outcomes, while the latter is a theoretical effort to understand the causalities.

However, the studies dealing with the various effects of variables and timewise training outcome comparison are limited (Grossman and Salas, 2011; Massenberg et al., 2017). As previously discussed, the evaluation of extended training intervention adopting technology should be able to (1) measure transfer performance adequately and (2) explain training effectiveness that could capture the states of trainees.

Researchers have tried to extend the existing models to measure the training outcome in the literature, and the CIPP is one of the most frequently referenced models (e.g., Adedokun-Shittu and Shittu, 2013; Finney, 2019). CIPP is a comprehensive framework for evaluation aimed at long-term, sustainable interventions (Stufflebeam, 2004), standing for context, input, process, and product evaluation. CIPP reviews four areas of training (Figure 2). Context evaluation focuses on the overall objectives. Input evaluation concentrates on resources (infra, curriculum, content, and material), while process evaluation sees the teaching–learning process or other activities. The attitudinal and behavioral changes are regarded as the outcomes during the product evaluation. Generally, the CIPP is often sequentially applied following the orders of context–input–process–product assessment to confirm the accountability of a program and review the quality of each step (Stufflebeam, 2004). Using CIPP, the evaluators could grasp how well and effectively the training outcomes are achieved.

However, there are several limitations to the CIPP model. First, although it is a robust model to measure training outcomes, several situations would not allow smooth evaluation due to the complicated dynamics of stakeholders (Robinson, 2002; Tan et al., 2010). Second, it requires tremendous effort and resources, leading to slow evaluation. Third, CIPP is a framework for the evaluator, not the trainee, and might lack actual feedback from the trainees unless adequately applied (Tan et al., 2010). Finally, as pointed out in the previous section, the dynamic aspect of training “effectiveness” should be considered (i.e., Alvarez et al., 2004), including transfer measures and post-training attitudes. The CIPP model requires further updates and theoretical concerns to prove itself as a more robust framework.

This study suggests a new conceptual framework based on the CIPP model, attempting to close the theoretical gaps unveiled above. Then, the new framework collects and analyzes empirical data. The unique effectiveness model that (1) is more concise, (2) enables practical evaluation, (3) realizes trainee-centered evaluation, and (4) could measure learning transferability is presented in the following section.

The study includes the following analytical procedures. First, this study suggests a novel CIP-R model that overcomes the limitations of the existing CIPP model. Second, evaluation data for traditional/pure digital/hybrid training methods were collected from the sales training sessions administered in 2019 and 2020. Third, frequency analysis of the collected data and descriptive statistics for each variable are presented. Fourth, the three training methods are comparatively analyzed, and ANOVA confirms the statistically significant differences. Fifth, factors affecting the learning transfer are identified and compared by general linear model analysis. For all the statistical analysis, the jamovi ver. 1.8.2 software (jamovi, 2021) was used.

The primary goal of this article was to analyze and compare the different modes of sales training enabled by digital technology and examine the factors affecting training effectiveness. As discussed in the “Advancing the CIPP evaluation model” section, the article adopted CIPP as the base framework and attempted to provide a new effectiveness model, further improving its limitations.

For decades, organizations have shifted from satisfaction to next-level constructs to measure training effectiveness (Mattox, 2013). Kirkpatrick (1998) argued that organizations should avoid using one simple “satisfaction measure” to measure training’s value. The rest of the stages in Kirkpatrick’s model focus on more critical aspects of learning transfer, which is training effectiveness (Mattox, 2013). Researchers such as Phillips (2012) and Brinkerhoff (2003) also advocated the necessity of “beyond satisfaction” measures in the post-training stages.

Satisfaction and loyalty (recommendation) are two concepts that have become crucial constructs in modern management (Kristensen and Eskildsen, 2011). The positive causal effect of satisfaction on loyalty has been proved in the literature (i.e., Tuu and Olsen, 2010). Tracking promoters propose organizations a compelling way to measure loyalty (Reichheld, 2003), and promoters who recommend services or products to others are loyal enthusiasts (Reichheld, 2006). This basic idea of focusing on recommendations is borrowed from the net promoter score (NPS) concept by Reichheld (2003). NPS measures the “recommendation” by the customers’ rating toward a simple question asking if they want to “recommend” the service or product they experienced to other close people such as friends, family, or colleagues, using a 0- to 10-point scale. The rating, which is believed to present loyalty, helps divide the respondents into three groups, namely, promoters (9–10), passives (7–8), and detractors (0–6). Considering recommendation factor in training is now commonplace across organizational units and utilized frequently as a valuable predictor and effectiveness measure for HR practices (Mattox, 2013) since it is not the driver but an outcome of training. Certainly, if trainees learn new skills and knowledge, they might want to apply the learning to their jobs and encourage others to do the same when successful. In this respect, recommending a training program could be a strong predictor of learning transfer.

Based on the above discussions, the existing CIPP model was reviewed and redesigned regarding the following two issues. First, the new framework should endorse digital training’s particular characteristics, enabling an apple-to-apple comparison. In this respect, the new framework excluded spatial factors to reflect the digital training’s characteristics and offline-only elements from the measurements. Second, ambiguous and almost-duplicative factors should be removed or simplified for clarity. For example, excessively detailed measures for context (e.g., objectives, goals) and input (e.g., content, curriculum) evaluation in CIPP may confuse the trainees while evaluating. As a result, a new training effectiveness framework, the CIP-R model, was presented (Figure 3). CIP-R is an acronym for content, instructional design, programmed learning, and recommendation. Recommendation measures loyalty. It was adopted for learning transferability toward individual and group levels, which is the key to training effectiveness. As discussed in the “Theoretical background” section, the C-I-P could be viewed as the training evaluation, and the R (recommendation) could be understood as a training effectiveness measurement.

As explained in the “Digital technology in a corporate learning context” section, the study takes the empirical data from STEP (sales consultant training and education program) of Company A, based in South Korea, an importer of global premium and mass automobile brands. Three HRD experts in automotive sales training and two Ph.D. researchers participated in the analytical procedure. Sales training data from March 2019 to October 2020 were used. Trainees assessed all training sessions through an online survey. To ensure unbiased evaluation, the survey did not ask for personal information. The collected survey results were sent to the firm’s training experience management system. The measurements of the CIP-R framework are shown in Table 1, and the 5-point Likert scale was applied to all items. STEP is a structured program designed to maximize the competence of sales consultants and aims to pursue customer delight through quality sales. The training is a 3-day, off-site, traditional classroom-type program. Due to the suspension of training during COVID-19, the company shifted to pure digital (from April to June 2020) and hybrid training (from July to October 2020) mixed online and offline.

In the traditional training conducted in 2019, 325 trainees participated in the 3-day session. Offline-based lectures, discussions, role-playing, and problem-solving sessions were implemented. A total of 140 sales executives participated in the pure digital training. Lectures and interactions were not provided, and the course depended on the learner’s self-directedness. Hybrid training was a well-blended version of offline and online. A total of 118 employees participated, and real-time live streaming sessions were provided. After the online session, instructors visited the learners’ offices to check their competencies and conducted role-play and problem-solving evaluations. First, the instructors handed out various scenarios, including multiple car purchase scenarios for customers with differentiated needs. Second, they evaluated each personnel based on structured measurement scales and checklists to assess sales consultants’ capabilities expected to be enhanced throughout the prior online learning sessions.

Although the three methods differed in their implementation, there was no significant difference in each course’s learning objectives and curriculum. The traditional method was the highest in terms of cost, the hybrid type was medium, and the pure digital training was the lowest. The details of the three training modes and frequency table are presented in Tables 2, 3. Brand: premium brand salespeople 59.2%, mass brand 40.8%. Grade: certified (42.5%), expert (33.3%), pioneer (23.5%), master (0.7%). Training methods: traditional (55.7%), pure digital (24.0%), and hybrid (20.2%).

To compare the effectiveness of the three training methodologies, the evaluation results by training participants were analyzed. A total of 583 data instances were used for the analysis. The basic summary statistics results are as follows (Table 4). The mean value of the items was 4.266–4.501. The item with the highest value was the “Sales and service philosophy” item in content evaluation, and the lowest item was the “Practice and participation” item in instructional design. The results indicate that the training participants were relatively satisfied with the training content delivering the firm’s philosophy and were less content with participatory and practical learning settings. The mean of all variables was 4.419. The average values of each dimension of content, instructional design, and programmed learning were 4.471, 4.386, and 4.409, respectively, and it was found that the content dimension received relatively higher ratings than the other three dimensions.

Then, all variables were divided by three training methodologies, and group statistics were derived (Table 5). By comparing the mean value of each item, it was confirmed in which area the differences in the training methodology developed significantly. As for the “recommendation” item that indicates training effectiveness, the traditional method appeared to be the highest (4.523), and the pure digital method was the lowest (4.093), while the hybrid method was in the middle (4.339). In the first half of 2020, when all the training interventions shifted from traditional to pure digital due to COVID-19, the item where the trainees’ rating decreased the most was the “Lecturer and instructor engagement” in the instructional design dimension (traditional = 4.686, pure digital = 4.086, difference = −0.600). The second highly impacted item was the “Duration of training” in the same dimension, showing a steep decrease (traditional = 4.560, pure digital = 4.114, difference = −0.446). The item that showed the slightest change was the “applicability” item in the programmed learning dimension (traditional = 4.520, pure digital = 4.386, difference = −0.134). In the second half of 2020, training shifted from pure digital to the hybrid method, and the overall ratings were improved. The “Lecturer and instructor engagement” item in the instructional design dimension indicated the most improvement (pure digital = 4.086, hybrid = 4.636, difference = + 0.550). The next most improved area was the “Practice and participation” in the instructional design dimension (pure digital = 3.971, hybrid = 4.263, difference = + 0.292). The item with the most negligible difference was “Confidence and overall competence” in the programmed learning dimension (pure digital = 4.286, hybrid = 4.364, difference = + 0.078).

Finally, an ANOVA was conducted for each variable. The result displayed that there is a statistically significant difference stemming from training methodology in all items except for two, “Confidence and overall competence” (F = 3.009, p > 0.05) and “Applicability” (F = 2.264, p > 0.05) in the programmed learning dimension (Table 6). Additionally, post hoc test results were provided to check the significant mean differences found in the variables based on the training methodology (Table 7).

Then, the study verified the results of the ANOVA omnibus test and the fixed-effects parameter estimates result (Gallucci, 2019) to confirm the variables’ effect sizes on training effectiveness and the coefficient for each term (Tables 8, 9). The three sales training methods—traditional, pure digital, and hybrid—showed statistically significant differences (F = 4.882, p < 0.01). The result confirmed a statistically significant difference between the pure digital method and the traditional training effectiveness (B = −0.121, p < 0.01). The difference between the hybrid training and the pure digital method was not statistically significant (B = 0.040, p > 0.05). The existence of five factors affecting the training effectiveness (recommendation) was confirmed. “Lecturer and instructor engagement” (F = 17.783, B = 0.134, p < 0.001), “Duration of training” (F = 27.798, B = 0.162, p < 0.001) in instructional design dimension and “Personalized learning” (F = 9.231, B = 0.121, p < 0.01), “Confidence and overall competence” (F = 52.304, B = 0.330, p < 0.001), and “Skill level for the job” (F = 11.221, B = 0.168, p < 0.001) in programmed learning dimension confirmed the clear difference by methodology and appeared to affect the dependent variable in a statistically significant way.

Finally, estimated marginal means of the dependent variable were calculated and compared for each training method to compare training effectiveness, reflecting the influence of each term included in the model. The result indicated that the traditional method has the highest effectiveness (mean = 4.429). The effectiveness decreased in the pure digital method (mean = 4.308) and then recovered after shifting to the hybrid training method that blended both (mean = 4.348). The result is presented in Table 10 and Figure 4.