To answer this research question, we conducted an online cross-sectional survey. We sent an email through SurveyMonkey to 24,000 teachers in Chile who had at least 3 months of experience accessing the evaluation module in Lirmi to participate in the study voluntarily. At the start of the questionnaire, participants were required to provide informed consent to participate in the survey. The participants were provided with detailed information in the informed consent regarding the research objectives, their involvement in it, their voluntary participation, the measures taken to ensure the security and confidentiality of their data (such as storing it in a secure repository accessible only to researchers), their right to withdraw their data before study publication, and their right to choose not to answer the survey questions without facing any consequences. The time frame for answering the survey was 3 weeks, from the last week of May 2023 to the second week of June 2023. The survey content is related to the constructs described in Table 1.

From the 24,000 invitation emails sent to teacher’s users of Lirmi, we collected 419 responses, but 100 were excluded from the analysis when the questionnaire completion was less than 100% to avoid missing values in the sample. The final sample comprised 319 teachers for all variables, except for sex, which has two missing values. Among the participants, 62% were women and 38% were men, from different ages, years of experience, and grades (see Table 2).

Regarding sample size adequacy for Structural Equation Modeling (SEM), no universal rule exists, as the required N depends on model complexity, number of estimated parameters, number of indicators of latent variables, effect sizes, indicator reliability, and the values of other parameter in the model (Wang and Rhemtulla, 2021). General guidelines suggest a minimum of 200 cases for SEM models with latent variables (Kline, 2023). While our sample size meets this conventional recommendation, we acknowledge that larger samples could enhance power and model stability. This limitation is discussed further in the limitations section.

To measure the variables to predict teachers’ use of the evaluation module in Lirmi, we designed a survey based on previous research (Ajzen and Schmidt, 2020; Chow et al., 2012; Davis, 1989; Venkatesh et al., 2003) and adjusted to fit the Chilean context and the technology we are focused on. To ensure the participants’ comprehension of the items, three Chilean in-service teachers checked the language, clarity, and comprehension of the items. Minor wording modifications were made based on the feedback received. Teachers responded to the survey using mostly agreed-upon Likert scale (1 = strongly disagree, 2 = disagree, 3 = neither disagree nor agree, 4 = agree, and 5 = strongly agree). The only item that use a different scale was “In a typical month, how often do you use the evaluation module?,” where the response options were about frequency of use (1 = Never; 2 = Once a month; 3 = Two to three times a month; 4 = Once a week; 5 = More than once a week) (see Table 3 for the complete list of items for each variable).

Following previous studies in the field (Scherer et al., 2019), the data analysis for this study was conducted using structural equation Modelling (SEM), a set of procedures that allows for the simultaneous examination of relationships between multiple observed and latent variables (Kline, 2016). We used Covariance-Based SEM because it is particularly well suited for our research aims, as it enables the assessment of both direct and indirect effects among the variables included in the extended version of the TAM while accounting for measurement errors and controlling for potential confounding factors (e.g., contextual factors). The analyses were performed in R Studio (R Core Team, 2024; version 2023.03.1 + 446) using the lavaan package (Rosseel, 2012; version 0.6–15).

Descriptive statistics were calculated for all measures to analyze the data and explore their distribution (Table 4). Second, we built a measurement model using Confirmatory Factor Analysis (CFA) to ensure the reliability of further analyses. We built a measurement model by assigning each item to its corresponding latent variable, as shown in Table 1. Third, we used modification indices to explore whether there was some residual covariance in the model that makes sense from TAM theory, so we could specify it and improve the model’s fit. The model fit was evaluated based on the suggestions of Whittaker and Schumacker (2022): χ² statistics (p > 0.05), comparative fit index (CFI; good fit >0.90), Tucker-Lewis index (TLI; good fit >0.90), root-mean-square error of estimation (RMSEA; adequate fit <0.08, close fit <0.05), and standardized root-mean-square residual (SRMR; good fit <0.05, acceptable fit <0.1). Fourth, we built a structural model based on the extended version of TAM, as shown in Figure 1. All analyses were conducted using R-Studio (version 4.2.2) with the lavaan package (0.6–15).

Even when the data were nested (teachers nested in schools), we could not perform multilevel analyses because there were few observations within each school. Consequently, the teachers were treated as not nested within schools.

Table 4 presents the descriptive statistics and Cronbach’s alpha coefficients of the latent variables. Regarding the means of the latent variables, CSE showed the highest score (4.09), whereas USE had the lowest average (2.92). Skew and Kurtosis are at the expected levels (below 2 and 3, respectively), according to Whittaker and Schumacker (2022), which means that the data are normally distributed. Moreover, all variables demonstrated high internal consistency, as indicated by Cronbach’s alpha values above 0.80. However, CSE had a slightly lower coefficient of 0.75, which is still acceptable (Taber, 2018).

We also report the Pearson correlation matrix for the latent variables in Table 5. All correlations were positive and significant (p < 0.001). Following the guidelines of Cohen et al. (2017), all variables had a medium or large positive correlation, except for CSE and USE, which had low positive correlations.

To assess discriminant validity, we follow suggestions given by Morrison et al. (2017) on using the Fornell-Larcker criterion (Fornell and Larcker, 1981), which states that each construct’s square root of the Average Variance Extracted (AVE) should exceed its highest correlation with any other construct. The results (Table 6) confirmed discriminant validity for most constructs. However, two exceptions were observed where the correlations were slightly higher than the AVE: PU and ATT (r = 0.862) and FAC and PEU (r = 0.796). Despite these results, we retained these constructs as theoretically distinct, as supported by previous research (e.g., Davis, 1989; Scherer and Teo, 2019). PU and ATT serve different functions in technology acceptance models: PU represents the platform’s instrumental value for completing tasks, whereas ATT reflects affective evaluations toward using the platform. Additionally, FAC and PEU are conceptually distinct, as FAC refers to external resources that support use, while PEU captures the individual’s perception of the effort required to use the platform (Venkatesh et al., 2003).

A structural model was constructed at both the measurement and structural level. Analyses were conducted using the ML estimator. We built a measurement model by specifying items belonging to each latent variable, as described in Table 1. The model showed an acceptable fit: χ²/df = 2.676, CFI (0.924), TLI (0.914), RMSEA (0.072, 90% CI [0.068–0.077]), and SRMR (0.046). We then used modification indices to improve the model fit. We included the items att_3 (“Using the evaluation module makes my work even more interesting”) and att_4 (“Is interesting to use the evaluation module”) as residual covariances because they have similar wording. After this adjustment, the fit of the measurement model slightly improved: χ²/df = 2.408, CFI (0.936), TLI (0.928), RMSEA (0.066, 90% CI [0.061–0.071]), and SRMS (0.046). We ran a chi-square test in both models to determine the model’s overall fit. The results showed a significant p-value, which suggests that the model did not fit the data well. However, it is well-documented that the chi-square test needs to be interpreted with additional fit indices (Whittaker and Schumacker, 2022), which in this case indicated that the model’s fit was acceptable.

Regarding CFA, all factor loadings were significant, positive, and above the thresholds Tabachnick and Fidell (2007) suggested (see Table 7). These results indicate that all items contribute to this factor (Yong and Pearce, 2013). In summary, our data support the measurement model, so we continued with the structural model (Finch and French, 2015).

Based on the measurement model defined above, we conducted a full structural model (Figure 2) to estimate the relationship between the latent variables.

The model showed an acceptable fit: χ²/df = 2.556, CFI (0.936), TLI (0.928), RMSEA (0.066, 90% CI [0.061–0.071]), and SRMS (0.046).

The data show that nine of the 12 hypotheses were confirmed (see Tables 8, 9). The USE of the evaluation module in Lirmi can be explained by both the behavioral intention (BI) (β = 0.196, p < 0.001) and attitudes (ATT) (β = 0.573, p < 0.001). This suggests that teachers with greater intent to use the evaluation module declare higher usage, and a more positive attitude towards using the evaluation module is associated with increased use of the module. Furthermore, we performed a path analysis to explore the indirect effect of ATT on USE mediated by BI. These findings indicate that the influence of ATT on USE is not only directly but also indirectly mediated by BI (β = 0.121, p < 0.01). This result highlights the role of BI as a mediator in the relationship between the ATT and USE. Considering the total effects, the combination of direct and indirect influences resulted in a significantly positive total ATT effect on USE (β = 0.694, p < 0.01). This finding implies that ATT exerts a substantial overall impact on USE directly and through its influence on BI.

The behavioral intention (BI) to use the evaluation module is strongly influenced by the attitude towards use (ATT) (β = 0.616, p < 0.001), and by the perceived usefulness (PU) (β = 0.224, p < 0.005). This suggests that the more positive an individual’s attitude and the more useful they find a technology, the stronger their intention to use it. Mediation analysis indicates that perceived ease of use (PEU) indirectly affects behavioral intention (BI) through its influence on perceived usefulness (PU) and the attitude toward using technology (ATT) (β = 0.075, p < 0.001). This statistically significant indirect effect supports the idea that PEU can shape BI by affecting perceived usefulness and attitudes toward technology. It should be noted that this effect is small. Regarding the total effects, indirect and direct influences resulted in a significant positive total effect of perceived ease of use (PEU) on behavioral intention (BI) (β = 0.691, p < 0.001). This suggests that PEU is a substantial predictor of BI, demonstrating that the easier the evaluation module is perceived, the stronger the intention to use it.

Attitude towards use (ATT) is also significantly influenced by perceived ease of use (PEU) (β = 0.427, p < 0.001) and perceived usefulness (PU) (β = 0.552, p < 0.001). In other words, teachers who considered the evaluation module easy to use and useful showed more positive attitudes toward using it.

Perceived usefulness (PU) was significantly influenced by subjective norms (SN) (β = 0.356, p < 0.001) and perceived ease of use (PEU) (β = 0.516, p < 0.001). Therefore, the perception of usefulness increases when teachers perceive that the evaluation module is easy to use and when they perceive others as significant for them (e.g., school leaders and colleagues) want them to use it.

The facilitating conditions (FAC) have a positive statistical influence on perceived ease of use (PEU) (β = 0.738, p < 0.001). This suggests that when facilitating conditions improve in their school or through the support provided by Lirmi, individuals perceive the evaluation module to be easier to use.

Table 10 highlights the significant explanatory powers of the variables in our model. Our model could explain 55% of the variance in teachers’ USE in the evaluation module using attitudes (ATT) and behavioral intention (BI). In addition, ATT and PU explained 67% of the variance in BI.

This study supports the applicability of TAM in a new context (i.e., the use of Lirmi digital monitoring system by teachers). Specifically, most hypotheses derived from the extended version of the TAM were confirmed, underscoring its robustness and versatility as a theoretical framework for understanding technology adoption behaviors in Learning Analytics.

Moreover, the findings suggest that some of the external variables hypothesized to influence Perceived Ease of Use (PEU) and Perceived Usefulness (PU) did not hold the same weight in our sample. For instance, the non-significant influence of Computer Self-Efficacy (CSE) on PEU and PU is notable and highlights the importance of exploring external factors that influence technology adoption. For instance, the results associated with facilitating conditions (FAC) are interesting because it means that perceived ease of use is malleable by appropriate support. This is related to the Barrier to Technology Integration Model (Ertmer, 1999), which states that external (like facilitating conditions) and internal (like attitudes) conditions for technology use are intertwined, and it is important to take into account both dimensions when identifying conditions for technology use.

We also confirmed a key relationship in the TAM literature: the positive relationship between PU and attitudes toward using the system. This validates the central role of perceived usefulness in shaping users’ attitudes towards technology.

The implications of these findings are relevant for both practitioners and policymakers. They emphasized the need to optimize the use of digital assessment data by increasing the perception of usefulness and fostering positive attitudes and intentions toward technology use among teachers. Further, understanding the importance of perceived usefulness (PU) and ease of use (PEU) can guide software design and professional development, ensuring that teachers perceive new tools as beneficial and straightforward to use. Moreover, the impact of subjective norms on perceived usefulness emphasizes the role of school leaders and peers in fostering a culture that encourages the adoption of digital assessment data.

In addition, the results invite school leaders and administrators to reflect on how they support teachers in adopting digital assessment data. Given the relationship between facilitating conditions (FAC) and perceived ease of use (PEU), it is important to design and deliver proper support for teachers when using digital assessment data. This can be done through teacher professional development, as we argued before, but also through peers acting as models for adopting new technology, starting with small steps in the adoption of the technology in the school to avoid overloading teachers with too much information.

This study has several limitations. First, our sample consisted of Lirmi users who declared high computer proficiency (mean = 4.07, sd = 0.83). Therefore, the results may not be generalizable to all users of Lirmi. Future research should ensure the sample is more diverse to represent a population with different perceived computer use skills. Second, the participants volunteered to participate in the study, which might have introduced self-selection bias. Third, while our model converged properly and presents acceptable fit indices, our sample size (N = 319) relative to the number of estimated parameters (82) results in an N:q ratio of 3.9:1, which is below the commonly recommended 10:1 or 20:1 ratio for SEM models (Kline, 2023). This may limit statistical power, particularly for detecting small effects or model misspecifications (Wang and Rhemtulla, 2021). Fourth, while the Fornell-Larcker criterion largely supported discriminant validity, two pairs of constructs (PU-ATT and FAC-PEU) exhibited higher than recommended correlations. Nevertheless, we retained these constructs separately due to strong theoretical distinctions and results from confirmatory factor analysis. Future studies should observe whether similar patterns emerge in different contexts, which could indicate a need for further theoretical refinement. Fifth, participant recruitment should be conducted at the school level in future studies for two reasons. First, this approach reduces self-selection bias, ensuring participation from a broader range of teachers beyond those who are highly self-motivated to participate. In addition, it increases the number of teachers per school, facilitating the use of hierarchical linear model. This statistical model, accounts for the nested structure of the data, addressing violations of independence assumptions, modeling random slopes and intercepts to allow the effect of predictors to vary between schools, and testing hypotheses about the influence of school-level variables on teacher-level variables. (e.g., leadership on teachers’ perception of usefulness). Finally, researchers could incorporate objective measures of technology use in future studies to corroborate the self-reported data and reduce potential bias (Lawless and Pellegrino, 2007).

First, future studies with a more heterogeneous sample of CSE scores could provide more information on its relationship with PEU. Second, more exploration is needed regarding the facilitating conditions teachers perceive as providing effective support and why. Is the training delivered by Lirmi or (in)formal support from peers in school? Moreover, do perceptions of effective support vary based on teachers’ experiences and competence in technology? This can help provide personalized and cost-effective strategies to support teachers on a large scale. Third, more variables can be included in the model, such as the voluntary use of digital assessment data, teachers’ age, and previous experience with technology (Venkatesh et al., 2003). Fourth, a longitudinal study would help follow teachers’ journeys on the platform over time, so it would be possible to shed light on how perceptions and usage patterns evolve. This could help design proper support for users at different stages in using digital monitoring system platforms. Fifth, rigorous teacher professional development interventions could be designed to improve teachers’ PEU, PU, and ATT to gain knowledge about how to change teachers’ beliefs to increase the adoption of digital assessment data.

This research can help in using digital assessment data in K-12 education. By understanding the factors that drive teachers’ adoption of digital monitoring systems, we can create a future in which data will be useful for improving teaching and learning.