To assess levels of motivational components, researchers typically use well-established motivation inventories which were developed to measure motivation as a trait (Midgley et al., 1998; Simzar et al., 2015) or state (Vollmeyer and Rheinberg, 2006; Freund and Holling, 2011; Freund et al., 2011). Simzar et al. (2015) and other researchers (Arvey et al., 1990) have found inconsistent relations between trait motivation and test performance (Sundre and Kitsantas, 2004). Indeed, motivation while taking an achievement test is also conceptionally related to a person’s motivational state in that situation. One questionnaire measuring current motivational state is the Questionnaire on Current Motivation (QCM) (e.g., Vollmeyer and Rheinberg, 2006), which several studies (e.g., Penk and Richter, 2017) have used to disentangle the relationship between current motivation, including the dimensions of anxiety, challenge, interest, and probability, and test performance (Freund et al., 2011). Findings from studies using the QCM indicate relations at moderate levels between interest and test scores (Freund et al., 2011). However, one at least partial limitation is that the QCM asks about current motivational state in a general manner. A measurement method closer to the test situation is to ask test takers how they estimate their current motivation before and after taking an achievement test (e.g., Baumert and Demmrich, 2001; Eklöf, 2006).

Eklöf (2006) developed the Test-Taking Motivation Questionnaire (TTMQ), which includes motivational components in line with the expectancy-value theory of achievement motivation (e.g., Wigfield and Eccles, 2000). The relations between these components and test performance were at low to moderate levels, indicating inconsistent findings (Wise and DeMars, 2005; Knekta and Eklöf, 2015; Penk and Schipolowski, 2015; Penk and Richter, 2017; Stenlund et al., 2018). Moreover, Penk and Richter (2017) identified changes in the motivational component of test-taking effort during test taking. Test takers’ self-reported test-taking effort decreased from the beginning to the end of the test in this study (Penk and Richter, 2017) and in other studies (Attali, 2016; Lindner et al., 2018). Test takers may easily recognize that the TTMQ items are intended to capture their motivational state and might thus respond in socially desirable ways. Hence, it is valuable to increase the validity of the TTMQ results by employing less subjective measures (AERA et al., 2014).

Test-taking effort has been investigated by different measures, for example, response times (Wise and Kong, 2005), time on task (Attali, 2016), or self-reports (Knekta and Eklöf, 2015). A study compared test-taking effort (measured by time on task) and performance in a high-stakes achievement test vs. subsequent low-stakes achievement test with the result that the majority of test takers replicated their high stakes performance in the low-stakes condition with little effort (Attali, 2016).

Some researchers have used response times to test the assumption of low test-taking motivation reflected in low effort (Wise and Kong, 2005; Hartig and Buchholz, 2012; Debeer et al., 2014; Rios et al., 2014), examining persistence levels in terms of response times on puzzle tasks, or response times on anagram tasks (e.g., Gignac and Wong, 2018). Other studies included changes in response times over the course of an achievement test as indicators for test-taking motivation (Hartig and Buchholz, 2012; Goldhammer et al., 2017). Meta-analytic results suggested higher correlations between test-taking response time effort and test performance than self-reported effort assessed mainly by the Student Opinion Scale (Sundre and Moore, 2002) and test performance (Silm et al., 2020). Test-taking effort estimated using response times decreased over the course of test taking in these studies (Hartig and Buchholz, 2012; Debeer et al., 2014).

In summary, changes in self-reported effort over the course of test taking suggest decreased effort, which raises the question of potential strategies for keeping test-taking effort levels. The TTMQ, based on expectancy-value theory, captures current test-taking motivation (state), and is a widely used measure in large-scale assessments including students at school. Researchers examined and proposed strategies with the intention to increase German school students’ test-taking motivation but examined relatively seldom changes in test-taking motivation or strategies to keep the level of test-taking motivation in education students in Germany (Silm et al., 2020). Based on expectancy-value theory and above-mentioned evidence, we focused on two motivational components among test takers: (1) the test-taking effort invested and (2) the subjective test-taking importance of the respective task (value component), while also considering the other components that are expectancies, concentration, and anxiety, as well as gender and age, as described below in the method section. Test-taking effort and importance are probably at higher levels when test takers are working on mock exam tasks than on cognitive ability tasks.

Test-taking effort (Knekta and Eklöf, 2015) with regard to the current test situation was measured three times (T1–3) during the mock exam with five items: in the baseline assessment (T1), after the first task battery, and after the second task battery (T3). An example item is “I am doing my best on these tasks.” McDonald’s ω_total = 0.95 suggested good internal consistency for the three-factor solution and each factor (T1 ω = 0.87, T2 ω = 0.89, T3 ω = 0.89). Subsequently, items assessing test-taking importance were presented.

Test-taking importance (Knekta and Eklöf, 2015) was measured three times (T1–3) with the same three items: in the baseline assessment (T1), after the first task battery and test-taking effort items as well as after the second task battery and effort items (T3). An example item is “The tasks are important to me.” McDonald’s ω_total = 0.93 suggested good internal consistency for the three-factor solution as well as the factors test-taking importance at T1 and T3 each except T2 with only acceptable internal consistency (T1 ω = 0.87, T2 ω = 0.60, T3 ω = 0.97).

Moreover, McDonald’s ω_total = 0.87 suggested good internal consistency for the motivational five-factors solution incl. expectancies (ω = 0.63), anxiety (ω = 0.74), concentration (ω = 0.69), test-taking effort (ω = 0.87), test-taking importance (ω = 0.87) at T1 and acceptable internal consistency of these factors each. These five motivational variables were included according to the introduced theoretical model (see Figure 1). Subsequent to the test-taking importance items, expectancies (Knekta and Eklöf, 2015) were assessed with three items adapted to the current study (T1). An example item is “Compared with other students, I think I am doing well on the tasks.” Test anxiety was assessed with three items and presented before the first set of tasks. An example item is “I am so nervous when I take the tasks that I forget things I usually know” (adapted from Knekta and Eklöf, 2015, p. 666). Concentration (Arvey et al., 1990) was assessed with four items at the end of the baseline assessment (T1). An example item is “It is hard to keep my mind on this test.” Expectancies, anxiety, and concentration were included as manifest covariates only to consider their effects on the criterion variables test-taking effort and test-taking importance at T3 since the theoretical model and previous findings suggested such relations (Knekta and Eklöf, 2015; Silm et al., 2020).

Pioneers of psychology already tested and described cognitive abilities such as perception (James, 1884), reasoning (Piaget, 1928), and visuo-spatial perspective-taking (Flavell et al., 1978). Since perspective-taking is highly important for education students’ social interactions with children, adolescents, and adults (Wolgast et al., 2019), we chose proven cognitive ability tasks as typical tasks for research purposes in psychology. These cognitive ability tasks were considered as personally less important low-stakes tasks because they were not part of the lecture or module curriculum and irrelevant for the exam the students had to take in order to finish the course. Sixteen tasks assessed the cognitive ability visuo-spatial social perspective-taking that is seeing what another person sees by putting oneself mentally in the target’s spatial position (Kessler and Thomson, 2010; Erle and Topolinski, 2015).

Erle and Topolinski (2015) used the visuo-spatial perspective-taking paradigm developed by Kessler and Thomson (2010). Each of the first 16 tasks involved a photograph (with friendly permission from Thorsten M. Erle for using the photographs in further research). The photograph showed a female or male target person sitting at a round table (arms on the table) from a bird’s-eye perspective. A book and a banana lay on the table close to the person’s left arm and right arm, respectively, or vice versa. The person’s position at the table rotated from photograph to photograph between 120, 160, 200, and 240° from the participant’s point of view. Previous research has found perspective-taking to be difficult at these angles (Janczyk, 2013). Each position was presented with a female target person in eight photographs and a male target person in further eight photographs (16 tasks). Participants indicated whether the book was lying closer to the target person’s left or right arm by pressing “n” (left) or “m” (right) on the keyboard (“n” is located to the left of “m” on German keyboards). There was no time limit. All cognitive ability tasks were presented in German. McDonald’s ω = 0.98 suggested almost perfect internal consistency.

Twenty single-choice mock exam tasks were developed to coincide with a lecture for undergraduate education students entitled “Educational Psychology.” McDonald’s ω = 0.61 suggested acceptable internal consistency for these tasks. The mock exam tasks were considered as individually important low-stakes achievement test because the students’ upcoming module exam consisted of tasks of this type with similar content. Hence, the undergraduates had the opportunity to practice this type of task in order to be well prepared for the module exam. An example mock exam task is “Which phenomenon related to a child’s reasoning did Piaget and colleagues investigate with the three-mountain task? (a) object permanence, (b) centering, (c) egocentrism, (d) logical contradictions.” The tasks were presented in German; the example has been translated into English for current purposes.

We used latent growth curve modeling (R Development Core Team, 2009; Rosseel, 2010) and weighted least squares with mean and variance adjustment estimation (WLSMV) (Rosseel, 2019) to test for within-test-takers’ changes and differences in responses on motivational items depending on condition. We set the significance-level at α = 0.05. The variables included in the modeling were grand mean centered.

First, we conducted a confirmatory factor analysis (CFA) and tested the theoretical six-factor model (test-taking effort and importance at T1, T2, T3) by the data. The unstandardized effort factor-loading of the fourth item (Item “E4,” Knekta and Eklöf, 2015, p. 666, adapted to university: I spent more effort on this test than I do on other tests we have in university.) was λ = 0.21 and statistically not significant with p = 0.15 in the EG1. Consequently, we excluded Item E4 from further analyses. The final two factor CFA model included the latent factor test-taking effort measured by the respective four items and their residuals at T1, T2, and T3, and the latent factor test-taking importance measured by the respective three items and their residuals over the three measurement points. Scalar measurement invariance is a prerequisite for latent growth curve modeling. Measurement invariance was tested using the two factor CFA model in a multi-group analysis across groups and time points. This CFA model including constrained factor loadings suggested scalar invariance, Delta Comparative Fit Index (Δ CFI) = 0.004; Delta Root Mean Square Error of Approximation (Δ RMSEA) = 0.003, according to recommended cutoffs (Hu and Bentler, 1999; Svetina and Rutkowski, 2014). The factor structure and intercepts found for EG1’s data were equivalent to the factor structure and intercepts found for the EG2’s data and at T1, T2, and T3. This CFA model is depicted in the Supplementary Figure S1. A simplified version of the second order latent growth curve model for the analysis of individual within-test-takers’ change in test-taking effort and importance is depicted in Figure 3 (see Supplementary Figure S2 for a technical version). The second order latent growth curve model was specified including random intercepts and random slopes by extending the CFA model as follows: At first order latent level, the variance of the factors test-taking effort and importance each at T1, T2, and T3 has been constrained to the same value for compound symmetry covariance structure (Rosseel, 2019, 2020). At second order latent level, test-taking effort intercept has been specified with the three latent factors test-taking effort at T1, T2, and T3 with each path fixed to one. The latent factor test-taking effort slope has been specified with these three factors and the paths fixed to 0, 1, 2 respectively. The means of expectancies, anxiety, and concentration from the baseline assessment as well as gender and age were included to predict the factor effort intercept because these covariates should explain different effort intercepts between the education students. The covariates’ category each existed before the study such as gender and age or were assessed before assessing test-taking effort and importance. Gender and age are included as covariates in SEM (Mutz and Pemantle, 2015) since previous findings consistently suggested their relations to test-taking motivation in educational contexts (e.g., DeMars and Bashkov, 2013; Silm et al., 2020). The covariates are included in SEM to consider their anticipated effects on the criterion variables test-taking effort and importance at T3 (see Mutz and Pemantle, 2015, for standards in experimental research).

“A mean structure is automatically assumed, and the observed intercepts are fixed to zero by default, while the latent variable intercepts/means are freely estimated” (Rosseel, 2020, p. 28). The sum scores of the cognitive ability tasks and exam tasks each were included to predict the test-taking effort slope (instead of test-taking effort intercept) because we experimentally varied the exam tasks’ position. We used the means and sum scores of predictor variables instead of measuring latent factors to keep the number of parameters as low as possible. Random intercepts and slopes for each latent factor were specified with correlations to themselves and to each other. The second-order latent factors test-taking importance intercept and test-taking importance slope were analogously specified and the same covariates included. Correlations between some test-taking effort indicators were allowed according to modification indices (see Supplementary Figure S2).

The model specification considered EG1’s and EG2’s data separately, so EG1’s data were analyzed without EG2’s data and vice versa. Criterion variables were latent test-taking effort intercept and slope as well latent test-taking importance intercept and slope.

To analyze participants’ response processes, we included the responses and response times in a latent trait diffusion model. The diffusion model allows researchers to examine response process components in binary decision tasks (Voss et al., 2013). Binary decision tasks are, for example, the presented cognitive ability tasks where test takers have to choose one of two response options or the questions in the mock exam where the test takers have to decide between the correct and the incorrect response (Molenaar et al., 2015). The diffusion model is based on the assumption that test takers continuously accumulate evidence for the two response options. A momentary preference is formed by weighting the evidence for the two response options against each other. As soon as the momentary preference exceeds a critical level, the test taker responds by selecting the more preferred option.

Diffusion modeling involves defining three process parameters. (1) Information accumulation is a measure of one’s mental simulation of two possible outcomes using the available information (drift rate, v). (2) The amount of information required for a response is reflected in the decision threshold (boundary separation parameter, a). (3) The response time includes time for reactions (e.g., moving one’s finger to the keyboard in a computer-based task) and/or other sensory, mental or motor responses aside from the time needed to make a decision (non-decision time parameter, ter) (Ratcliff and McKoon, 2008; Voss et al., 2013). The drift rate (v) provides insights into information uptake latency, with high uptake speed reflecting high performance, and is a manifestation of a test taker’s capability. The lower the drift rate, the more difficult the task is in relation to a given individual latent trait (e.g., ability or attitude, see Voss et al., 2013). Low drift rates are reflected in low response accuracy and long response times. The boundary separation reflects the response caution. It is assumed to be a manifestation of a test taker’s effort or importance, reflecting their carefulness when responding. Low levels of the boundary separation are reflected in low response accuracy and short response times, two typical signs of rapid guessing.

In addition to the parameters in manifest diffusion modeling, the psychometric diffusion modeling under the item response theory allows to estimate the latent person contribution and task contribution to the response process. The person contribution refers to information processing (latent trait θ) and response caution (latent trait ω) of a person as well as investigate relationships between these latent traits (θ, ω) and constructs such as test-taking effort or importance. The task contribution refers to the task difficulty.

Two model types are distinguished, the D-diffusion (Tuerlinckx and De Boeck, 2005) and the Q-diffusion model (Van der Maas et al., 2011). They basically differ in their parameterization. In the D-diffusion model, the effective drift rate is the difference of the latent trait of a test taker and the corresponding intercept. This parameterization allows task probabilities from zero to one. In the Q-diffusion model, the effective drift rate is the quotient of latent ability and the corresponding intercept. The Q-diffusion model requires task solving probabilities of at least 50% for calculating the diffusion parameters. In case of solving probabilities lower than 50%, the D-diffusion model can be used with consideration of its predictions.

We applied diffusion modeling within the R environment (R Development Core Team, 2009) using the R package diffIRT (Molenaar et al., 2015). The non-decision time (ter) was constrained to control delays resulting from the different laptops we used on the non-response times. The R code can be obtained from the authors.