Problem solving is one of the main activities in mathematics and has been an integral part of mathematics education research since the end of the 20th century. Not only is problem solving necessary for working mathematically, it is also crucial for learning mathematics (Rott et al., 2016). Due to its significance in teaching and learning, engaging with mathematical problems is crucial from an early age, starting in primary school (Palmér and Bommel, 2018; Pehkonen et al., 2013). In the German school system, for example, developing problem-solving skills is therefore a mandatory part of the curriculum (Fritzlar and Rott, 2016; Kultusministerkonferenz, 2022). With regard to mathematical problems, the German curricula draw on a definition of non-routine problems. While the English term problem is used synonymously with mathematical task (Pehkonen, 2003; Rott et al., 2016), the German language differentiates between tasks and problems. In this sense, a task is a routine-problem for students, if they are already familiar with a suitable solution procedure (Schoenfeld, 1985), and a task is a non-routine task for students, if they are not familiar with a direct solution procedure. Accordingly, “problem solving refers to a cognitive process in which an individual determines how to solve a problem that he or she does not readily know how to solve” (Rott et al., 2016, p. 14). In this context, the term “barriers” is commonly used to refer to factors that prevent a direct solution from being found (Pólya, 1973; Rott et al., 2016; Schoenfeld, 1985). Thus, tasks cannot be categorized as a problem per se; rather, this definition relies on the interplay between the problem and its solver (Mason, 2016; Schoenfeld, 1985).

Research into mathematical problem solving covers a wide range of topics and has undergone certain shifts in focus over time (Santos-Trigo, 2024). A comprehensive overview of key research topics and current developments is provided by the review articles by Giang et al. (2024), Lester and Cai (2016), Pamungkas et al. (2023) and Santos-Trigo (2024). One area of research closely related to feedback is the analysis of students’ errors in problem-solving and how teachers respond to them (Fritz, 2022; Heinrich, 2010). However, by addressing feedback more generally, we deviate from this approach because these studies classify incomplete processes or partial results as (strategic) errors. In our view, this conflicts with the definition of problem solving, which involves trying different approaches to find a solution, even if it involves detours (Rott et al., 2021; Wilson et al., 1993). As Pólya (1973), p. 99 noted “Many a guess has turned out to be wrong but nevertheless useful in leading to a better one”. Therefore, getting stuck is not an error, but rather a necessary part of the process. In this line of thought, students getting stuck should be seen as an opportunity for teacher feedback. Thus, this paper aims to examine teacher feedback in problem-based teaching more closely.

Feedback is one of the key aspects of formative assessment, which can be defined as “formative to the extent that evidence about student achievement is elicited, interpreted, an used by teachers, learners, or their peers, to make decisions about the next step in instruction that are likely to be better, or better founded, than the decision they would have taken in the absence of the evidence that was elicited” (Black and Wiliam, 2009, p. 9). In the context of mathematics teaching, feedback can be defined as a deliberately designed, subject-specific response to observations of mathematics-related activities or statements made by a learner (Klopfer, 2018).

Feedback plays an important role in the learning process. From a psychological point of view, the basic purpose of feedback is to induce correct responses (behavioral viewpoint) and provide the information needed to correct ineffective behavior (cognitive viewpoint, e.g., Kulhavy and Stock, 1989; Narciss and Huth, 2004). In educational research, feedback is often “conceptualized as information provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one’s performance or understanding” (Hattie and Timperley, 2007, p. 81). However, providing information about the performance is usually considered only one aspect of feedback. Feedback provides learners with information about their current learning or performance to guide their progress toward the intended learning outcomes (Narciss et al., 2022). This definition emphasizes the role of feedback in adjusting and improving learning in a structured way, a point also highlighted by Shute (2008). Shute (2008) further refines this concept, describing feedback as information communicated to learners with the specific aim of modifying their thinking or behavior to enhance learning. This definition underscores the transformative nature of feedback, emphasizing its role in driving changes that lead to better educational outcomes. Overall, feedback cannot be regarded as a clear, consistent treatment, but rather as a collective term for a wide range of different feedback categories (Wisniewski et al., 2020).

The concept of feedback can be characterized by a variety of characteristics. Söderström and Palm (2024) distinguish between different feedback level-specific categories related to the levels of feedback, as defined by Hattie and Timperley (2007), as well as level-independent variables (see Figure 2). These include aspects such as the way feedback is provided (oral, written, digital, etc.), the timing of the feedback and the context to which it refers (Söderström and Palm, 2024). In this context, Narciss and Huth (2004) speak of “formal and technical aspects related to the presentation of the feedback message (e.g., frequency, timing, mode, amount, form)’ and ‘semantic aspects related to the content of the feedback message” (Narciss and Huth, 2004, p. 4).

Levels of feedback are a common feature of various frameworks and form the basis of the instrument developed in this study. Please note that, despite the term “level” often suggesting this, this categorization does not imply any hierarchy. The following overview examines how different theoretical approaches conceptualize these levels. The aim is to identify relevant similarities and differences in order to develop a comprehensive analytical tool for classroom feedback. Although these approaches originate from different disciplinary traditions (educational sciences, mathematics education and cognitive psychology), they all share the goal of identifying the characteristics and structures that facilitate effective learning. However, their key questions and analytical focus differ. For example, Hattie and Timperley (2007) examine at what stage in the learning process feedback becomes effective. Studies based on Leiss (2010), on the other hand, focus on how teachers promote student autonomy through adaptive interventions, including feedback. This means that aspects such as the role of the teacher and the timing of feedback in the classroom are also considered. Approaches from cognitive psychology focus on questions such as how complex feedback needs to be to promote learning. These approaches primarily consider the information content and cognitive processing depth of feedback. Together, these perspectives can contribute to a more comprehensive understanding of feedback as a multidimensional construct, which is outlined briefly below.

Hattie and Timperley (2007) propose a feedback model that identifies the conditions under which feedback has the greatest impact on learning. According to this model, there are four levels of feedback: task performance, process, the self or personal, and self-regulation. Feedback at the task level informs learners of their performance in relation to completing a task. Söderström and Palm (2024) distinguish between feedback that provides information about the correctness or quality of the answer, and feedback that explains subsumptions. Feedback at the process level focuses on the main processes that learners need to understand in order to complete the task (Hattie and Timperley, 2007). It may include information on how to solve the task and on describing or explaining errors (Söderström and Palm, 2024). Feedback at the self or personal level is directed at the students themselves and is not related to the specifics of the task at hand. Feedback at the level of self-regulation refers to students’ ability to monitor their own learning processes (Hattie, 2012, 2023; Hattie and Timperley, 2007). Studies based on Hattie and Timperley’s work (2007) stem from various fields of educational science and address different subjects, including mathematics (e.g., Husband and Nikfarjam, 2022; Smit et al., 2024; Söderström and Palm, 2024) and English teaching (Brooks et al., 2019).

Leiss (2010) also distinguishes between different areas of feedback. He identifies content, strategic, affective, and organizational levels of feedback (Scharnberg et al., 2024), whereby the affective level corresponds to Hattie and Timperley's (2007) self level. Similar distinctions can be found in the work of Stender (2016) and Krammer (2009). Stender (2016) investigated the support offered to ninth-grade students during modelling tasks. He expanded the models developed by Leiss (2010) and Zech (1996) to include the level of confirmation support and a hybrid level of content-strategic support. Krammer (2009), meanwhile, analyzed the provision of individual learning support during mathematics lessons at Swiss schools, distinguishing between feedback at the content-mathematical level and feedback containing exclusively organizational information. Additionally, Leiss (2010) distinguishes between the various triggers that lead to teacher intervention, the intention behind the intervention (Tropper et al., 2015), and the trigger itself. Teacher interventions can be responsive—initiated by students’ requests—or invasive, when the teacher provides feedback without being prompted (Leiss, 2010). A third approach to categorizing levels of feedback derives from psychology. It classifies feedback according to its complexity, distinguishing between simple and elaborate forms (e.g., Narciss and Huth, 2004; Shute, 2008). Although this model was developed primarily in laboratory settings rather than in subject-specific contexts, it has occasionally been applied in studies on mathematics education (e.g., Rakoczy et al., 2013; Schuster et al., 2021).

While these three approaches highlight important aspects of feedback, their different areas of focus mean they only capture some of the complexity of feedback in authentic classroom settings (Carless and Boud, 2018; Lipnevich and Panadero, 2021). Building on the level distinction proposed by Hattie and Timperley (2007) and the domain-specific adaptations by Leiss (2010), this study has developed an integrated analytical instrument. Building on the work of Leiss (2010) and Krammer (2009), this study adds an organizational level to the existing level structure by Hattie and Timperley, while also distinguishing between a content and strategic orientation at the process level (see Figure 3). This enables for more in-depth and detailed examination of teacher feedback in problem-oriented mathematics instruction. Furthermore, since feedback is shaped by the epistemic and representational practices of each subject, a discipline-specific perspective is required (Palm et al., 2017). Kingston and Nash (2011) demonstrated that the subject area significantly influences the effectiveness of feedback (operationalized by the effect size in their study). In their study, formative assessment practices were significantly more effective in English/language arts (mean effect size 0.32) than in mathematics (0.17) or science (0.09) (Kingston and Nash, 2011). Teachers also require in-depth, subject-specific knowledge to implement formative assessment and associated feedback effectively in the classroom. Feedback tools should therefore be specialized for the respective subject (Bennett, 2011), as subject-specific feedback can improve students’ understanding. Similarly, a mathematics-specific analysis tool can contribute to teacher training and improve teaching quality (Stovner and Klette, 2022).

While the different levels of feedback provide a useful structural framework, further differentiating those levels by using level-specific categories are essential for further analyses of the teachers’ actions. Consider the following two excerpts from different lessons. The first sequence involves an interaction between teacher C and the student S.

This short feedback situation takes place at the task level because the teacher informs the student that the result is incomplete. At this point, no further information is provided by the teacher, for example about a possible cause of the error or the next steps in the solution process. The following sequence from teacher D’s lesson can also be classified as being on the task level:

In this case, the teacher intervenes directly by crossing out a mistake and highlighting it. This also indicates that the result is incomplete. Although both situations can be classified as being on the task level, the teacher’s actions differ significantly. Teacher C verbally indicates that the result is incomplete. Teacher D, on the other hand, intervenes physically by crossing something out, as well as giving verbal instructions. Teacher C highlights the error without offering further assistance with the process. Teacher D addresses the error by correcting it and providing assistance with the correction. These differences will be examined in more detail below.

Following Söderström and Palm (2024), who identified specific realizations for each level of feedback (see Figure 2), these are referred to as feedback level-specific categories. Hattie and Timperley (2007) also indirectly list various feedback level-specific categories. Feedback at the task level is “most common and is often called corrective feedback or knowledge of results, and it can relate to correctness, neatness, behavior, or some other criterion related to task accomplishment” (p. 91). However, further details are only provided indirectly through references to other studies. Similar listings of feedback implementations can be found in several studies (Krammer, 2009; Leiss, 2010; Shute, 2008; Wisniewski et al., 2020), which, although not explicitly termed feedback level-specific categories, correspond to these in content and will be examined in more detail below. Figure 4 additionally provides an overview of central feedback models with regard to the dimensions level of feedback and feedback level-specific categories, highlighting both similarities and differences.

At the Self level, Söderström and Palm (2024) distinguish between three feedback level-specific categories (Personal characteristics, Work and Effort). However, this study does not consider these categories further, as research shows that feedback on Work and Effort correlates, making it difficult to distinguish between them (Wisniewski et al., 2020). By contrast, Wisniewski et al. (2020) distinguish between Reinforcement/Punishment, Corrective and High-information feedback. “Forms of reinforcement and punishment apply pleasant or aversive consequences to increase or decrease the frequency of a desired response or behavior” (Wisniewski et al., 2020, p. 7). A similar distinction can also be found in Leiss (2010). Regarding problem-solving activities, we distinguish between Praise and Motivation in the case of positive reinforcement, following Zech (1996). The Organizational level of feedback is not explicitly distinguished in Hattie and Timperley (2007) but has been added in line with Leiss (2010) (see Figure 3). Leiss (2010) distinguishes three different level-specific categories at the Organizational level: Organization of work, Organization of discipline and Organization of writing/presentation (Leiss, 2010; Scharnberg et al., 2024; Tropper et al., 2015).

Feedback at the Task level corresponds to Knowledge on task constraints and thus corresponds to simple feedback. Shute (2008) lists various forms of simple feedback, from which the feedback level-specific categories Verification, Correct response, Try again, Error flagging and Response contingent can be derived. The feedback level-specific category of Verification is also listed by Söderström and Palm (2024) as Correctness and as Corrective Feedback by Wisniewski et al. (2020). Furthermore, the category of Error flagging largely corresponds to the category of Localization used by Nelson and Schunn (2009).

Feedback at the Process level mostly corresponds to elaborated feedback, which Shute (2008) defines as “information about particular responses or behaviors beyond their accuracy and tends to be more directive than facilitative” (p. 157). Content-related feedback at the Process level includes several level-specific categories: Topic Content, Hints/Bugs/Misconception (Shute, 2008), Demand for Explanation (Stender, 2016), Summary and Offering a solution (Nelson and Schunn, 2009). Topic content refers to teachers providing information related to the topic of the task (Shute, 2008). According on Stender (2016), the category Demanding an explanation refers to teachers asking students to explain their (current) work status or describe their approach. Nelson and Schunn (2009) state that some teachers explicitly provide students with part of the solution through their statements or actions. This is coded as the level-specific category Offering a solution. Another level-specific category that Nelson and Schunn (2009) were able to identify is a Summary in which the teacher summarizes the students’ solution process so far. Such summaries may reorganize or condense students’ previous statements to highlight key aspects, such as “You used the experiences of African Americans, women, and immigrants from Asia as support for your thesis” (Nelson and Schunn, 2009, p. 378). Hints/ Bugs/Misconceptions is a collective category that includes information that does not provide the correct solution but gives hints in the right direction (Shute, 2008). This category also includes strategic hints, which, based on the division into content-related and strategic feedback in this paper, will be referred to in the following as strategic hints, which are also mentioned by Leiss (2007). According to Leiss (2007), such interventions are intended to stimulate strategic approaches. However, a more precise definition is not provided.

At the level of Self-regulation, feedback refers to students’ ability to monitor their own learning processes. It can improve learners’ self-assessment skills, encouraging them to engage with the task and seek feedback on their own learning process. Feedback at this level often takes the form of reflection questions about process-related and strategic knowledge (Hattie, 2012), which are categorized as reflection questions. Söderström and Palm (2024) also refer to “questions or prompts that help students to activate monitoring and controlling meta-processes during their task-solving. Such feedback includes prompts for students to employ metacognitive strategies through self-monitoring statements to guide their problem-solving process” (p. 14). However, it should be noted that previous research on feedback at this level has been predominantly focused on digital or analog feedback tools that were to be used independently by students (Kauffman et al., 2008; Kleitman and Costa, 2014; Santos and Semana, 2015; Shin and Bryant, 2017). No further level-specific categories of feedback given by teachers aimed at students’ self-regulation activities could not be found in the literature.

In summary, the present study focuses particularly on two dimensions: first the levels of feedback and second the feedback level-specific categories, which relate to a particular level (see Figure 5). This classification of feedback level-related categories provides a structured basis for the development of an analysis tool for describing and analyzing teacher feedback. This will be supported and further developed in the following through empirical analyses. This is necessary for further analysis of feedback in terms of its classroom use or effectiveness (Harris et al., 2014).

Video-recorded classroom sessions were used as the primary source of data, enabling feedback to be analyzed in authentic teaching contexts. A classroom study was conducted to minimize possible confounding factors, such as an unfamiliar environment, and to capture an authentic teaching situation. While using multiple cameras for classroom recordings may restrict authenticity, multi-perspective approaches have the potential to provide new insights into social dynamics and interactions (Beeli-Zimmermann et al., 2020). Two tripod cameras were positioned at opposite ends of the classroom to capture the whole classroom. In addition, the teachers wore a mobile camera (GoPro, see Figure 1) on a chest strap. This could be used to reproduce statements made by the teachers and students, as well as the students’ work.

The study includes 13 lessons (45 min each) from 11 primary school teachers. As part of a separate exploratory study, two lessons by teachers F and I in the same class were recorded. The teachers were given pseudonyms in the form of single letters, starting with A, in the order in which their lessons were recorded. All the teachers are primary school teachers from different primary schools in Germany. In order to see a wide selection of feedback situations, resulting in as many different levels and level-specific categories as possible, the teachers were selected to cover as wide a range of professional experience as possible. At the time of data collection, the youngest teacher had been fully trained for 2 years, while the oldest had been teaching for 27 years (see Table 2). Furthermore, the schools are attended by students from a wide range of socio-economic backgrounds. While all the teachers in the sample are female, this does not significantly bias the sample, given that over 90% of primary school teachers in Germany are female (IT.NRW, 2021). Two teachers taught a third-grade class, six taught a fourth-grade class, and five taught a second-grade class. Ms. K.’s lesson will be discussed in more detail later. To ensure the lesson focused on solving a problem-based task,¹ teachers were asked to select a task from a catalog of problems provided by Theile (2024) when planning their lesson. By selecting tasks from diverse mathematical domains (e.g., arithmetic, geometry), we captured diverse feedback across mathematical contexts. The two problem-solving tasks most frequently chosen by teachers were the coins task and the chessboard task (see Table 1). Ms. K. introduced a third problem-solving task, Figurate Numbers, in her lesson.

The aim of the evaluation was the analysis of teacher feedback in problem-oriented mathematics lessons. Given the nature of the study, it was reasonable to approach this aim using both deductive and inductive categories. Therefore, and because of the qualitative orientation of the study, the data was evaluated using qualitative content analysis according to Mayring (2000, 2014). The deductive categories are formed from levels and level-specific categories that were identified in the literature review (see Section 2.2, Figure 5). Bellow, we describe the process of deriving inductive categories from the data.

In the pilot phase of the study, six lessons were analyzed. During the coding process, inductive categories were added at both the levels of feedback and level-specific categories.

These adjustments were then used to create the coding manual. This manual formed the basis for the final evaluation of all data. Only a few changes were made to the category system during this process. These mainly concerned the criteria for defining the differences between individual categories. A discussion of the intercoder agreement check can be found in Section 5.

The evaluation of feedback situations was carried out according to the following procedure, building on the established category system (see Figure 5). First, all feedback situations were identified. According to Hattie and Timperley's (2007) definition of feedback “as information provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one’s performance or understanding” (p. 81), we define feedback situations as (inter)actions between teacher and student in response to a task-related approach (see Table 3).

A task-related approach is defined as the mental or written processing of a task by students, while addressing its specific requirements. It excludes phases in which the teacher provides general organizational instructions (e.g., on social forms or material) before the students have dealt with the specific conditions and content of the task. Interactions can be verbal (e.g., by opening a conversation) or non-verbal (e.g., by the teacher making notes on the worksheet). Let us take another look at the example from the introduction. The entire transcript excerpt shows one teacher-student interaction. The conversation begins when the teacher asks the student what he has written so far. The student then explains his approach. This is a feedback situation, as there is evident interaction between the teacher and the student. A task-based approach is apparent, as the student verbally explains his method of counting the number of squares of different sizes on different worksheets. His approach involves counting the number of squares of different sizes on different worksheets. It ends when the teacher turns to another student and leaves the workstation.

Once a feedback situation has been identified, the next step is to establish which level(s) of feedback have been addressed in that situation (see Figure 6). Each feedback situation addresses at least one level of feedback. However, it is also possible for feedback to address several levels within a single feedback situation. While the level of feedback describes the area or aspect of the learning process on which the feedback is to have an effect (Hattie and Timperley, 2007), we define the level-specific-categories as their concrete realization in implementation (see Table 3). After identifying the feedback level(s), the final step of the analysis is to check how the feedback is implemented within each level (level-specific category). As in the previous step, several level-specific categories may occur within a level.

In the introductory example, this means that the statement at minute 16:47 relates to the solution process and can therefore be assigned to the process level. In the specific implementation, the teacher shows the student a part of the solution that he had not yet considered. This corresponds to the level-specific category Counter examples, which was determined inductively and will be described in more detail in section 4.1. However, another level of feedback and level-specific category can be identified in this situation. At minute 17:11, the teacher instructs the student to document their findings in his preferred manner. This task has no discernible content-related reference; rather, the feedback relates to the Organizational level. As the teacher describes how the results should be presented, this situation corresponds to the feedback level-specific category Notation, which was determined deductively.

The structure of the final analysis tool is illustrated below by presenting and providing examples for selected feedback level-specific categories. A summary of all levels of feedback and level-specific categories can be found in Figure 7. A complete overview of all level-specific categories, including definitions and anchor examples, is available in the online supplement.

The following section will examine at how the analysis tool described above can be used to analyze teacher feedback. Some examples from the data will be used to illustrate the process. The lesson of teacher K is used as an example. This lesson took place in a fourth-year class at a German primary school. The focus was on working on the figurative number task (see Table 1, problem-task 3). The lesson can be divided into three phases: First, a brief introduction was presented in a plenary meeting. The task was displayed on the board, and the students were asked to gather some initial ideas for solutions. Feedback situations could already be observed during this phase. These were characterized almost exclusively by feedback at the Self level, specifically Praise, and Reception signals. The following situation begins when a student called on another student to share another idea:

In this brief feedback situation, the student suggests the idea of multiplying as an initial task- related approach. The teacher’s response is directed at the student and can therefore be classified at the Self level. In this instance, the teacher praises the student. As the feedback situation ends and another student suggest a new approach, Praise is the only level-specific category coded for this situation.

After the short introduction the children worked on the task either alone or in groups. During this phase, the teacher moved around the room, observed, asked occasional questions and supported the students with feedback. While feedback in the introductory phase was largely limited to Praise and Reception signals, with only one level-specific category coded per feedback situation, several level-specific categories can usually be identified during the work phase. The following excerpt illustrates this. The feedback situation begins when the student approaches the teacher and asks:

After the student explains her proposed solution (task-related approach), the teacher asks her to write down a task for each sub-figure. This response was coded as Notation. However, the student shows a lack of understanding, asking “Um, how?” for clarification. The teacher then guides the student through the approach step by step, looking at the arithmetic structure of the three figures shown on the worksheet (from 20:02 to 20:12). This procedure was coded as Offering (part of) a solution because the teacher’s actions partially specify the relationship between the figures, thus providing part of the solution. At the end of the situation (at minute 20:14), the student is asked again to write a problem under each figure. This was also assigned to the level-specific category Notation. Thus, two codes or level-specific categories were assigned to this feedback situation in total.

The lesson concluded with a plenary session, during which the solutions to both parts of the task were discussed. The teacher focused on the approach chosen for the task. At the end of the lesson, the teacher explained that these figurative sequences have special names – square and triangular numbers. The teacher’s role was primarily to accompany and support the students, making the lesson ideal for demonstrating the application and differentiation of the developed category system.

The procedure that was previously demonstrated using selected examples was carried out throughout the lesson (see Figure 8). This enables the practical application of the analysis tool to be understood. It also allows recurring patterns in teachers’ feedback behavior to be identified. A total of 50 feedback situations (see section 3.2) were identified during the lesson. These situations were distributed across all levels of the category system, with content-oriented Process feedback (n = 24) and feedback on the Task level (n = 26) dominating. Feedback at the Self (n = 12) and Organizational (n = 5) levels occurred less frequently. Only two situations involved feedback at the Self-regulation level.

During the lesson, both deductively derived and inductively supplemented categories were observed. The inductively supplemented level-specific categories include the categories Reference to task condition (n = 6) and Request for looking back (n = 2). The latter can be illustrated by the following situation:

The level-specific category Request for looking back is coded if, after working on the problematic task, the teacher asked the student to review their solution process before, for example, starting another task. This request for verification must be independent of the result – regardless of whether it is correct or incorrect. In the previous excerpt, the teacher encourages reflection on the procedure and, despite several questions from the student, does not address whether the result is correct or incorrect.

It is also notable that the majority of feedback provided by teachers is responsive. Most feedback situations (n = 43) were initiated by the students, with only seven arising through teacher intervention. The exception to this was the discussion of the results at the end of the lesson. Five of the seven instances of invasive feedback occurred during this phase. Various focal points can be identified regarding the distribution of the individual level-specific categories. Initially, at the beginning of the lesson, feedback was almost exclusively limited to the level-specific categories of Reception signals and Praise. During the subsequent work phase, Verification of the result initially dominated before a greater variety of level-specific categories emerged as the lesson progressed. Nevertheless, certain categories stand out because they occurred with above-average frequency. Notably, these were Praise (n = 8), Verification – result (n = 11) and Offering (part of) a solution (n = 10). The high frequency of Verification of the result and Offering (part of) a solution is consistent with other research findings, in which result- or performance-oriented feedback dominates (Benecke and Kaiser, 2023; Brooks et al., 2019; Gan et al., 2021; Theile and Rott, 2024). These findings are also confirmed by Lipnevich and Smith (2009). They found that, apart from verification, praise was the second most frequently used level-specific category in the classroom. In summary, this lesson revealed a wide variety of level-specific categories. At the same time, depending on the phase of the lesson, a concentration of individual level-specific categories can be seen in some cases. These observations provide initial insights into how feedback is used in problem-oriented teaching in primary schools.

Analyzing the lesson has revealed how various types of feedback can be identified and understood. While this detailed examination provides initial insights into the typical patterns and functions of teacher feedback, it does not reflect the diversity of feedback practices across the entire dataset. To gain a more comprehensive picture, all 13 recorded lessons were coded and evaluated. A total of 601 feedback situations were identified, in which 974 level-specific feedback codes were assigned. On average, there were 46 feedback situations and 75 codes per hour, corresponding to an average of 1.63 codes per feedback situation. The number of feedback situations per hour varied between 30 and 77, and the number of level-specific feedback codes per hour between 40 and 115. Figure 9 shows the overall distribution of level-specific feedback codes and thus forms the basis for future analyses, which will examine both recurring structures and context-dependent differences in more detail.

This study aimed to develop a framework for identifying and analyzing teacher feedback in problem-oriented mathematics teaching, with interrater agreement as the primary quality indicator. The following structured discussion applies selected quality criteria to assess the category system’s overall quality, which are central to Mayring’s qualitative content analysis and are procedure specific. The focus is particularly on criteria of objectivity, especially intercoder agreement, and validity (Mayring, 2000, 2014). Mayring (2014) defines intercoder agreement as the independence of the results from the person conducting the analysis. This is determined by comparing the results of two coders, with an agreement calculated using the percentage correctness approach (Jacobs et al., 2003).

In this study, determining intercoder agreement, is a three-part process. First, raters must determine the time codes at which feedback situations occur. This is followed by identifying the levels of feedback and feedback level-specific categories (see Figure 6) at each time point, since a comparison of the feedback codes can only be made once common time codes have been established (Rott, 2013). Table 13 shows an excerpt from the coding comparison. First, the time segments are compared. Then, the levels of feedback are assigned. Finally, the feedback level-specific categories are assigned. The two raters mostly coded the same levels of feedback in the excerpt (Table 4), except for the first feedback situation. In this situation, Rater 1 coded feedback at the task level, which Rater 2 did not. Following a consensus meeting, Rater 2 agreed with the coding. Furthermore, all codes for the level-specific categories in the corresponding excerpt were identical.

Intercoder agreement was determined for a total of four lessons (30.77% of the data), by comparing the codes of the first author with those of a second, independent rater. Following Jacobs et al. (2003), the time codes were determined using a percent correct approach. For the selected lesson (Teacher K), the percentage agreement was 94.34%, compared to 94.29% (Teacher H), 85.29% (Teacher G) and 84.38% (Teacher F) for the other lessons. These results indicate good intercoder agreement as all double-coded lessons show agreement scores around or above the suggested agreement score of 85%. However, when determining agreement in feedback codes (in terms of both levels of feedback and feedback level-specific categories), the percentage agreement should be viewed critically, as it is a non-random measure (Rott, 2013). Therefore, Cohen’s kappa was chosen to determine levels of feedback and feedback level-specific categories, as it is based on percentage agreement, since it is based on percentage agreement but also considers the ratio of observed to expected agreement.

The Cohen’s kappa for the levels of feedback for the selected lesson (Teacher K) is 𝜅 = 0.87. For the other lessons, Cohen’s kappa is 𝜅 = 0.78 (Teacher F), 𝜅 = 0.86 (Teacher G) and 𝜅 = 0.82 (Teacher H). The Cohen’s kappa for the feedback level-specific categories for the selected lesson (Teacher K) is 𝜅 = 0.78. For the other lessons, Cohen’s kappa is 𝜅 = 0.72 (Teacher F), 𝜅 = 0.86 (Teacher G) and 𝜅 = 0.83 (Teacher H), indicating a substantial (Landis and Koch, 1977) or sufficient (Mayring, 2000) intercoder agreement. Given that coding interactions is subject to subjective and interpretative factors, this level of agreement is satisfactory.

According to Mayring (2014), validity is another quality criterion. This involves verifying the results of the analysis using comparative data on similar topics and subjects. The aim is to ensure that the results can be reconciled with existing findings. While the analyzed lesson is only exemplary, the results show some similarities to those of previous studies. For example, Brooks et al. (2019) found that feedback tends to be more prevalent at the task level in primary schools. While the results of the analyzed lesson align with this trend, they require verification in additional studies, since Brooks et al. (2019) did not conduct a mathematics-specific analysis. The dominant level-specific categories in the lesson Praise, Verification – result, and Offering (part of) a solution also correspond to those frequently identified in other studies (Benecke and Kaiser, 2023; Brooks et al., 2019; Gan et al., 2021; Theile and Rott, 2024). Pieper (2023) investigated how feedback situations are triggered, demonstrating that in dyadic individual interactions, feedback can arise invasively or responsively. However, in polyadic group interactions, invasive situations predominate. In the analyzed lesson, most feedback situations were triggered responsively, despite the students working both individually and in groups. While the findings of the exemplary analysis do not directly contradict Pieper’s (2023) results, they are surprising. Furthermore, the category system was discussed and reflected upon several times during the research process in collaboration with other researchers.

Naturally, this study is subject to certain limitations. For instance, rather than conducting a systematic review of articles on the topic of feedback. Instead existing reviews were utilized (Lipnevich and Panadero, 2021; Söderström and Palm, 2024). Due to our language skills, only articles published in English or German were considered. Another significant limitation of the analysis is the small sample size resulting in limited generalizability. Further research is therefore needed to verify the categorization’s validity in classroom settings and enable more concrete statements to be made (Lipnevich and Panadero, 2021).