We are applying a qualitative approach via a validation framework for developing and validating science LPs described by Jin et al. (2019c), which outlines research stages for developing an LP. Herein, we describe the development and scoring of the CLIMb framework and its associated instrument, which captures salient patterns of students’ reasoning using learning theories, think-aloud interviews, and open-ended survey evidence. We progressed from the development to the scoring stages using a series of research cycles. Each research cycle utilizes the assessment triangle (National Research Council (U.S.) et al., 2001) composed of the following three phases (Figure 1): (1) the cognitive phase, where a hypothesis is generated based on previous observations, (2) the observation phase, where data are collected via surveys or interviews, and (3) the interpretation phase, where themes are identified and coded through thematic coding and constant comparative method (Naeem et al., 2023). After Phase 3, we will begin a new cycle of the assessment triangle by revising both the hypothesized framework and instrument (questions and rubric) used in the previous data collection phase. This iterative, qualitative process refines the framework and instrument for more accurate readings of student reasoning at different levels. We have completed three cycles of the assessment triangle, as described below (Jin et al., 2019c).

A summary of the refinement of the CLIMb framework and instrument through each cycle of the assessment triangle is provided in Figure 1. The first two research cycles are based on data collected and interpreted from open-ended surveys and think-aloud interviews of undergraduate biology majors and non-majors enrolled at a public university on a quarter system and with high research activity (R1) (Institutional Review Board (IRB) protocol STUDY00013024). The third research cycle was based on think-aloud interview data collected from participants enrolled at a public land-grant R1 university on a semester system (IRB protocol 6558).

The developmental stage began with the first research cycle guided by the assessment triangle (Jin et al., 2019c). A 3-level LP framework was hypothesized during the cognitive phase of this first cycle, including the most novice and expert levels (i.e., lower and upper anchors of the LP framework, respectively). The hypothesized expert level applies concept-based reasoning using two core immunology concepts aligned with the core biology concept of Systems (AAAS, 2011; Pandey et al., 2023). These concepts were considered necessary for the understanding of immune functions during primary and secondary responses, as well as representing the overall purpose (i.e., function) of the activated system (i.e., immunity). Concept (A) is based on how the immune system works as a network: “the immune system is a coordinated network of macromolecules, cells, tissues, and/or organs within an organism.” Concept (B) describes how immunological memory results as a purpose of the immune system’s network actions: “Immunological memory plays a critical role in protective immunity” (Pandey et al., 2023). These concepts were chosen based on our experiences with students and within the field of immunology, and they can capture systems thinking and its development, starting at the novice level. These concepts are also broadly applicable to an audience that has not had formal instruction in immunology. This initial hypothesized CLIMb framework integrates both concepts as one ‘progress variable,’ whereby reasoning about immunity integrates both concepts at each level as students gain mastery of the complexities of the immune system yielding long-term immunological memory and immunity. We then designed a questionnaire (i.e., instrument) composed of six open-ended questions (i.e., items), each designed to challenge students’ understanding of how the immune system functions in different contexts. These items were designed to elicit responses that use logic relating to our core concepts and to be adaptable to think-aloud interviews and surveys. Think-aloud interviews allow interviewers to observe how students reason through problems presented to them in real time (Scott et al., 2020; Doherty et al., 2023). Our initial instrument focused on vaccination immunotherapy as a means of capturing novice understanding, including those who have not received formal instruction on immunology, a key component of a developing LP (Jin et al., 2019c). Items were designed to characterize the breadth of student ideas and to uncover misconceptions about the central ideas of developing immunity. The misconceptions included within explanations about how vaccinations induce immunity have the potential to reveal how students think about concepts in immunology (Bauer et al., 2021; Kahlon et al., 2022).

We then entered the observation phase, recruited students from an introductory biology course (last of a three-quarter course series) after the immunology module was completed, and collected 439 survey responses. We also recruited from a general biology course (mid-quarter) designed for pre-nursing and attended by non-biology majoring students, and collected 144 survey responses. Twelve students volunteered for think-aloud interviews, which were recorded through Zoom and transcribed using Zoom closed-captioning. Students were incentivized via course credit to increase participation and, therefore, a large range of thinking within these courses. Students opting out of sharing data were excluded from the study but were still asked to complete the survey assignment and received the same extra credit opportunity associated with the assignment. We finally entered the interpretation phase and used thematic analysis and the constant comparative methods to analyze student responses collected in the previous observation phase and identified emergent patterns or themes in student thinking (Jin et al., 2019c; Scott et al., 2023). We found that some students did not clearly credit the immune system for the host’s immunity developed based on an inoculation. We revised the hypothesized framework by adding a new lower anchor below our hypothesized Level 1 to be inclusive of responses that did not involve the immune system (Figure 2). The instrument (i.e., questionnaire and associated rubric) was also revised to remove one item addressing herd immunity because responses elicited were focused on societal and cultural explanations, rather than biologically relevant explanations. We also included two additional items. One item provides an opportunity to unpack how antibodies are connected to the immune system because many explanations in this cycle, including lower levels, volunteered the term ‘antibodies’ but with little or no details about other elements of the immune system. Another item prompted students to reveal vaccine content in order to better evaluate student thinking about what is foreign to the body versus what is a component of a biological system, which was not clearly distinguished within student responses in this cycle (Supplementary Table S1).

We began a second research cycle guided by the assessment triangle and entered the scoring stage of the validation process, during which we began scoring responses (Jin et al., 2019c). A 4-level framework and an 8-question instrument were hypothesized during the cognitive phase, resulting from the interpretation of the data within the previous research cycle. For the observation phase, we recruited undergraduate students’ mid-quarter from an introductory Biology course (the first of the three-course series), an Immunology course, a Science & Society course designed for biology majors, and a Biology survey-course designed for non-biology majors. We collected 200 open-ended survey responses and three recorded think-aloud interviews, incentivized with course credit as described in the previous cycle. We expected a large number of respondents and, therefore, took the opportunity to test and compare two iterations of two items. Interviews were recorded through Zoom and transcribed using Zoom closed-captioning.

We again used thematic analysis and constant comparative methodologies in this cycle’s interpretation phase. Through our process of developing scoring rubrics, we found that students integrated two key elements that progressed across our student populations, whereby conceptual reasoning involving concept (A) develops independently from concept (B) until both concepts are integrated at higher levels. We, therefore, significantly revised the CLIMb framework to include two distinct progress variables and split Levels 1–3 into parallel tracks. The first progress variable we identified described how students identified and used the immune system to explain biological processes involving immunity. The second progress variable described how students identified and used immunological memory when explaining how protective immunity is established. In addition, we included an intermediate Level 4 to capture explanations that include mechanistic details of how the immune system coordinates a response as a network and makes connections with the resulting immunological memory, albeit with misconceptions and/or gaps in reasoning (Figure 2). To confirm the upper anchor of the CLIMb framework and assess whether our instrument could capture explanations applying concept-based reasoning and a thorough understanding of immunity, we recruited graduate and faculty volunteers from the Immunology Department to refine the upper- and expert-level reasoning and collected nine expert survey responses. The instrument was also modified, merging the first item with a prompt to reveal vaccine content because some responses to item 1 integrated these details, giving good insights into student thinking. We included items addressing immunity as a change the body undergoes after birth rather than an inherited protection, because responses revealed an understanding of immunity as an inherent resistance to external threats. We also included items addressing immunological memory more specifically as a modification of the immune system. This change was based on responses that suggested an understanding of memory as a function of external interventions and/or the presence of foreign material throughout the duration of the protective memory, rather than as a direct function of a biological system. More details about instrument modifications are shown in Supplementary Table S1.

We continued to score our responses within this research cycle, which remained within the scoring stage of the validation process (Jin et al., 2019c). A 5-level framework and an associated instrument with 10 items were hypothesized during this cycle’s cognitive phase, based on the interpretation of the data from the previous research cycle. As we entered the observation phase, we recruited five participants for an hour-long recorded think-aloud interview, incentivized with a gift card and using flyers posted across campus. Participants consisted of four undergraduate students (three non-biology majors and one environmental science major) and one veterinary student. Interviews were recorded through Teams and transcribed using the Microsoft Transcription tool. We again used thematic analysis and the constant comparative methodologies during the interpretation phase. A prompt addressing immunization records, without specifying how antibodies were developed, was added to the instrument. Our resulting instrument and framework are refined and discussed based on this recently completed cycle (Figures 2, 3).

This progress variable requires that student responses identify and/or use the immune system to explain biological processes involving immunity. Immune network Level 1 explanations describe external interventions as being responsible for developing immunity. Students at this level describe immunity as a function of an external intervention, rather than a function of the body or immune system. Responses with this logic are consistent with the therapeutic use of a drug (e.g., antibiotic), such as “A vaccine fights off a virus…” (Table 2: A). We also found this to be true within certain responses that included an understanding of Central Dogma (the process of gene expression, including DNA transcription resulting in RNA, followed by translation of this RNA molecule into protein) or cell biology (Table 2: C). Similarly, a common misconception we found at this level is not attributing immunity as a function of the body or immune system (Figure 3).

At the immune network Level 2, explanations shift to describing immunity as a function of the body, albeit with little or no mechanism (Figure 3). Explanations often use terms like “the body” or the “immune system” when attempting to explain the origins of antibodies or memory cells, without elaborating on the process. For example, “[Vaccines] act as an infection to the body to help build immunity to those infections” (Table 2: I). A misconception we uncovered at this level is the description of many action(s) being carried out by one component of the immune system. For instance, “(…) the antibodies would have a memory that the infection is a threat and will kill it.” Here, both immunological memory and the killing of pathogens are entirely carried out by one component (the antibodies) of the immune system (Table 3: C). Explanations also tend to include anthropomorphic language like “jobs” and “fighting” or to describe functions driven by a cell’s ‘needs’ or ‘wants’, rather than by a signal received or receptor binding (Table 2: E–K). A misconception found at this level is that the vaccines are sometimes described as viral infection that can be used as ‘practice’ for the immune system (Table 2: I–K). Another common misconception we found, starting at this level and persisting through Levels 3 and 4, is the idea that the immune system is ‘overwhelmed’ when exposed to multiple antigens; most often elicited by item #5 in Supplementary Table S1 (Table 2: G).

The immune network Level 3 is the highest level of understanding we found, which does not include connections to immunological memory (the other progress variable). Explanations at this level show some mechanistic understanding of how the immune system responds to stimuli by describing at least one relationship among the components relevant to a particular function (Table 2: L–P; Table 3: G). The immune system is described as interacting components to address a goal rather than an amorphous entity acting as a single unit (Figure 3). For example, lymphocytes are said to have uniquely generated receptors that “match” an antigen or a function carried out by macrophages (Table 3: N&O). However, we did also identify misconceptions appearing at this level and persisting through Level 4, describing antigen-receptors as molding to the antigen introduced into the body (Table 3: O). This misconception was sometimes associated with another misconception of an overwhelming immune system when multiple antigens are introduced into the body. This logic is also frequently associated with an understanding of immunological memory that results from “storing” the molded antigen-receptor for later use; a Level 2 immunological memory thinking (described in the next section) (Table 3: G). Other misconceptions we found were based on confusion about correctly assigning a function to a structure. For instance, participants attributed many functions to B cells, such as producing antibodies, using antibodies and directly attacking infecting pathogens. The descriptions of antibody functions were not very clear and oftentimes involved directly injuring targets or “attacking” targets without involving relevant components (e.g., Neutrophils or Complement System). Although some relationships were described at this level, we found that students had a general tendency to discuss B cells and antibody production, seldom discussing T helper or T killer cells.

This progress variable requires that student responses identify and/or use immunological memory when explaining how protective immunity is established and maintained. Immunological memory Level 1 explanations invoke external means rather than internal biological processes to describe long-term immunity and immunological memory. Students often include the misconception of external interventions as the source of immunological memory or neglect to include this process in their reasoning about immunity, despite including details of the immune system in certain cases. For instance, “I think vaccines work by injecting a substance that either improves immunity or fights against a particular pathogen in an organism” (Table 2: R&S).

Immunological memory Level 2 explanations explicitly mention that memory is left behind (after the antigens are cleared from the body) and maintained after inoculation, but do not identify the responsible relevant immunological components. Explanations include the presence of structures for the duration of the protective immunity (Table 2: U–X). At this level, the term ‘memory’ is often used colloquially, such as ‘computer memory’ or ‘cognitive memory,’ rather than immunological memory, and often includes anthropomorphic language like “becoming familiar” or “storing for later.” Although explanations acknowledge a change in the body allowing for long-term immunity, these explanations often include misconceptions about how this long-term protection was established. For instance, immunological memory is described as a form of storage of information (mimicking a computer hard drive) or an insertion of the introduced structure (e.g., gene, antigen) into the host cell’s DNA (Table 3: I). For example, “The vaccine will cause the body to remember how to create antibodies when it encounters the pathogen again” discusses immunological memory in terms of cognitive memory. Alternatively, “[the body] stores the vaccine information as a code… for the proteins that make up portions of the antibody” discusses immunization in terms of computer storage (Table 2: U–W).

To reach immunological memory Level 3 reasoning, students displayed two elements: (1) the use of immunologically relevant components, such as antibodies or memory B cells, and (2) the protective immunity these components provide to the body. Although not required, a connection between these two elements (the component(s) and the resulting immunological memory) begins to appear at this level. Reasoning at this level, however, is limited in the mechanistic reasoning of a primary immune response resulting in the long-term presence and maintenance of immunological memory and immunity (Figure 3). Students begin unpacking how a secondary immune response is improved through multiple exposures; however, explanations lack the mechanistic details of how the primary response contributes to changes within secondary responses. For example: “… B cells can attach to the antigen and then start a signal cascade [causing] an increase in B cells that will create antibodies that will attach to the antigen and then memory B cells that will be able to start the process again if the antigen returns” (Table 2: Y–AA). A misconception we uncovered at this level is the reliance on antibody titers to explain immunological memory, rather than involving memory cells. Although antibodies are a requirement for developing immunity, memory cells are responsible for the memory response. If memory cells were mentioned, memory T cells (killer and helper) were seldom mentioned (Table 3: A&E). Another misconception found within some responses at this level and persisting to Level 4 is the idea that the biochemical structure (i.e., shape) of antigen-receptors is not generated through DNA rearrangement during T and B cell development but is instead inherited (Table 3: H).

The key aspect of upper-level reasoning is the synthesis of both progress variables, beginning at Level 4. Explanations at this level describe how immunological memory results from multiple (at least two) coordinated network interactions of an immune network, albeit awkwardly or containing gaps in knowledge. Explanations include mechanistic details of structures and their relationships during a primary immune response, highlighting a cause with its effect and resulting in a modification in the immune system (Figure 3). Furthermore, students begin to develop an understanding of emergence, which is a property that results from interactions among the system’s components but does not appear in any individual component (Momsen et al., 2022). Our observations reveal a potential for misconceptions in equating immunity to being impervious to infectious disease, such as “the problem with the COVID vaccine is that they called it a vaccine. Everyone’s like, ‘So I’m not gonna get sick?’ Well, you might get sick, but the symptoms will be less. And then people were like, ‘so it’s not a vaccine.’” This participant reflected on how vaccines can be perceived as impervious. If a vaccine is not entirely effective, it should be named differently, which suggests a misconception of vaccines as a barrier (Table 2: CC&DD). Although the participant acknowledges a potential lack of public awareness, the explanation suggests immunity as a form of being impervious to disease. In other words, explanations describe a small amount of immunity as a lack of immunity. Misconceptions based on a misunderstanding of mechanisms appearing at lower levels can persist at this level: (1) misassociating functions with structures, such as erroneously describing memory cells as having the function of antibody production, (2) molding antigen receptors to their epitopes or pathogens, and (3) overwhelming the immune system when too many antigens are introduced. Despite details of immune components and relationships, we can identify one or more of these misconceptions. For example: “The antibodies are highly mutable (through VDJ recombination) to allow for the best fit with a future antigen” (Table 2: DD).

Level 5 (upper-anchor) explanations apply concept-based reasoning. Responses at this level clearly describe cause and effect relationships between relevant immunological components and show emergent properties that result in long-term immunological memory through different contexts. Explanations also demonstrate an understanding of immunity as scalable rather than binary or absolute, and that ‘low-grade’ immunity is still helpful during secondary exposure. Despite feeling ill during a secondary exposure to an antigen, explanations at this level acknowledge that the immune response is better than during the first response, a notion missing from Level 4 responses. We have found a few expert-level responses among student participants; however, those that reached Level 5 and applied concept-based reasoning in response to scenarios described how errors during a primary immune response alter memory responses and their outcomes (Figure 3). For example: “Every person’s immune system is different. The types of immune cells produced by the immune system are completely random since these cells shuffle their DNA to produce millions of random combinations of immune receptor proteins. It is possible that Sasha’s immune system created naive T and/or B cells against the flu but not against the polio virus” (Table 2: GG).

Our evidence supports the need for two progress variables to develop a deeper understanding of immunity and how immunity develops. The first progress variable we have isolated describes how students identified and used the immune system to explain biological processes. The second progress variable describes how students identified and used immunological memory when explaining how protective immunity is established. These progress variables have distinct lower levels (Levels 1–3), but in upper-level reasoning, student reasoning must merge these progress variables.

Our observations uncovered three key features of our students’ performances that progressed across the lower three levels of reasoning. First, not all participants who could describe immunological memory with relevant components of the immune system (immunological memory Level 3) could identify at least one relationship among the immune system structures relevant to a particular function (immune network Level 3). Second, the opposite was also observed, albeit less frequently, with students reaching Level 3 reasoning using the immune network progress variable but not the immunological memory progress variable. Finally, some students who exhibited Level 3 reasoning on both progressions could not quite explain how the outcomes of interactions between structures of the immune system during a primary response could influence, or lead to, long-term immunological memory. In other words, these participants could not fully integrate the two progress variables when reasoning about immunity, suggesting that student reasoning can express ideas that fall into different levels within each progress variable (Wilson, 2009; Table 3).

It is with expert reasoning that these progress variables are appropriately and smoothly merged while applying system thinking. Within upper-level reasoning, students begin to develop systems thinking by identifying more distant relationships, nested hierarchies and emergence; however, students at Level 4 still struggle with the use of systems-thinking concepts to construct explanations. Levels 3-5 align well with Levels 3 and 4 of the LP on the ecosystem, which integrates both the use of systems thinking and 3D learning; discussed further in the next section (Jin et al., 2019b).

The CLIMb LP is a cognitive model and provides a perspective on how students’ informal ways of thinking and reasoning develop into expert-level scientific reasoning. We are using an established approach to validating this LP: first, defining a theory; second, using the theory to make predictions about student reasoning and make empirical checks against observations; and third, as the theory develops, it generates new predictions evaluated with empirical checks. We share our findings of a developing framework supported by rich qualitative data from interviews and student-written responses, giving insights into student reasoning about immunity (Mohan et al., 2009; Jin and Anderson, 2012; Wyner and Doherty, 2017; Jin et al., 2019b, 2019a; Scott et al., 2023). This empirically driven method of validating LPs provides, at this stage, not only insights into how students think but also reveals their misconceptions based on the level of reasoning and a roadmap of incremental learning that instructors can use to tailor their instruction, their assessments, and their curriculum coherently to support 3D learning (Jin et al., 2019b, 2024).

This emerging empirically based reasoning framework highlights distinct levels of reasoning. These levels are organized into a ‘roadmap’ or ‘path’ detailing incremental learning through five levels of achievement, beginning at a novice level. These different reasoning levels are defined based on student knowledge, abilities, and misconceptions at each level (Figure 3). This study demonstrates some parallels with frameworks that incorporate systems thinking. Previous studies by Jin et al. (2019b) describe a 4-level LP for systems thinking used to reason about the ecosystem developed based on K-12 data. Within higher education, Momsen et al. (2022) proposed that biology systems-thinking (BST) can be organized into four levels (Momsen et al., 2022). The BST framework was synthesized from prior research on systems and systems thinking and was proposed as a potential framework for organizing instruction and assessment in undergraduate biology. This framework organizes concrete outcomes into a hierarchy from low to high complexity. However, very different biological disciplines, systems thinking is a broadly applicable skill required for expert reasoning on different contexts, such as ecosystems and biological systems, like the immune system (Jin et al., 2019b; Momsen et al., 2022).

The lowest level of the CLIMb framework (Level 1) does not involve the immune system and, similar to the lower anchor of the ecosystem LP by Jin et al. (2019b), students either have no idea or do not describe any relationships among organisms. Moving from CLIMb Level 1 to Level 2 (in both progressions) requires a shift in reasoning from thinking about immunization as directly responsible for killing the infecting pathogen (i.e., an outside element is doing all of the ‘work’) to thinking the body as primarily responsible for killing and clearing pathogens (i.e., the body is doing all of the ‘work’). Moving to Level 2 requires an acknowledgement of the immune system, or one of its components, as being responsible for the development of immunity and for immunity itself.

In CLIMb Level 3, the students’ progress in adding details about immune components, but without connecting the two progress variables. Here, at least one relationship is described between two components of the immune system. However, descriptions have yet to include relationships connecting the two progress variables. We found that the data align well with the mid-level (Level 2) complexity of Momsen’s BST framework, which involves reasoning about relationships within a system (Momsen et al., 2022).

It is not until CLIMb Level 4 that both progress variables are successfully linked within a student’s explanation. Although students have yet to successfully explain the nuances of an immune response, and their explanations may be awkward and include misconceptions, the students unpack how the relationships within components of the immune system lead to long-term immunity. The upper level of the ecosystem LP by Jin et al. (2019b) describes student thinking as identifying relationships and patterns of interactions within the ecosystem; however students still lack the ability to construct explanations using systems thinking. Student reasoning about the immune system can be further unpacked because we found a clear transition from proximal relationships (in CLIMb Level 3) to more indirect relationships that connect the two progress variables and demonstrate properties of emergence (in CLIMb Level 4). Similarly, Momsen’s BST upper level of complexity is where emergent biological phenomena begin to appear (Momsen et al., 2022).

We finally describe CLIMb Level 5 as the expert level reasoning on immunity, which applies concept-based reasoning, systems thinking, with immunological memory as an emergent property of the immune system. Similarly, reasoning about the system as a whole reaches an upper level of complexity in Momsen et al. (2022)’s proposed framework of systems thinking in undergraduate biology. Also, observations within K-12 about the ecosystem support an upper anchor level of reasoning that applies systems thinking. At this level, secondary students use systems-thinking concepts to construct causal mechanisms that explain phenomena about interactions within the ecosystem (Jin et al., 2019b). The CLIMb LP expert level reflects systems thinking and incorporates all three of the 3D learning dimensions: scientific practices, disciplinary core ideas, and crosscutting concepts (National Research Council, 2013).

Studies show that active learning enhances student performance; however, the level of effectiveness is predicated on the approach used and its implementation (Freeman et al., 2014; Bruns, 2021). The effectiveness of student-centered approaches relies on an awareness of student understanding. Going beyond content knowledge, instructors must draw from pedagogical content knowledge (PCK) and pedagogical knowledge (PK) to effectively implement student-centered approaches. Differences in PCK and PK explain, at least in part, why different implementations of the same active-learning approach lead to different student outcomes. If an instructor gains more insights into students’ thinking, as part of their PCK, they tend to become more grounded in evidence from students, and their instructional strategies tend to focus more on student thinking. Instructors who access and interpret students’ reasoning as students engage with challenging questions foster instructor PCK development, which, in turn, fosters great student learning. The CLIMb framework and associated instrument leverage student thinking to support the development of instructor PCK, bettering implementation of active learning approaches and enhancing student learning (Andrews et al., 2019; Gehrtz et al., 2022; Waugh et al., 2025).

The levels defined within the CLIMb reasoning framework provide guidance for what students at earlier stages of learning may achieve on their way to developing a strong understanding of the target key concept(s) (i.e., progress variables) required for a deep understanding of immunity. LPs are tools for instructors to support student learning of concepts and to help them make profound shifts in reasoning through a series of incremental steps with appropriatly scaffolded instruction (Jin et al., 2024). Instructors can use the CLIMb framework to gauge student understanding and plan approaches to target misconceptions we have identified based on the levels of reasoning demonstrated by a student body and inform instructional approaches, stimulate better engagement, and prevent student frustration (Baumert et al., 2010; Sadler et al., 2013; Chen et al., 2020; Groß-Mlynek et al., 2022).

LPs build powerful assessments and inform both researchers’ and instructors’ development of assessments that uncover students’ reasoning and measure progress toward goals. The CLIMb framework and associated instrument can be used to not only track student progress through an immunology or biology curriculum but also to assess the effectiveness of instructional interventions and design of an immunology curriculum; all of which are evidence-based recommendations of National Research Council (U.S.) et al. (2001).

The CLIMb reasoning framework identifies meaningful learning outcomes. An effective method of designing a curriculum is the use of backwards design, which is so named because learning outcomes and assessments (such as CLIMb) are designed prior to lesson plans. The first step of backwards design is to “identify desired results” or the learning outcomes, next to “determine assessment evidence” and, finally, “plan learning experiences and instruction” (Wiggins and McTighe, 2005; Withers, 2016). The CLIMb reasoning framework provides meaningful learning outcomes and identifies key learning stages learners can reach (Scott et al., 2019). This approach to curriculum design has assessments and learning outcomes that drive decisions on course content, in-class activities, and homework for best alignment with appropriately set student expectations at the start of the course.

Bridging the gap between students’ ideas and scientists’ ideas involves multiple steps requiring coordinating adjustments in students’ concepts driven by instruction and drawing on the knowledge and reasoning resources that students bring to the classroom (Jin et al., 2024). Our study provides insights into students’ thinking about immunity and systems; however, other factors beyond cognition, known as epistemic cognition, influence whether and how students incorporate new knowledge within their current understanding.

Our instrument uncovered some epistemic cognitive factors that we will explore further. A respondent in our study shares a first-hand experience as a source of their understanding: “I worked in a pediatric doctor’s office, where we were giving vaccines, all the time. I’ve had a lot of exposure with vaccines, and I noticed a lot of parents requested (…) metal or (…) preservative free [vaccines].” First-hand experience and testimonials influence how individuals are likely to accept new information, referred to as ‘Source of knowledge.’ For instance, if an individual relies solely on the authority that provides the information, they may be less critical of the information (Latifian and Bashash, 2004; Bromme et al., 2010; Chinn et al., 2011). Learners relying on first-hand experience and testimonials may not accept new information without a more direct experience, which can be a barrier for instructors (Bromme et al., 2010; Chinn et al., 2011). Another factor, known as ‘certainty of knowledge’, is how tentative or stable the knowledge is perceived to be. Students have shown poor understanding of scientific uncertainty (Hofer et al., 2010; Sinatra et al., 2014) relative to expert responses we have observed that share their understanding of immunity within scientific uncertainty, such as “It is not entirely known but some memory responses seem to be longer lasting than others.” This quote was provided by a participant who scored high on the CLIMb framework. Unlike experts, novice learners tend to be destabilized by the uncertainty of knowledge, particularly in science (Rosenberg et al., 2022; Moulder et al., 2023).

Epistemic cognition is an intrinsic factor that students bring to class and, with them, potential barriers to integrating new information. Student intrinsic factors, along with environmental factors controlled by the instructor, influence student beliefs and behaviors, a notion well established by Bandora’s Social Cognitive Theory (SCT). This framework, which has been adapted into various frameworks (e.g., Eccles and Wigfield, 2020; Međugorac et al., 2020), has shown wide applicability in psychology, education, business, and health (Schunk and DiBenedetto, 2020). This study can provide instructors with a means to address student cognition, facilitate the transfer of knowledge, and better train our students.

Although our data were collected at two different institutions, both are rather demographically homogenous R1 institutions, and few undergraduate students reached Levels 4 and 5, which presents a limitation in our current participant pool but a focal point of future research cycles as we progress into later stages of the validation process. The CLIMb framework and assessment tool need refining as we continue through the scoring stage and begin the generalization stage of the project (Jin et al., 2019c).

Instructors can use this current framework for assessing the effectiveness of their interventions and curricula in immunology. Our findings on the use of concept-based reasoning about immunity provide insights into how students reason about a system and its purpose at different levels of reasoning (AAAS, 2011; Pandey et al., 2024). Thus, understanding how students develop expertise in immunity has broader implications in epistemic cognition, systems thinking, and the development of immunology and biology literacy.

The concept of systems is a core concept of both biology and immunology (Bruns et al., 2021; AAAS, 2011; Pandey et al., 2023). This study provides insights into how students think about the immune system, which can be applied in different contexts, such as responses against allergens, tumors, transplanted grafts, and self-antigens, as well as the regulation of wound healing. However, more research is needed to elucidate the details of misconceptions and reasoning within these contexts. Although there are other important concepts in immunology, this LP provides insights into how students reason about the immune system, which can support future work involving other core concepts in immunology. The emerging CLIMb framework provides a foundation for future, deeper explorations that focus on other immunological phenomena.

In addition, our findings support a 5-level LP framework on how students develop expert-level reasoning in immunology by developing a broadly applicable skill of systems thinking. Our work provides an important step toward understanding systems thinking, particularly as it is applied using a biological system. The student responses in the context of complex systems, such as the immune system, reveal elements of systems thinking and, therefore, have applications in systems thinking using other systems across contexts. Although systems thinking is a challenging skill to develop, what we learn from how students reason about immunity and the immune system can shed more light on how reasoning frameworks can be used to scaffold learning in systems thinking (National Research Council (U.S.) et al., 2001; Assaraf and Orion, 2005, 2010; National Research Council, 2012; Jin et al., 2019b; National Research Council, 2000).