There has been widespread effort over the last decade to improve the quality of teaching in undergraduate STEM courses (American Association for the Advancement of Science, 2011; President’s Council of Advisors on Science and Technology, 2012). Researchers have identified scientific teaching practices that relate to improved student outcomes. Examples of these practices include inclusive teaching, active learning, formative assessment, and the modeling of science reasoning and metacognitive strategies (Handelsman et al., 2004). Workshops and other faculty development opportunities have become increasingly available to teach faculty how to incorporate these practices into their instruction (Simonson et al., 2022). To help make the course modification process more manageable, faculty are often advised to make small, incremental changes to their courses (Stein and Graham, 2020). However, if these changes are too small, the new activities or teaching practices might not have a noticeable or immediate impact on student outcomes. Without positive feedback from students, faculty may prematurely abandon these teaching strategies and resist making subsequent modifications. The current study investigates the frequency with which newly trained faculty were able to implement scientific teaching practices into their courses and provides initial guidance about the level of implementation needed to impact student outcomes.

There is interest from teachers, researchers, administrators, and policymakers to improve learning outcomes and student retention in STEM fields (Handelsman et al., 2004; President’s Council of Advisors on Science and Technology, 2012; Freeman et al., 2014; Bradforth et al., 2015). While there are several factors that can impact retention in STEM majors, offering high-quality teaching can have a dramatic, positive effect (Sithole et al., 2017). Research on evidence-based teaching practices such as active learning, formative assessment, and inclusive teaching has been shown to both increase engagement and the likelihood that students will remain in a science major (Hanauer et al., 2006; Auchincloss et al., 2014; Corwin et al., 2015). Nevertheless, implementing evidence-based teaching practices has been an enduring challenge in higher education—with up to 90% of STEM students citing concerns about teaching quality in their introductory courses (Seymour and Hewitt, 1997; Akiha et al., 2018). There is evidence to suggest that student dissatisfaction with teaching is at least partially due to incongruence between their expectations and their actual experience (Brown et al., 2017). For example, many first-year students are used to smaller high school classes that support active engagement with the content, peers, and instructors (Meaders et al., 2019). On the other hand, many introductory STEM courses in higher education still rely on passive lectures where students are more likely to report feeling isolated, dissatisfied, and disconnected from the content (Seymour and Hewitt, 1997; Cooper and Robinson, 2000).

Fortunately, STEM educators have recognized the importance of improving teaching quality in introductory STEM courses (American Association for the Advancement of Science, 2011; National Research Council, 2012; President’s Council of Advisors on Science and Technology, 2012). An increasing number of colleges and universities host faculty development workshops to improve the quality of teaching. There are also national programs, such as the Summer Institutes for Scientific Teaching, that have provided instruction on how to implement evidence-based teaching practices. Initial evidence indicates that these faculty development efforts have: (a) positively impacted participants’ motivation (Meixner et al., 2021); (b) increased pedagogical knowledge; (c) heightened the value faculty place on teaching (Palmer et al., 2016); (d) improved syllabi (Hershock et al., 2022); and subsequently improved their students’ outcomes (Durham et al., 2020).

Despite the increasing prevalence of faculty development opportunities and the positive impact of these programs, implementation of evidence-based teaching practices still varies considerably across courses and instructors (Fairweather and Paulson, 2008; Durham et al., 2018). Participation in these programs, at most institutions, is voluntary. Faculty frequently cite misaligned incentives and time constraints as barriers to both attending training and fully implementing learner-focused instruction (Brownell and Tanner, 2012). Therefore, despite widespread efforts, lecture is still the most common instructional method used in introductory STEM courses (Stains et al., 2018).

In an effort to obtain faculty buy-in and to make the course (re)design process more manageable, faculty are often encouraged to make small changes over several semesters with the goal of incrementally improving their course over time (Stein and Graham, 2020). Nevertheless, gradual implementation may not produce immediate results. Without an immediate impact on student outcomes or satisfaction, faculty may prematurely conclude that the new instructional methods are not effective in their specific learning context and revert to their prior pedagogical strategies. Therefore, it would be beneficial to understand the level of implementation that is necessary to see an impact on student outcomes. This would help faculty and faculty development professionals determine how to pace their initial modifications and increase the likelihood that both faculty and students see benefits.

Research that examines how frequently these teaching principles need to be applied within and across courses is still in its early stages. In a recent study, Reeves et al. (2023), found that utilizing evidenced-based teaching across multiple courses and instructors had a cumulative benefit to student learning and persistence in STEM. In that study, however, the frequency of implementation was not examined, as students were just asked to indicate if various evidence-based teaching practices occurred at all during the course. The purpose of this study is to examine how the frequency of implementation, as perceived by students, impacts student outcomes. Such findings would have the potential to reduce perceived (or real) barriers to initial implementation and encourage faculty to persist with the implementation of evidence-based teaching practices.

Two types of analyses were conducted. The first type of analysis looked at EBT exposure at the individual level and the second was conducted at the classroom level.

The study examined how the level of EBT implementation impacts student outcomes. A majority of instructors who completed the training at the summer institute utilized each EBT practice (e.g., active learning) several times a semester. This is consistent with advice given by faculty developers to make gradual incremental changes to instructional methods (Stein and Graham, 2020). Nevertheless, small degrees of implementation may not produce noticeable differences in student outcomes. The results from both analyses—classroom level and student level – in this study suggest that the perceived frequency of EBT implementation is associated with student outcomes. Including EBTs every couple of classes relates to more positive student outcomes when compared to only including these practices several times across the semester. Intuitively, the result makes sense as it follows the adage “more is better,” but also implies that instructors may need to make more sudden or substantial changes to courses that primarily relied on lectures and teacher-directed learning activities.

The population of instructors in this study were predominantly employed at large research institutions, which reflects the participants at the Summer Institutes. Faculty at these institutions may have different pressures (i.e., rewarded for focusing more on research than teaching) and be more likely to teach larger classes, particularly at the introductory level when compared to faculty from teaching-focused colleges (Brownell and Tanner, 2012). These factors could influence the time that faculty can dedicate to instructional design and building trust with their students (Wang et al., 2021). However, the results suggest that it is important for these faculty to dedicate time to including EBTs regularly throughout the course. Including more EBTs may help mitigate student dissatisfaction with learning expectations that can be common in large classes by making learners feel more engaged in the content (Seymour and Hewitt, 1997; Cooper and Robinson, 2000). Future studies should examine whether the impact of the frequency of EBT implementation varies based on class size or institutional teaching practices. It is possible that small class sizes and the overall level of an institution’s adoption of EBTs could impact the amount of implementation needed to impact student performance within a particular course. However, it is also possible that there is a level of implementation that has diminishing returns (e.g., students get tired of reflecting everyday).

The study had several limitations. First, the implementation of EBTs by instructors was reported by students. The students may not have been fully aware of when instructors were trying to utilize some techniques and could have inaccurately perceived, or remembered, the frequency with which they were incorporated into the course. Measuring frequency with a Likert scale may have mitigated the impact of these inaccurate perceptions, though it also limits the precision of the measurement and any subsequent instructional recommendations. Future studies would benefit from taking an observational approach to measuring implementation. If done in conjunction with student surveys, this would also help researchers investigate the variability in student reporting of EBTs within classrooms.

Second, the outcome data was also self-reported by students. The factor analyses provided evidence that the subscales are measuring unique concepts. However, this result could be strengthened with the inclusion of knowledge tests or other indicators of student achievement in future studies. Finally, the faculty who were recruited for this study attended the Summer Institutes for Scientific Teaching, which may signify a higher level of interest in pedagogy and instructional design, when compared to other faculty. Successful implementation of EBTs likely varies based on the motivation of faculty (Richardson et al., 2014). Future studies would benefit from examining how a gradual course design process interacts with faculty motivation for teaching. Some of the faculty in this study also may have included EBTs in their instruction before their training, so we cannot be sure of how many modifications were made to courses because of the summer workshop. This information would be useful for making more specific recommendations on how many changes to instruction should be made at any given time.

Despite the limitations, these findings present initial evidence to suggest that more frequent utilization will lead to larger benefits, and incorporating EBTs every couple of classes will likely lead to a more noticeable impact. Prior to this study, the impact of the frequency of EBT had not been examined. Future studies should evaluate faculty perceptions of their implementation to determine if the frequency of EBT use impacts their perception of student performance, teaching self-efficacy, job satisfaction, and intention to continue modifying their instruction to include more EBTs. A range of optimal implementation likely exists for each teaching practice. This study provides evidence that infrequent implementation does not lead to maximum benefits. Additional insight into these variables would allow researchers and faculty developers to provide better guidance on how to pace and scale course modification in a manner that benefits both faculty and students.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving humans were approved by Yale University IRB Human Subjects Committee. The studies were conducted in accordance with the local legislation and institutional requirements. The ethics committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants’ legal guardians/next of kin because it was designated as exempt survey research.

PR: Writing – original draft, Writing – review & editing. MB: Writing – original draft. JG: Writing – review & editing. CW: Writing – original draft. DH: Writing – review & editing. MG: Writing – review & editing.