During the Spring 2020 semester, the activity was administered on instructor–provided tablets within a single three–hour laboratory (Context – formalized) and with a teaching assistant in person. The students completed the activity with a partner (Communication – tightly coupled pairs). The partners shared an iPad and had to advance through the AR experience together; however, they were each responsible for submitting individual work. During the spring semester, students would document their predictions and provide explanations on the worksheet while comparing and observing the AR components in the App.

The Fall 2020 semester was the first full semester of remote learning due to the COVID-19 pandemic. The students completed the activity in their own space (Context – independent), using personal devices. Students had an entire week to complete the activity. While students could reach out to their teaching assistant via Zoom throughout the week, students did not work with partners (Communication - isolated learners). During the fall semester, students only wrote their explanations on the worksheet. Each AR experience was modified to require students to compare, make a prediction, and make an observation within the App.

The face-to-face data were collected from 114 students across six laboratory sections in the spring. The remote data were collected from 154 students across eight laboratory sections in the fall. All participants were concurrently self-enrolled in the Organic Chemistry II laboratory course lecture. Students participated in the study by taking a pre-and post-survey to gather information about their perception and user experience. Participants were not given incentives to complete the activity or survey as it was part of their coursework; however, they had to consent for their feedback to be used for research per our Institutional Review Board guidelines. In the face-to-face and remote environments, most students were sophomores or juniors (80.7, 67.5%) and self-identified as female (57.9, 54.6%). They reported learning about ¹H NMR in their lecture course before the laboratory (64.9, 61.7%). Less than one-third of students reported previously using AR (32.9, 29.2%).

Using Qualtrics software, participants completed a pre-survey before the activity and a post-survey immediately after. We analyzed the data related to each research question within each environment and compared environments.

To investigate student user experience (research question one, RQ1), we used the user experience questionnaire (UEQ) in the post-survey (Laugwitz et al., 2008; Schrepp et al., 2014). The questionnaire contains 26 items across six scales:

The UEQ is a semantic differential survey in which students are given two opposing words and asked to indicate their preference using a seven-point scale (−3 to +3). Responses can be positive (greater than +0.8), neutral (between +0.8 and-0.8), or negative (less than −0.8) (Santoso et al., 2016). All six scales showed relatively acceptable levels of reliability across both semesters (Supplementary Table S1), indicating that every item in the scale was measuring something similar to other items within the scale (Taber, 2018). A MANCOVA was utilized to analyze data from the UEQ across semesters. The outcome variables for the analysis were the six UEQ scales, with the independent variable being the learning environment. Research has shown that prior knowledge influences experience and performance (Kohl and Finkelstein, 2006; Rittle-Johnson et al., 2009; Braithwaite and Goldstone, 2015). The analysis also controlled for students’ prior experience with AR technology and NMR content. All assumptions of the statistical test were met (Field, 2009).

The post-survey contained more items to understand students’ overall perception of the activity (RQ2). Students rated each item on a scale from 1 (Strongly disagree) to 5 (Strongly agree). The prompts were:

Students were asked to indicate if they experienced challenges with the worksheet and the App via a yes/no item. In both cases, students could provide an open-ended response to describe their experiences with the activity:

Data from items relating to student perceptions of the App and worksheet violated the assumption of normality, and therefore, non-parametric tests were utilized. Mann–Whitney U tests were run for each item, with perception as the outcome variable and learning environment as the independent variable. A separate chi-square analysis was used to explore the frequency of students who stated they experienced challenges using the worksheet and application in each environment. The open-ended responses underwent a content analysis to identify common patterns across the participants within each environment (Patton, 2002). One coder analyzed 20 % of the data and developed a codebook. The codebook was shared with a second coder who coded the same 20 % of the data. The coders met and refined the codebook using constant comparative analysis (Glaser, 1965). Both coders separately analyzed the remainder of the data. After independent coding, the two researchers discussed their codes until a 100% negotiated agreement was reached (Campbell et al., 2013; Saldaña, 2013). Peer debriefing and negative case analysis were used to ensure the credibility of the findings (Lincoln and Guba, 1985).

Lastly, we used pre-post survey questions to investigate if the activity adjusted students’ perception of the importance of chemistry visualizations or student knowledge of ¹H NMR (RQ3). The first four items were adapted from a survey to evaluate BiochemAR, another augmented reality educational tool (Sung et al., 2020). The last two items were author-generated and specific to ¹H NMR. Students rated the following items on a 1 (Strongly disagree) to 5 (Strongly agree) scale:

Data from these items violated the assumption of normality, and reverse score and log transformations were used to correct the data. A mixed (2×2) MANCOVA was run to understand differences between environments over time. The outcome variables for the analysis were item responses at the pre-and post-survey, with the independent variable being the learning environment. Interaction effects (differences between environments over time) and main effects (differences between environments or differences over time) were investigated.

Across both settings, students overwhelmingly valued the 3D visualizations and how they helped them to understand the content (see R005, Table 4) and “make connections between the theory and the practice” – R084. Students discussed how the color coding helped them link the hydrogen atoms to specific chemical shifts, and the animations helped them visualize how electrons interact with magnetic fields. However, some students struggled with the fact it was augmented. Like R103 in Table 4, participant R078 stated, “I did not like how I could not “freeze” the screen and look at it while I was looking at the questions. It made it hard to hold my phone up and also read on my laptop.” Freezing the screen would make the image appear more like a 3D object on a screen than an augmented reality image. These sentiments lead to questions about the content and contexts in which AR affords more than non-augmented virtual representations on a desktop or website application when learning chemistry.

While the statistical analyses controlled for student exposure to AR and NMR, the open-ended survey responses revealed that students who had not covered ¹H NMR in their lecture struggled with the activity. This sentiment was present regardless of environment and can be evidenced by participants R111, F091, and F109 in Table 4. As participant R082 stated, completing this activity without prior content exposure requires learning “new material and devices I was not familiar with.” This could potentially increase student cognitive load, a significant challenge in using AR in teaching chemistry (Cheng and Tsai, 2013). Students may perceive AR activities more positively if they have some experience with the content. Other studies have been designed in which the AR component was incorporated towards the end of the instructional unit (Behmke et al., 2018) or after foundational knowledge of the topic had been covered in the course (Cai et al., 2014; Sung et al., 2020). Educators could benefit from research that provides insight into how user prior knowledge, the amount of material, and task complexity impact chemistry learning with AR tools.

Students’ perceptions scored higher in the face-to-face environment where students could communicate with a partner. However, most perceptions were still rated positively in the remote and independent environment. Students in the face-to-face environment gave significantly higher ratings for their willingness to reuse the App, the effectiveness of the App in helping them study the content, the worksheet’s effectiveness in assisting students with course content, and overall lab engagement. In both environments, students reported an increased understanding of ¹H NMR concepts and problem-solving ability, but this increase was significantly lower in the remote learning environment. While we cannot disentangle the remote from the independent work or the in-person from the collaborative, studies show that interpersonal interactions influence students’ attitudes toward science (Wei et al., 2019), and when students collaborate with mobile learning activities, they perceive greater improvements in learning than without collaboration (Burke et al., 2021). Students who work collaboratively are more likely to discuss their reflections and ask one another questions than students who work independently (Chi and Wylie, 2014). Students in the remote environment echoed this point, as one stated they were challenged “without the feedback and engagement of a lab partner/table to do it with” (R060). Although mobile applications have the affordance of portability, there are other nuances to consider that can impact student learning and experience (Frohberg et al., 2009; Sung et al., 2020). Research around AR use in collaborative vs. independent chemistry learning environments, especially if some students are inexperienced with AR, would be beneficial to support educators in adapting these technologies in the classroom.

Although the results show significantly lower ratings in user experience in the remote environment, student ratings still fell within the neutral zone of the scale. When face-to-face, students had paper copies of the worksheet and used the tablet and App to examine the worksheet. When remote, most students did not choose to print the worksheet. They reported using a mobile device to examine the worksheet projected on a laptop or desktop. This overuse of screens is not an efficient way to use AR and likely decreased student user experience and perceptions of the activity (R127, Table 4). Furthermore, though mobile devices are widely adopted, providing students with a device may be most beneficial, especially if using an app for the first time. Our data demonstrate that using personal devices made some students frustrated by the screen size or the battery’s strength (see R135 and R103 Table 4).