Although teaching in Spanish universities, as in many other countries around the world, is offered primarily through traditional face-to-face classes, these had to be suspended during 2019 and 2020 due to the COVID-19 pandemic. As a result, faculty had to rely on educational technology so as to adapt their teaching to an online or blended format (Cahapay, 2020; Chung et al., 2022). This has led to increasing interest in the assessment of learners’ satisfaction with this new approach to education, it being recognized that satisfaction is crucial to successful and effective learning (Thurmond et al., 2002; Basith et al., 2020; Barrutia et al., 2021; Flores et al., 2021). Academic satisfaction is generally defined as learners’ appraisal of the extent to which their expectations, needs, and demands have been met during their educational experience (Gento and Vivas, 2003; Moore and Shelton, 2014; González-Peiteado et al., 2017). Given that satisfaction is positively related to learners’ achievement, it is an important factor to consider when designing courses (Bown, 2006; Bernard et al., 2009; González-Peiteado et al., 2017).

Moore (1997) referred to distance education as a concept describing the universe of instructor–learner relationships that exist when learners and instructors are separated by space and/or by time. Moore (1989) described three types of interactions which are essential for effective learning in this context: learner–content interaction, learner–instructor interaction, and learner–learner interaction. The first refers to a learner’s interaction with course contents, lessons, learning activities, learning objects, videos, websites, and projects. The learner-instructor interaction implies two-way communication between learner and instructor, which is necessary to clarify contents, receive and give feedback, and minimize the impact of online education on communication. Finally, learner–learner interaction refers to two-way communication between a learner and other learners. This type of interaction can occur, for example, via email and discussion boards. Palloff and Pratt (2001) added a fourth type of interaction that Strachota (2003) labeled learner–technology interaction, referring to learners’ ability and level of comfort in their interactions with online environments (e.g., use of computers, software, and the Internet). Empirical research has shown that these different types of interaction play an important role with regard to achievement outcomes, the experience of meaningful learning, and learner satisfaction (Driver, 2002; Frey and Alman, 2003; Strachota, 2003; Finlay et al., 2004; Chang and Smith, 2008; Bernard et al., 2009; Abrami et al., 2011; Chang, 2013; Kuo et al., 2013; Alqurashi, 2019; Basith et al., 2020; Ngo et al., 2021).

Drawing on the aforementioned interactions, Strachota (2003) developed the Online Satisfaction Survey (OSS), a 42-item self-report instrument that measures learner satisfaction with online courses. In addition to considering the four types of interaction mentioned above (learner–content, learner–instructor, learner–learner, and learner–technology), the OSS also includes a general satisfaction section. Strachota (2003) performed an exploratory factor analysis that provided support for a structure based on these five factors. The reliability of test scores was satisfactory, with Cronbach’s alpha coefficients ranging from 0.89 to 0.97. Strachota also analyzed differences by sociodemographic characteristics of her sample. Overall, the results showed that females were more satisfied than males with learner–instructor interaction. In addition, learners enrolled in health-related studies and liberal arts and sciences courses were significantly more satisfied with learner–content interaction. Learners from the health occupations were also significantly more satisfied with learner–instructor interaction. Business learners were significantly more satisfied with learner–technology interaction than were their peers in the liberal arts and sciences, and those in health occupations. In general, learners in health occupations and the liberal arts and sciences were the most satisfied with online courses.

More recently, Chang (2013) developed a short version of the OSS, which he called the Learner Satisfaction Survey (LSS). The LSS comprises 25 self-report items reflecting the aforementioned five factors, although with some of the questions reworded so that the questionnaire could be used to assess learner satisfaction with different learning settings (i.e., online, blended, or traditional education). Overall, Chang (2013) found that distance learners were less satisfied with their interactions with content, instructors, and other learners than were traditional learners, but more satisfied with technology interactions. In addition, he found no significant difference in satisfaction between male and female students in the online setting. Importantly, however, he did not analyze the psychometric properties of the LSS. Thus, to the best of our knowledge, there is no validity evidence based on the internal structure of the LSS or on relationships between LSS scores and those on other variables.

Using different instruments for measuring academic satisfaction, including ad hoc questionnaires, other researchers have found that learner satisfaction with online courses is related to learners’ personality characteristics and motivation. Bower et al. (2001) found that learners who were thinkers, emotionally stable, conscientious, and self-assured (using the 16 Personality Factor Questionnaire) were more likely to be satisfied with online courses. Cohen and Baruth (2017) found that learners who scored higher on openness to experiences and conscientiousness (using the Big Five questionnaire) tended to be more satisfied with online courses. Shih et al. (2013) found that extraversion and conscientiousness (using the NEO Five-Factor Inventory) predicted satisfaction with online courses, although motivation was a stronger predictor of satisfaction than was personality. Maqbool et al. (2020) also found positive correlations between learner satisfaction and learner intrinsic motivation, identified regulation, and external regulation (using the Situational Motivation Scale).

Although there are several instruments in Spanish for measuring learner satisfaction, most of them are focused on face-to-face courses, for example, the Student Satisfaction with University Education scale (Mejías and Martínez, 2009) or the Satisfaction Questionnaire for University Students (Gento and Vivas, 2003). In the online setting, the majority of studies has either used ad hoc questionnaires without reporting their psychometric properties (Recio and Cabero, 2005; Bautista et al., 2020; Mercado-Rey et al., 2021) or is focused on a specific approach to or element of online courses (e.g., the use of podcasts or massive, open online courses; Alarcón et al., 2017; Gil-Jaurena et al., 2017; Alarcón and Blanca, 2020). Consequently, there are very few instruments with adequate psychometric properties for measuring satisfaction with online courses among Spanish learners. One exception is the adaptation by González-Peiteado et al. (2017) of the Student Experience Questionnaire for Graduates (Kember and Leung, 2005), which they used to analyze satisfaction with university life of learners who studied at Spain’s National Distance Learning University (the UNED). The adapted version assesses capabilities and the teaching and learning environment through four factors: capabilities, teaching, content, and communication. To our knowledge, however, there are no instruments available in Spanish for measuring learners’ satisfaction with online courses based on the aforementioned four types of interactions.

Given that learner satisfaction plays an important role with regard to academic achievement and is key to successful and effective learning, it is crucial to have adequate tools in our cultural context for measuring this construct in an online environment. The aim of this study was therefore to adapt the LSS into Spanish and to provide evidence of its psychometric properties. The LSS allows a complete evaluation of learner satisfaction in relation to learner–content, learner–instructor, learner–learner, and learner–technology interactions, as well as providing a measure of general satisfaction. This assessment can provide instructors with useful information about the areas most in need of improvement. The main strengths of the LSS are that it can be used at all levels of higher education, its relatively small number of items and its potential applicability to other learning settings (e.g., blended and traditional), thus enabling comparison of learner satisfaction across these different approaches. In adapting the instrument, the Standards for Educational and Psychological Testing (American Educational Research Association et al., 2014) and the recommendations of International Test Commission (2017) were followed. First, confirmatory factor analysis (CFA) was performed to provide validity evidence based on the instrument’s internal structure, including factorial invariance across gender. Second, the reliability of test scores and validity evidence based on relationships between LSS-S scores and scores on measures of personality characteristics and motivation were analyzed. Finally, differences by gender and area of knowledge were analyzed.

The aim of this study was to analyze the psychometric properties of the LSS-S in a sample of Spanish higher education learners. Validity evidence based on the internal structure was obtained, as well as based on relationships with other variables (personality and motivation). In addition, the reliability of test scores and differences by gender and area of knowledge were examined. Overall, the results indicate that the LSS-S has satisfactory psychometric properties and that it is an adequate tool for measuring satisfaction with online courses.

Regarding validity evidence based on the internal structure, the results obtained by CFA indicated a second-order structure with good fit indices. The first-order factors (i.e., learner–content, learner–instructor, learner–learner, and learner–technology interactions, and general satisfaction) are consistent with those described in previous studies (Strachota, 2003; Chang, 2013). The second-order factor subsumes the scores obtained on these factors and supports the use of a total score for satisfaction with online courses. In addition, factorial invariance was found across genders, showing configural and metric invariance. This indicates that the LSS-S has a stable factor structure across male and female students. The reliability of the LSS-S factor scores was also satisfactory, with values of McDonald’s omega ranging from 0.80 to 0.86, similar to the Cronbach’s alpha coefficients obtained with the original version of the instrument (Strachota, 2003).

In terms of validity evidence based on relationships with other variables, correlations of small magnitude were found between LSS-S factor scores and scores on the IPIP-BFM-20. Overall, the most relevant finding was the positive correlations between LSS-S factor scores and scores on conscientiousness, as well as the negative correlation between both learner–technology interaction and total scores with neuroticism. These results indicate that people who are organized, diligent, and efficient tend to be more satisfied with online courses and with learner–content, learner–learner, learner–instructor, and learner–technology interactions. Conversely, people who are anxious, nervous, and prone to anger and irritation tend to be less satisfied with technology and, accordingly, with online learning. These findings are partially consistent with other research that has likewise reported a positive correlation between conscientiousness and academic satisfaction with online courses (Shih et al., 2013; Cohen and Baruth, 2017). The results are also in line with those of Bower et al. (2001), who found that those learners who were emotionally stable and conscientious were more likely to be satisfied with online courses.

The present analysis also showed that LSS-S factor scores correlated positively with intrinsic motivation and identified regulation, and negatively with amotivation, although in this case the magnitude of relationships was moderate or strong. These results suggest that learners who enroll in an online course for the pleasure and satisfaction derived from doing it, and whose conscious values provide them with internal motivation to achieve the course objectives tend to be more satisfied with online courses and with the different types of interactions that form part of the teaching-learning process. Conversely, learners who do not perceive the contingencies between behavior and its consequences tend to be generally less satisfied with the online format. These findings are consistent with previous research suggesting that academic satisfaction is more strongly related to motivation than to personality characteristics (Shih et al., 2013).

Regarding gender differences, the results showed that females reported a higher level of academic satisfaction, except in relation to learner–technology interactions, where scores were higher among male students. Strachota (2003) only found that females were more satisfied than males with learner–instructor interaction, whereas Chang (2013) did not find significant differences between male and female students in satisfaction with online courses. These discrepancies may be related to procedural issues. For example, in the studies by Strachota (2003) and Chang (2013), learners voluntarily chose the online course, and there was a higher proportion of female and full-time workers in the sample. Conversely, in the present study, learners were obliged to take part in online education (due to the COVID-19 pandemic), the sample consisted of a similar number of males and females, and the majority of participants was full-time students and did not work.

With respect to differences by area of knowledge, students from the field of Engineering and Architecture had the lowest scores for academic satisfaction with online courses, except in relation to learner–technology interactions, where they scored the highest. This finding may be related to the contents of these degree programs, which are more technical and require the use of different educational resources to those used in other areas of knowledge, as a result of which, these students found it more challenging to adapt to exclusively online learning. The fact that students from Engineering and Architecture were more satisfied with learner–technology interactions may be a reflection of their greater understanding of technology and a greater ability to manage it.

Overall, the LSS-S scores obtained in the present sample indicate that learners were moderately satisfied with online courses. The mean for total scores on the LSS-S was 56.20, out of a maximum of 100, which indicates that the teaching-learning process needs to be improved. Scores on the LSS-S have practical implications, as examining them can provide instructors with useful information about the areas most in need of improvement. In the present sample, for instance, scores suggest that particular attention would need to be paid to learner–learner interactions. The fact that improvements are indicated is not, however, surprising, insofar as the COVID-19 pandemic meant that faculty had to adapt all their face-to-face teaching to an online format in a short period of time, including revising contents and implementing the use of new resources and technology. When adapting their teaching to an online environment, instructors must take into account the different types of interaction involved and remain mindful that teaching in this environment requires specific skills and techniques to create an effective learning experience. Course design is crucial for promoting learning, and it is important to ensure the inclusion of meaningful instructional content and activities that increase students’ motivation and facilitate their reasoning, problem solving, and critical thinking skills. Instructors should also be proactive in promoting and facilitating online communication with their students, providing feedback, and supporting them with individualized attention. Similarly, they should foster the exchange of information among learners, encouraging engagement with peers through the discussion of ideas, opinions, etc. Finally, students must also have the technological skills required for participation in online learning environments. In this respect, universities must be proactive, providing technological support and designing activities that help students to improve their skills and succeed in online learning.

This study has a number of limitations that should be acknowledged. First, participants were recruited solely through the University of Malaga and this might limit the generalizability of results. It would thus be interesting in future studies to include learners from other geographic areas of Spain, as well as from Latin American countries. Second, validity evidence for the LSS-S was provided based on associations with motivation and personality traits, and it would be interesting in future research to study the relationship with other variables such as self-efficacy, stress, and academic achievement. Finally, relationships between variables were examined using a correlational analysis, and hence, no causality can be inferred. Further longitudinal studies are therefore needed to extend knowledge about the nature of these relationships.

Despite these limitations, this study has a number of strengths, not least that it is the first study to analyze the psychometric properties of the LSS in a relatively large sample of university learners. Specifically, validity evidence based on the instrument’s internal structure have been provided, including factorial invariance across genders, as well as validity evidence based on relationships with other variables such as personality characteristics and motivation (variables that have not previously been considered in relation to the LSS). Overall, the results indicate that the LSS-S has satisfactory psychometric properties and that it is an adequate tool for measuring satisfaction with online courses among Spanish learners in higher education. Importantly, the LSS-S can also be used to assess academic satisfaction with traditional and blended learning approaches. Further research is required to provide evidence of its psychometric properties in these contexts.

The present study provides evidence of the psychometric properties of the LSS-S in a sample of Spanish university learners, a population not previously studied. The results reveal a factor structure with adequate fit indices based on five first-order factors (learner–content, learner–instructor, learner–learner, and learner–technology interactions, and general satisfaction) and one second-order factor (total score of satisfaction). Reliability of LSS-S factor scores may be considered satisfactory. Scores on the LSS-S were positively correlated with scores on conscientiousness, intrinsic motivation, and identified regulation, and negatively correlated with scores on neuroticism and amotivation. Although the magnitude of correlations with personality traits was small, those with motivational factors were moderate or strong. The results indicate that the LSS-S has satisfactory psychometric properties and that it is an adequate tool for measuring satisfaction with online courses among Spanish learners in higher education. Furthermore, it is a short and simple questionnaire that can be administered collectively. The scores obtained provide instructors with direct feedback from learners regarding their level of satisfaction with different aspects of the teaching-learning process, thereby highlighting areas in need of improvement. Learner satisfaction is a key factor for successful and effective learning and instructors should take into account the different types of interaction involved to create an effective learning environment.

The original contributions presented in the study are included in the article/Supplementary Material, and further inquiries can be directed to the corresponding author.

The studies involving human participants were reviewed and approved by the Experimentation Ethics Committee of the University of Malaga (45-2021-H). The patients/participants provided their written informed consent to participate in this study.

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.