According to Activity Theory (Engeström and Sannino, 2020; Leont’ev, 1978), individuals engage in an activity because of a motive, which arises from a match between their need and an object that can satisfy it—something they expect to achieve through participating in the activity. The most powerful motives create personal meaning—a sense of value and significance of the activity in one’s life. Therefore, an educational system can be considered effective if it offers objects that align with students’ needs, allowing them to perceive the activity as meaningful and valuable (object–need fit).

Ace Yourself aims to support students in their transition by encouraging them to achieve specific “objects” in terms of Activity Theory—namely, improvements in social, study, and personal skills (see Table 1). These skills are based on the framework “Vaardighedenraamwerk voortgezet onderwijs - hoger onderwijs (VO-HO)” developed by the Ho-skills working group of the Rotterdam Partnership Working together for a better connection with vo-ho (2023). In essence, Ace Yourself assumes that students need these skills to facilitate their university transition. However, students may encounter problems and tasks requiring different skills, solutions unrelated to skills, or they may lack an understanding of how specific skills could address their challenges (Thompson et al., 2021). This misalignment—referred to as a lack of need–object match—raises the first research question: How adequately do the skills suggested by the Ace Yourself app address the university-related problems students experience?

In the more recent adaptation of Activity Theory, performing activities can lead to various outcomes—the tangible results of those activities (Engeström and Sannino, 2020). Following this perspective, I propose that an educational system can be considered effective if students achieve what was intended, creating an outcome–object fit. Ace Yourself primarily offers students scientifically-based information about social, personal, and study skills, including strategies, step-by-step guides, and examples of relevant problems and tasks. The app assumes that these skills will facilitate students’ transition to university. Consequently, the two primary expected outcomes of learning with Ace Yourself can be defined as “Supporting the transition to university” and “Developing study, social, and personal skills.” This brings me to the second research question: How does the Ace Yourself app contribute to students’ basic psychological needs (as defined by Ryan and Deci, 2020), the development of study, social, and personal skills, and the resolution of students’ university-related problems?

According to Activity Theory (Engeström and Sannino, 2020; Leont’ev, 1978), a tool acts as a mediator between an individual and the actions that constitute an activity, serving as an instrument to facilitate the performance of actions and the achievement of goals. Ace Yourself app.

Therefore, I propose that an educational system is effective if its tools support students in carrying out actions directed toward their desired goals. In this context, the Ace Yourself app can be considered a tool. This leads to the research question: What features of the Ace Yourself app support or hinder students in achieving their learning goals?

According to Activity Theory (Engeström and Sannino, 2020; Leont’ev, 1978), every activity occurs within a specific context, and individuals adjust their actions based on the conditions of their environment. Therefore, I propose that an educational system is effective if it aligns with the physical and psychological learning environment. The Ace Yourself app, designed to enable students to learn anytime and anywhere, must also adapt to these conditions. This leads to the research question: Where and when do students prefer to use the Ace Yourself app for learning?

According to Activity Theory (Engeström and Sannino, 2020; Leont’ev, 1978), individuals have different motives tied to various activities, which are organized hierarchically. These motives and activities could compete, with people allocating more time and effort to those they prioritize. Based on this, I propose that an educational system is effective if it helps students prioritize it over competing activities, ensuring time and space for learning. Since the Ace Yourself app is not part of the core curriculum, students are likely to face significant interferences. This leads to the research question: What factors help or hinder students in prioritizing the Ace Yourself app among other activities?

Activity may involve various rules and roles that influence its execution (Engeström and Sannino, 2020; Leont’ev, 1978). The division of labor defines how actions are distributed among the roles involved, aiming to facilitate collaboration and achieve the desired object. Based on this, I propose that an educational system is effective when all necessary roles are clearly identified, and actions are distributed in a way that supports participants in reaching their goals. This leads to the research question: How do students perceive their roles and responsibilities when learning with the Ace Yourself app?

All the aforementioned elements of activity are interconnected, and tensions or contradictions may arise between them (Engeström and Sannino, 2020). Thus, it is essential not only to analyze each element separately but also to evaluate the entire system holistically. An educational system can be considered effective when potential contradictions between its elements are identified and addressed. Adopting this systematic perspective can reveal unanticipated aspects of the activity system—here, the learning experience with the Ace Yourself app. This leads to the final research question: What tensions, contradictions, and insights can be identified between the aforementioned elements?

To evaluate the app’s contribution to students’ basic needs and their study, social, and personal skills, I conducted within-group (pre-post survey) comparisons for each group and between-group (post-survey) comparisons between the experimental and control groups. Due to non-normal data, the Wilcoxon signed-rank test was used for within-group comparisons, and the Wilcoxon rank-sum test for between-group comparisons. Additionally, a thematic analysis was performed to categorize students’ reported problems, supplemented by proportion and frequency analyses to identify differences within and between groups.

Table 2 indicates that in the within-group (pre-post survey) analysis, the Wilcoxon signed-rank test revealed no significant changes in students’ needs satisfaction, except for a decrease in competence frustration in the personal group (M_Pre. = 4.09; M_Post. = 3.55), W = 15, p = 0.00, p_adj = 0.02, with a large effect size (r = 0.61). Similarly, Table 3 shows no significant changes in skills, apart from an increase in confidence in one’s own abilities within the personal group (M_Pre. = 4.35; M_Post. = 4.87), W = 64, p = 0.048, p_adj = 0.048, with a medium effect size (r = 0.47).

Table 4 reveals that in the between-group (post-survey) analysis, the Wilcoxon rank-sum test showed no significant differences in needs satisfaction between the control and experimental groups, except for a reduction in relatedness frustration in the social group (M = 2.08) compared to the control group (M = 3.05), W = 593.5, p = 0.01, p_adj = 0.03, with a medium effect size (r = 0.36). Similarly, Table 5 indicates that no significant differences in skills were observed between the control and experimental groups.

It is important to note that the sample size was relatively small, varying from 20 to 41 respondents per group across different comparisons. Consequently, post hoc power analysis revealed generally low statistical power for the post-survey, with values ranging from 0.05 to 0.19 for non-significant results. This suggests that the study may have been underpowered to detect smaller effects. Still, the effect sizes for non-significant results were relatively modest, with the highest effect size being 0.28.

To measure learning outcomes, students were asked to list the problems they face at university, and their responses were thematically analyzed. A total of 146 students identified 485 problems in the pre-survey and 550 problems in the post-survey. These problems were categorized into personal, social, and study domains based on the skills framework underlying the Ace Yourself app (see Table 1). A four-sample test for equality of proportions without continuity correction was conducted to examine differences in the frequency of study, social, personal, and other problems across the study, social, personal, and control groups in both pre- and post-measurements.

As shown in Table 6, no significant differences were found, except for study-related problems in the post-survey (X² = 7.988, p = 0.046). However, pairwise comparisons of proportions with Bonferroni adjustment revealed no statistically significant differences between any specific group pairs for study problems. This indicates that there were no meaningful differences between the experimental and control groups in the frequency of reported problems. Additionally, Figure 1 illustrates that the overall frequency of reported problems remained nearly unchanged from the pre- to post-survey.

To evaluate the alignment between students’ needs and the skills provided by the app, I analyzed the types of problems students face at university and how they value the skills suggested by the app. First, students were asked to describe the problems and tasks they encountered or anticipated at university. As shown in Figure 1, the most frequently mentioned issues were related to study and personal challenges, followed by other problems. Social problems were the least mentioned in both the pre- and post-surveys. Most of the reported problems aligned with the app’s skills framework (See Table 1). However, 117 out of 485 problems in the pre-survey and 126 out of 550 in the post-survey were categorized as “other problems,” which fall outside the current framework.

*Several skills from the current version of the Ace Yourself app were not represented in students’ reported problems: Create a diverse environment (7.5%), Presenting (7.5%), and Research skills (15.6%).

Second, to evaluate students’ perceived value of the skills suggested by the Ace Yourself app, we analyzed reflections from 82 students in the experimental groups on which skills they found valuable or unclear, and why. While five students did not respond, the remaining 77 expressed appreciation for the skills, with 173 categories extracted from their reflections indicating this sentiment. However, some students (N = 19) found the value of certain skills unclear, as reflected in 30 categories. The red boxes in the center of Figure 1 highlight the three most valued skills: “confidence in your abilities” from the personal domain (39 categories; 22.5%) and “study strategies” (29 categories; 16.8%) and “planning and predicting” (25 categories; 14.5%) from the study domain.

Students provided four main reasons for valuing a skill and five reasons for questioning its value. The most common reasons for considering a skill valuable were its perceived fundamental importance (37% of responses), its usefulness in overcoming challenges (25%), its contribution to academic success (24%), and its ability to foster a positive mindset (3%). An additional 10% of responses did not align with these themes. On the other hand, students questioned the value of a skill primarily due to a lack of perceived utility or implementation challenges (43%), already possessing the skill (30%), doubts about the app’s ability to develop the skill effectively (17%), vague or unclear skill descriptions (7%), and limited personal experience with the skill (3%).

Figure 2 shows that no one from experimental groups used the app daily and mostly students preferred to use the app 1–3 times a month.

To explore how the Ace Yourself app supported or hindered learning, we analyzed reflections from 82 students in the experimental groups. These reflections included their general impressions of the app, features that facilitated or distracted from learning, reasons for not completing the modules, and their experiences with other tools designed to develop personal, social, and study skills.

First, among the 79 students who reflected on their general impressions of the app, most had a positive view: 72% of the 131 extracted categories indicated a positive impression, while 28% reflected a negative impression. Eighteen categories expressed a general positive impression, while the remaining 113 were divided across six more specific topics that shaped their perceptions (see the upper left part of Figure 3). Positive impressions were primarily linked to the app’s learning effectiveness (27% of categories) and content quality (15%), whereas negative impressions were mostly related to a perceived lack of learning effectiveness (9%) and lack of engagement (5%). Ease of use, visual appearance, and technical issues were also mentioned by students, though less frequently than the categories discussed above.

Second, feedback was collected from 77 students on helpful features of the app and 72 on distracting ones. Among the 156 extracted categories, 32% reported no distractions, 2% found all features helpful, and 51% identified specific helpful features. Conversely, 3% felt nothing supported them while studying, while 15% mentioned distracting features. As shown in the upper-right part of Figure 3, among the nine identified features, the most frequently mentioned helpful aspects were the modules themselves (19%) and the content’s quality, clarity, and usability (10%), although 4% highlighted content overload as a distraction.

Third, 63 students shared what prevented them from completing the modules. As shown in the lower-left part of Figure 3, the primary reason was a lack of time (39%). Finally, we asked students how they determined that they had learned something. From the 77 responses, 86 categories were extracted. Among these, 6% indicated that students felt they had not learned anything, while only 8% credited the app’s assessment features for helping them track their learning progress. The lower-right part of Figure 3 illustrates that most students gauged their learning through practical application of the skills gained from the app (31%) and by observing changes in their thoughts and behavior (23%).

Finally, we asked students to compare Ace Yourself with other tools for developing study, social, and personal skills. Among 28 students, 70 categories were extracted: 43% reported experience with other tools, such as faculty courses (20%), coaching (17%), practical exercises (13%), personal improvement activities (10%), classroom learning (7%), and mindfulness exercises (7%). In comparison, Ace Yourself was highlighted for seven advantages: clarity, foundational support, conciseness, interactivity, convenience, guidance, and flexibility. However, students also noted 19 disadvantages, including lack of personalization (21%), learning alone (21%), absence of practice support (16%), and other limitations (21%).

We asked students where and when they preferred to study with Ace Yourself (context), what other activities interfered when they considered studying with the app (activity system), and how they perceived their role and responsibilities while learning with it (division of labor). Figure 4 illustrates the percentage of categories extracted from students’ reflections on these themes.

First, from the responses of 79 students, 80 categories related to location and 61 categories related to time were extracted. The upper part of Figure 4 illustrates that most students preferred learning at home (79%), typically during the evening or night (44%). Among the 71 students who provided answers, 85% reported no situations where the app’s design prevented them from learning in place and time where they wanted to. However, a few students cited challenges, including technical issues (7%), crowded or noisy environments (6%), and difficulties completing assignments requiring peer collaboration (3%).

Second, students were asked whether they encountered situations where they had to choose between learning with Ace Yourself and engaging in another activity, and why they chose Ace Yourself. From the responses of 76 students, 77 categories were extracted: 42 indicated choosing Ace Yourself, while four chose another activity. The bottom-left part of Figure 4 highlights that the primary reasons for choosing Ace Yourself were the desire for self-development and curiosity (33%), as well as the app’s perceived utility value (24%).

Finally, students were asked how they perceived their roles and duties in learning with Ace Yourself. From 55 responses, 62 categories were extracted. The bottom-right part of Figure 4 reveals that most students saw themselves as active and self-directed learners (35%) or focused on testing the app and contributing to the research (31%).

To evaluate how effectively the skills suggested by the Ace Yourself app address the university-related problems students experience (RQ1), I further discuss the alignment between the app’s suggested skills (objects) and students’ needs—specifically, whether students perceive these skills as meaningful and whether the app addresses all the problems students encounter. The upper left box of Figure 5 summarizes the key findings for the elements “object” and “meaning,” showing that the majority of students valued the skills provided by the current version of the app. Especially because if the skills perceived fundamentality and utility. Future versions of the Ace Yourself, and similar learning mobile apps, could include brief descriptions explaining why each skill is fundamental and how it addresses common student challenges. Such explanations could help students who may lack meaning and doubt the relevance or value of certain skills. Future research can examine how the specificity with which an educational activity communicates what it suggests to students influences students’ impression and perceptions of its value (Vansteenkiste et al., 2004; Priniski et al., 2018).

The upper central box of Figure 5 summarizes the key findings related to the element “need.” Most of the problems listed by students aligned with the skills framework underlying the Ace Yourself app (see Table 1), indicating that the framework addresses the majority of students’ challenges. However, this skills framework does not address significant areas such as study-life balance, well-being, and mental health, which were also frequently highlighted by students. Future versions of Ace Yourself and similar learning mobile apps could benefit from considering not only the problems and needs students experience on general but also which of these are prioritized and most likely to occur.

Now, I discuss how Ace Yourself contributed to element “student’s outcomes”—basic needs in autonomy, competence, and relatedness; social, study, and personal skills; and their experience of university-related problems (RQ2). The central box of Figure 5 shows that quantitative analysis of changes in students’ basic needs, social, personal, and study skills, as well as the number of problems they reported, revealed no significant differences, except for decreased competence frustration and increased confidence in their abilities. However, the small sample size may have limited the ability to detect significant results. Additionally, due to the app’s design—text-based content that explains how skills work and how to develop them—it may primarily lead to foundational learning outcomes, such as remembering and understanding, rather than the behavioral changes emphasized in this study. Measuring various levels of learning outcomes—from knowing and understanding to applying and doing—can provide a deeper understanding of the value and limitations of mobile learning (Krathwohl, 2002). Additionally, considering non-learning outcomes, such as joy, connection with others, and creating socially valuable impacts through projects, could enrich the evaluation of educational systems. After all, education is not solely about learning; it is an integral part of life (Reigeluth and An, 2020).

The right box of Figure 5 summarizes the key findings for the element “tool,” focusing on Ace Yourself features that supported or hindered learning (RQ3). Most students found the app effective and helpful for learning, primarily due to its structured and clear content. However, the most frequently cited barrier to learning was a lack of time. This suggests that it’s important to consider students time capacity when designing the mobile app’s by streamlining the content or finding new ways to fit to students’ schedules—for example, by integrating the app into their main curriculum.

The bottom three boxes of Figure 5 describe the final elements of the activity system—roles and division of labor, context, and activity network (RQ4–6). Most students perceived their role within Ace Yourself as either self-directed learners, focused on independent study, or research participants, focused on testing the app. Adding self-regulated learning (SRL) support features to the mobile app could enhance its effectiveness (Baars et al., 2022; Lobos et al., 2021). Furthermore, solo learning often lacks opportunities for feedback, discussions, or satisfying the need for connectedness. Including instructions on how students can self-organize peer learning teams and study collaboratively could further improve their learning experience and outcomes in mobile learning (Tlili et al., 2024).

The app was well-suited for studying, as neither the context nor other activities posed significant obstacles to learning. However, most students preferred to study at home in the evening, raising questions about the advantages of using a mobile application compared to other media, such as online courses or books. Some students may live in environments that are oversaturated with educational possibilities and demands. Future research could investigate how different educational tools can help students free up time or better align with their routines, opening new contexts for learning. Being mobile does not automatically equate to being flexible or well-suited to students’ needs and routines.

As an activity system is more than just the “sum of its elements,” it is crucial to examine the relationships between different elements, as this can reveal contradictions and insights that might remain hidden when elements are analyzed separately (RQ6). The yellow note №1 in Figure 5 highlights a contradiction between students valuing skills suggested by Ace Yourself and the problems they identify. For instance, “Confidence in your abilities” was among the most valued skills but was not initially mentioned as a problem by students, possibly being part of a broader, less defined “motivation” issue cited by 7–8% of students. After using the app, 2.91% explicitly mentioned “confidence in your abilities” as a problem. Similarly, “Study strategies” was valued but rarely identified as a problem (<3%), while higher-level issues like self-management (7%) and metacognitive knowledge use (6%) were more frequently cited.

This contradiction—students valuing skills but not articulating related problems—prompted further investigation. The yellow note №2 in Figure 5 shows the personal group, focused on “confidence in yourself,” had similar vague problem descriptions pre- and post-survey (6 and 7, respectively) but fewer vague descriptions than the control group post-survey (7 vs. 10), despite similar initial numbers. The specificity of problem descriptions may reflect what students learned from the app, aligning with Vygotsky’s theory of scientific and everyday concepts (Vygotsky, 1962; Eun, 2019). Students may lack the scientific concepts needed to analyze and articulate motivational problems or fail to connect these concepts to their experiences, leaving them abstract. Mobile apps that lack the ability to target higher-order learning outcomes, such as behavioral change, could instead focus on helping students identify new, previously unrecognized details of their experiences through the application of scientific concepts. Future research could focus on identifying strategies that help students connect scientific concepts to their everyday experiences (Romashchuk, 2023).

The yellow note №3 in Figure 5 highlights that the most frequently mentioned problems in students’ needs analysis are not addressed in the current version of the app. This result may explain the lack of measurable learning outcomes, as students likely prioritize addressing urgent problems not covered by the app, leaving limited opportunities to practice its provided skills. However, the planning and predicting skills—despite being included in the app and identified as key challenges by students—also showed no measurable improvement, indicating a need to revise and enhance these modules. Considering whether the intended learning outcomes of a mobile app align with the actual problems and needs of learners could be beneficial: If the app targets potential or future challenges, think about how students will practice these skills if they are not currently encountering these problems. Future research could explore how to complement mobile learning with simulations that enable students to practice solving problems they have not yet encountered yet or rarely face.

Nevertheless, students valued all the skills included in the app, praised its content quality and structure, and had a positive overall impression. This suggests that achieving complex learning outcomes may not always be essential for maintaining motivation; a perceived value of content for addressing potential problems might suffice to sustain engagement and encourage continued learning. Future research could focus on identifying the “border” where skills are still considered valuable even though they are not immediately used, and when potentially beneficial future skills loose their value.

The yellow note №4 in Figure 5 highlights that the lack of implementation and practice support was both a reason for students devaluing certain skills and a disadvantage of the app overall. This underscores the interplay between the educational form or tool and the skill it aims to develop. Different educational formats—such as a mobile app, lecture, or workshop—can target the same skill, and the same format can support the development of various skills. However, students may evaluate the effectiveness of an educational form based on their perception of the skill it addresses, and vice versa. To maintain students’ motivation to learn, it is crucial to distinguish these two aspects in their perception. For example, students could be informed about alternative ways to develop the same skills they practiced using the mobile app. The app can serve as an entry point, guiding students toward other educational formats such as books, workshops, or comprehensive training programs, creating a smooth network of educational systems. Future research could explore ways to help students distinguish between the skill or intended learning outcome and the educational format used to achieve it. Do not judge the skill based on the educational format, and vice versa.

The yellow note №5 in Figure 5 suggests that although students reported no issues with interfering activities or unsuitable contexts, this does not imply the app is adaptable to all potential learning environments. Based on students’ feedback about their preferred times and places for learning, the app’s design seems to predominantly support the “home in the evening” learning context, potentially overlooking other opportunities for engagement. For instance, incorporating just-in-time or procedural information to assist students during moments of difficulty while studying could expand the contexts in which the app is used (Van Merriënboer et al., 2024). This adjustment could enable learning in real-time, addressing problems as they arise.

This study has several limitations that should be acknowledged. First, the small sample size limits the generalizability of our findings and reduces the statistical power to detect subtle effects. While the results provide valuable insights, a larger sample would allow for more robust conclusions and greater confidence in the observed patterns. Second, the absence of behavioral measures, such as detailed data analytics from the app, represents a significant limitation. These measures could have offered objective insights into participants’ engagement and usage patterns, which would complement the self-reported data and provide a more comprehensive understanding of the intervention’s impact.

This paper evaluated the effectiveness of the Ace Yourself app, designed to enhance university students’ personal, study, and social skills, through the lens of Activity Theory. While students appreciated the app, several drawbacks were identified. The study emphasizes the importance of viewing education as a holistic system—encompassing needs, objects, meanings, outcomes, tools, context, division of labor, and related activities. It highlights the value of not only focusing on individual elements but also exploring the insights and contradictions that arise from their interconnections.