According to previous studies (Lizzio, 2006), the transition to tertiary education should be supported from different aspects (or “senses”) of an individual’s life, for example, learning, economy, social, career development, etc. However, supports from the above listed aspects are difficult to provide and to study their effect. A mobile application offers the ideal platform as it is available 24/7, highly personalizable, and can be deployed at scale. Therefore, a mobile application, myUniMate, was developed as the major support-delivery medium. A four-phase UCD design protocol was applied in the design and development process of the myUniMate app.

The main objective of this phase was to gather requirements for the initial design of the myUniMate mobile app and understand the needs of first year university students. Methods used in this phase were as follows: literature review, questionnaires, group discussion, and existing knowledge in the project team.

Requirements gathered in Phase 1 was further analyzed and refined by the project team and a set of requirements were formalized as feature specifications. Low-fidelity prototypes were implemented based on the feature specifications to enable the evaluation of current designs in a tangible manner.

Low-fidelity prototypes were demonstrated to and circulated around first year students, teaching staff, and project team members. Discussions of the current design were held and comments and suggestions on the design were gathered. After Phase 3, the iteration of the design and evaluation process starts, until all project team members considered the current design was deployable.

Once the design was considered deployable, a final working prototype was implemented and deployed to the field trial.

First year students, teaching staff, and project team members, such as researchers and developers involved in this project, took part in the UCD design process. There were two main sources for the initial requirements of myUniMate: (a) comments and suggestions derived from our previous study; (b) requirements proposed by all three participating parties. After the first phase, the requirements were gradually refined, on which low-fidelity prototypes (wireframes of different components of myUniMate) were implemented (Phase 2) and demonstrated to and circulated around all three parties of the project team (Phase 3). Based on the comments and suggestions gathered in Phase 3, low-fidelity prototypes were redesigned and re-evaluated. Several iterations of design and evaluation was conducted, and a working prototype was implemented at last.

The “5 Senses Model” was adopted as a guideline to initiate the design of myUniMate to cover possible aspects of students’ lives and to evaluate the final design. Especially, myUniMate was designed with the aim of directly covering the Capability, Purpose, and Academic senses of the “5 Senses Model,” indirectly providing the Connectedness sense and the Resourcefulness sense. The rationale for this design decision is that, in the context of education, a sense of connectedness is best provided to an individual through close interactions with peers, family, and university staff (Lotkowski et al., 2004). On the one hand, students are already using mature commercial chatting platforms as their daily virtual communication media (Woodley and Meredith, 2012) which makes it unnecessary to include such functionality into myUniMate. On the other hand, although a discussion board or a chat application can bring students together, integrating such a functionality into myUniMate was considered beyond the scope of this project to avoid the inclusion of a new online communication platform for both the developers and the participating students. In terms of resourcefulness, we argue this can be classified as a separate research topic. Almost all information useful for a first year university student is made available through the university website or course-specific websites, but as the students are new to the websites they may have trouble locating the exact information that they are looking for. One of the solution to this issue can be a redesign of the website to display information in a more comprehensible and easy-to-search manner, and thus can be conducted in a separate interface and interaction design research. Furthermore, experience-based information is usually derived from students’ personal past experiences or through interaction with other students, which is then within the scope of Connectedness. Therefore, we focus on three of the senses from the “5 Senses Model.” myUniMate contains six components (ToDos, Calendar, Measures, Notifications, Mood, and Feedback) which form into four major functions: Task Tracking, Visualization, Mood Tracking, and Notification. It follows a detailed description of the functions.

Table 1 demonstrates the design of the field trial. Study participation included two questionnaire sessions and at least 28 days (4 weeks) of using the myUniMate mobile application. The questionnaire sessions were conducted in the week before and the week after the 4-week app-usage period. During the first questionnaire session, all participants were asked to complete a set of questionnaires and during the second questionnaire session, participants were asked to complete the same set of questionnaires. After the first questionnaire session, participants were instructed to install myUniMate on their personal mobile phone and enter her/his own tasks into the app, which means a participant was expected to know what she/he needed to do (Purpose). During the following 4 weeks, participants were asked to use the mobile application in the ways that they prefer. The second questionnaire session took place after the app-usage weeks.

This study was approved by the University of Sydney Human Ethics Research Committee (protocol 2015/473). The recruitment was conducted in the first lecture of a first year engineering course in an Australian university in the year 2016. The course lecturer introduced the project to the students and provided contact details to allow students to opt in the study. 11 participants (3 females/8 males) took part in the trial. All participants gave written informed consent before the study. At the time of the study, all participants were aged between 19 and 21 and were doing their first or second semester study during their first year of university. All participants were regular mobile phone users with a mobile phone running Android or iOS system. A gift card worth of 20 AUD was given to each participant to compensate her/his time.

The Five Sense Model given by Lizzio was used as a guideline to evaluate the design of myUniMate. The target of the model was to enhance first year university students’ satisfaction, engagement, and persistence. In order to quantitatively measure students’ satisfaction, engagement, and persistence in learning, five questionnaires were used in this study: Background Information Questionnaire (BIQ), Academic Motivation Scale (AMS) (Vallerand et al., 1992), Basic Needs Satisfaction in General (BNSG) (Deci and Ryan, 2000; Gagné, 2003), Learning Climate Questionnaire (LCQ) (Bartram et al., 1993), and Revised Study Process Questionnaire (R-SPQ-2F) (Biggs et al., 2001). The first questionnaire was created by the researchers involved in this study, while the other four questionnaires were standard questionnaires that had been evaluated formally. BPNS, LCQ, and R-SPQ-2F are all instruments revolving around the SDT.

The five questionnaires were used in the field trial to evaluate the effectiveness of myUniMate. When applying the AMS, BNSG, and R-SPQ-2F questionnaires, students were instructed to use the engineering course that they were recruited from as context with an emphasis on the use of myUniMate. The reason for this request is that originally, these three questionnaires were designed to gage students’ learning motivation, basic needs satisfaction, and learning approaches employed in a certain context. The context is normally a course taught by a lecturer and the teaching methods the lecturer uses. However, in our case, we would like to know students’ thoughts on the mobile app rather than the overall learning environment. Therefore, to have a more accurate measure of the influences of myUniMate on the students we asked the participants to take the mobile app into consideration when answering questions during the questionnaire sessions. On the other hand, LCQ was used to measure students’ self-perceived autonomy support from the overall learning environment as a reference.

The data used in the analysis are from two data sources: (a) app interaction data (tracked within myUniMate); (b) questionnaire answers from the five questionnaires (details can be find in Section “Questionnaires”). Both quantitative and qualitative mechanisms were used to analyze the data.

For the app interaction data, we used quantitative method to calculate the means and SDs of different types of interactions performed by the participants. The tracked interactions include: screen view counts, screen navigations, button clicks, and data entry. We also separate data entry actions from pure “click-throughs” as sometimes clicking through different screens can be a random action. The result of this analysis is an overview of participants’ preferences over different functions provided by the app.

The BIQ required text-based answers, and we used affinity analysis to analyze answers that are related to students’ university experiences, learning experience, and comments on the myUniMate mobile app. Different themes were extracted from answers to the same question given by different participants, which means for each question, we discovered at least one theme of all the answers. This process was conducted by three researchers to enhance reliability and disagreements were resolved through discussion. The other four questionnaires require value-based answers and, thus, only quantitative methods were used. We conducted paired-sample t-test using SPSS (SPSS Inc. Released 2013. IBM SPSS Statistics for Macintosh, Version 22.0. Armonk, NY, USA: IBM Corp.) to gage the changes between pre- and post- questionnaire answers and the Cohen’s d value was calculated to analyze the effect size.

To assess participants’ engagement with myUniMate, we analyzed the usage logs and post-study questionnaire answers. Three measures are used as indicators of participants’ engagement: the number of time a screen is accessed by a participant, the number of time data is entered on a screen, and app-usage duration.

Not surprisingly, the engagement peaked at the beginning of the 4-week app-usage period and plateaued afterward, which can be explained by the novelty of myUniMate. As shown in Table 2, on average, myUniMate was accessed 6.55 times per day per participant, and 3.09 entries were made daily. This result is much better than the study result of myUniMate (v1), which suggests that the new version has a significant improvement over the previous version.

The “App Usage Duration” is only used as a proxy for measuring the length of user interaction with myUniMate, as actual interaction time is very hard to be measured.

In Table 3, we break down the measures shown in Table 2 into different screens. Components with higher number of access rate have been highlighted in bold and italic font. The ToDo component was the most used during the trial, which suggests that this component was perceived more useful by the participants. This is consistent with answers in the post-study questionnaire. In the post-study questionnaire, we asked what feature of myUniMate participants liked best. Nine participants (out of 11) mentioned that the ToDo component was (one of) the most useful. P-3 mentioned, “…, it was really helpful as it usually reminded me of tasks that I hadn’t done.” No negative comments were given to the ToDo component.

Apart from the ToDo component, the Measure component and the Mood component were also used more than the others. However, mixed feelings on these two components were expressed in the post-study questionnaire answers. For the Measure component, P-11 mentioned that “At first I didn’t know what the Measure function was for, but gradually I discovered that it allows me to enter any type of information that I want to record. The design is fine, [but] it is just that I’m not familiar with tools that allows me do this.” Similarly, for the component Mood, P-1 mentioned, “…, that’s cool to trace back and see how I’ve been so far.” However, P-6 wrote “…, I don’t know what ‘mood’ is really for.”

Basic Needs Satisfaction in General and LCQ measure students’ self-perceived needs satisfaction and autonomy support, respectively. These two aspects, needs satisfaction and autonomy support, are considered related in SDT (Deci and Ryan, 1985, 2000; Deci et al., 1999; Ryan and Deci, 2000). Specifically, normally when the innate needs of an individual are satisfied, she/he tend to feel more autonomy supported. We, therefore, use both questionnaires together to see how myUniMate was perceived by participants in terms of need satisfaction and autonomy support.

The “Academic Motivation Scale” was used to assess participants’ motivation toward learning. A summary of the comparison between the pre-test and post-test data is given in Table 6 below.

From the beginning of the study, the participants already have a positive academic motivation, as the mean score of the first six motivation categories are all above 3.5 (half of full mark, 7) and the AM category received a score of 1.48 which was very low. Specifically, the IM categories (To know, Toward accomplish, and To experience stimulation) and the EM categories (Identified, Introjected, and External regulation) were both positive. The IM “to experience stimulation” was relatively low, which was 3.91, compared to the other two categories, 5.25 and 5.00. Similarly, the introjected external motivation was low compare to the other two external motivation categories, Identified and External regulation, being 3.77 out of 7.

After the app-usage period, all three IM categories had a statistically significant increase. IM “to know” increased from 5.25 to 5.57 [t(10) = −2.43, p = 0.04] with a Cohen’s effect size value of d = 0.73. This suggests a moderate to high practical significance. IM “toward accomplishment” increased from 5.00 to 5.77 [t(10) = −4.09, p = 0.02]. The Cohen’s effect size value (d = 1.23) suggests a high practical significance. Similarly, IM “to experience stimulation” increased from 3.91 to 4.91 [t(10) = −5.16, p < 0.001] and its Cohen’s effect size value (d = 1.56) suggests a high practical significance.

From the qualitative feedback given by the participants, we also observed a strong motivation in learning. Some participants expressed external drivers of their motivation, such as jobs and salary. For example, P-10 commented that “I’ve always wanted to know wether [whether] my bachelor/major is easy to find a job or not” and p-12 commented that “[I would like to know] how much I can earn after graduate… [and] what is the salary trend for my profession.” The motivations expressed by the other participants were more intrinsic with a focus on learning. For example, P-4 commented that “[I would like to know] how our courses go in the future” and P-5 commented that “[I would like to know] which unit of study and which lecture of that unit is good.”

As can be seen from Table 7, no statistically significant differences were observed in terms of learning approaches. However, participants expressed a preference in deep approaches in terms of learning in their comments. When asked how the participants were planned to study, P-9 gave the answer “follow the instructions, do what I am asked to do, and learn new things while practicing.” This answer shows the initiative that the participant would like to take, which include a feedback loop. This way of learning is of the deep learning approaches. Similarly, P-4 answered “accumulate knowledge through practicing, staring from familiarizing with contents and work on exercises with gradually increasing difficulties. I’m still trying to find the best way but this might work for me just now.,” which is a more specific learning plan. These comments echo well with the quantitative analysis on the R-SPQ-2F data.

In the education domain, being autonomous has long been considered as one of the properties of a quality learner (Dickinson, 1995), and specifically, an autonomous learner tends to have a higher motivation and will have better and more effective work. The support of autonomy is normally provided by the course lecturers and the teaching methodologies that they use, for example, the flipped-classroom approach (King, 1993) supports students’ autonomy as its various implementations require students to prepare themselves with materials before lectures take place and instead of one lecturer dominating the classroom, students are normally encouraged to ask questions and start discussions during a flipped-classroom lecture. On the other hand, autonomy support that is provided by software is hard to design, as different design approaches would affect one’s motivation differently, and even a subtle wording difference on the software could result in different perceptions of the software itself. What is worse, the differences in design are impossible to be quantified. The only thing we can do is trial-and-error.

When trialing the previous version of myUniMate, the researchers were in charge of the entering and deleting of entries in the “Task Tracking” component, and the participants requested more freedom in the use of that component (Zhao and Pardo, 2015). However, in the trial of the current version of myUniMate, participants were given full access to add, update, and delete entries in the “Task Tracking” component. Instead of the expected one-sided positive comments, they still want more. One example comment from P-10 is “I think our classes and each assignment should be [automatically added] to the calendar, it will be more clear [clearer] for students to check their time.” The most obvious suggestion of this comment is that the students need something in the middle of fully controlled and fully autonomous functions on the app.

Based on the results of BNSG and LCQ, we concluded that myUniMate had a negative influence on the participants’ self-perceived autonomy support, which is an interesting finding. As no comments directly point us to the reason of this result, we consider the most probable cause of this result to be the use of notifications to deliver reminders and suggestions to the students. During the first questionnaire session, we purposefully instructed the participants to use myUniMate in ways that they prefer, which means they could stop using it whenever they wanted to. Therefore, there is a good chance that the negative influence was caused by the notification which the participants could not stop. Further study is needed to have a clear answer to this question.

As explained in Section “The Design of the myUniMate App,” the Connectedness sense and the Resourcefulness sense of the “5 Senses Model” were indirectly supported in the app. Based on the responses of the post-study questionnaire, both senses are proved to be integral parts of transition support, which in turn emphasized that the “5 Senses Model” can also be used as a guide in designing software tools to support transition to tertiary education.

Specifically, in terms of Connectedness, different social features were proposed by participants. P-6 mentioned, “…Secondly, [the feature] might [could] be socializing features to connect the users via tagging themselves with interests. Also maybe working as a[n] announcement board for local events as well.” Similarly, P-8 mentioned that “[a] discussion board [could be implemented] so that users can help each other.” From the above-listed comments, two points should be noted:

Social-related features should not replace in-person socialization, which resonances with our design decision mentioned in Section “The Design of the myUniMate App.”

As to the Resourcefulness sense, only limited course-related information was available in myUniMate, and no instructions or guidelines for navigating the university information system were provided by the research group during the field trial. However, there is a strong need to such information from the participants, especially for the information about their degree and future development, which is consistent with our previous study. As commented by P-9, “[I would like to know], which unit of study and which lecture [course] of that unit is good.” P-10 mentioned, “I’ve always wanted to know wether [whether] my bachelor/major is easy to find a job or not.” And P-11 mentioned, “[I want to know] how much I can earn after I graduate, and how hard I need to work in order to get a role [job] that I want.” According to the SDT (Deci and Ryan, 1985, 2000; Deci et al., 1999; Ryan and Deci, 2000), the more information one has about the current context and the more one understands the current context, the most likely she/he will be autonomous, and thus more likely to have a better psychological well-being. Therefore, provision of such information is important in supporting educational transitions and should be listed as future works.

Although a Notification component was implemented in myUniMate, all notifications sent during the field trial period was common among all participants. However, notifications or feedbacks tailored to individual’s needs are always preferred. P-2 mentioned that “regularly providing personalized feedback would make more sense since we are entering a lot of information.” P-3 had a similar comment, “notifications should be personalized, otherwise [the notification component is] useless.” This echoes with the results reported in previous studies (Brinkworth et al., 2009; Crisp et al., 2009; Thalluri and King, 2009).

We consider transition to tertiary education as a special case of behavior change, which have been studied extensively in the field of HCI, Persuasive Technology, and Pervasive Computing. Therefore, mechanisms proved useful in supporting behavior change could be applicable in the supporting of transition to tertiary education. The personalized notification mentioned in the previous paragraph can then be generalized to tailored interventions which is widely used in the support of behavior change. Goal setting theory should also be considered in future versions of myUniMate.

In this study, myUniMate was implemented as a mobile application that is able to function on its own, and no other widely used software applications were integrated into myUniMate, which is one of the limitations of this study. P-5 mentioned in the post-study questionnaire that “[myUniMate should] be a good hub and leave the function implementation to the other apps rather than do everything in one.,” and a specific example was given by P-10, “[myUniMate should] import or sync data with other calendar & todo app[s].” Integration with Google Calendar and iOS Calendar was considered when designing myUniMate, but since one of our design goals was to make myUniMate self-contained with no dependency on other platforms in the field trial, we decided not to implement the integration. However, we do admit that an integration with various widely used tools is a must in the production version of the myUniMate app. As suggested by the quantitative results of R-SPQ-2F, myUniMate had no influence on participants learning approaches. This is an expected result since apart from information entered by the participants, the app did not have built-in support of learning. In order to integrate the support of learning into myUniMate, a connection with existing learning management systems should be built.

In the context of transition support, providing only one mobile application as the source of support could be another limitation, as mobile phones are not designed to be used in all conditions. The most obvious feature (or limitation) of mobile phones is their much smaller screen sizes compared to computers. When displaying charts within the Feedback component, only two charts are able to fit into the screen, however, on a computer, not only more charts can fit into the screen, but more sophisticated interactions or data manipulations can be performed.

The transition to tertiary education plays a crucial role in students’ overall success. Even though educational institutions have designed and implemented various transition support programs, most of them are underutilized. With the high adoption rate of smart phones among university students, mobile applications can be used as a media of personalized support throughout the transition. In this paper, we presented the design and implementation of myUniMate, a mobile application that aims at providing transition support to first year university students. The field trial result showed that the mobile application was able to positively alter students’ self-perceived competence and relatedness, and improve their IM of learning. However, although the overall learning context was autonomy supportive, the mobile application was shown to have a negative influence on students’ self-perceived autonomy. Possible improvements and future works were discussed.