Participants were recruited from a convenience sample of members of staff at a London university, their family, and friends, as well as subscribers to an alcohol reduction mailing list. Inclusion criteria were people interested in reducing their alcohol consumption and who had an Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) score greater than 5, which reflects potentially harmful levels of drinking (60). A purposeful sampling approach was taken in order to ensure the views of disadvantaged groups were gathered; half of the participants in both studies had no post-16 educational qualifications, were unemployed, or were employed in a routine/manual occupation. Participants were given £20 in compensation for their time.

Of the 12 participants in the “think aloud” study, 50% were female and 50% were from disadvantaged groups. Their mean age was 42 years, and the mean interview length was 59 min.

On opening the app for the first time, users were presented with the 10-item AUDIT, on completion of which the user’s AUDIT score and brief information about what the score indicated were provided. Users were then asked to complete baseline demographic measures, after which registration was complete.

Following registration, users were asked to indicate how they thought their drinking compared to (1) other people in the UK and (2) other people of their age and gender, using a dial mechanism (Figure 1A). Users were then given feedback which showed how their drinking actually compared to people in the UK and people of their own age and gender, using the same dial mechanism and other graphical representations (Figures 1B,C).

Participants were able to self-monitor their consumption of alcohol and the consequences of consumption. To monitor alcohol consumption, participants tapped a large plus sign in the middle of the navigation bar at the bottom of each screen, choose from one of six drink types, and then choose various options for the selected drink (Figures 2A,B). To self-monitor the consequences of consumption, users recorded a score for mood, productivity, clarity, and sleep quality each morning on a slider (Figure 2C). Users were prompted to record their consumption and their mood scores each day by way of an onscreen alert and message within the app (Figure 3).

Several forms of feedback were provided. The total amount of alcohol (in units), calories consumed from alcohol, and spend on alcohol was displayed in graphs on the dashboard (Figure 4A). The dashboard also displayed summary feedback about progress against goals and provided links to three types of other goal-related feedback: (1) whether the previous week’s goal had been achieved or missed, (2) progress against the goal for each completed week since the app had been downloaded, and (3) a summary of how many times each goal had been achieved or missed (Figure 5). The calendar provided an overview of a user’s recorded drinks (Figure 4B), with each day underlined according to whether a user had drank (colored orange), not drank (colored green), or not made an entry for that day (colored gray). Users could tap any day to see details of drinks entered; these records could be edited, added to, or deleted.

Feedback about the consequences of consumption was presented on the “Your hangover and you” screen, which contained four graphs comparing a participant’s mood, productivity, clarity, and sleep quality on days after drinking with days after not drinking (Figure 4C).

Action planning was presented within a “Create and View Action Plans” section. At the top of the screen was information about the benefits of setting an action plan and an example of one (Figure 6A). The term “Action plan” was used in place of the more accurate, but potentially less well-understood, “implementation intentions.” The Create an Action Plan screen asked users to fill in two fields corresponding to the If and Then components of an implementation intention (Figure 6B). Other screens displayed the action plans users had already set and provided further information about, and examples of, action plans (Figure 6C).

The cognitive bias re-training game presented users with either an image of an alcoholic drink or an image of a non-alcoholic one. Users were instructed to use their finger to push pictures in portrait form (“tall”) away from them and to pull pictures in landscape form (“wide”) toward them (Figure 7A). The total score for each game was the number of images correctly pulled or pushed in a 60-s period. Other screens provided instructions about the game and displayed a graph of previous scores over time (Figure 7B).

The Identity Change section contained three elements: (1) flipsides of drinking, which showed images and text representing a positive or benefit of drinking with a negative or cost of drinking (Figure 8A); (2) memos, which allowed users to record video messages to watch at a later date, for example, they could record a message when sober to remind themselves of their goal during a night of drinking (Figure 8B); and (3) “I am…,” which allowed users to select personal values of importance to them, such as being honest or responsible, and then reflect on how these values might be affected by alcohol consumption (Figure 8C).

Users were able to set an overarching reason for drinking less and were presented with links to set new goals and view existing ones. They were also given information about how to set good goals. Users could set goals for any combination of the number of units and/or alcohol-free days they wanted to have each week or month, the maximum number of calories, and/or the maximum amount of money they wanted to spend on alcohol each week or month (Figure 9).

Participants using the app for the first time frequently expressed confusion about how to use the app and how to navigate through it. Confusion was most pronounced when participants first started using the app after completing the registration process.

Registration is an expected, familiar, and uncomplicated process that participants worked through sequentially. When complete, participants were automatically taken to their dashboard, a screen that contained an empty graph and a number of links to other modules in the app. The abrupt appearance of this screen, its lack of visual concordance with the screens that preceded it, and the number of links available confused the participants, who were unsure if registration had finished and which link they should start with. This created a poor first impression, with almost all participants expressing a desire for a stepped guide to walk them through their initial use of the app.

Participants often felt disorientated within the app and were unsure how to navigate through it. They were not comfortable exploring the app and clicking links at will, often because they thought there were things they should be doing to set the app up but weren’t clear what these things were. When unsure where to go next, participants tried to retrace their steps and became frustrated when there was no easy or obvious way for them to do so. In the absence of guidance, some participants worked logically through the app, moving left-to-right through the horizontal tab bar at the bottom of the app and top to bottom on the screen. If the order of items did not make sense, if links took participants to an unexpected place, or when the navigation was inconsistent (on some screens the horizontal tab bar was hidden) participants felt confused and annoyed.

Participants wanted an app whose use required minimal effort. Some said they may be willing to invest more time than they would with other apps because this app was designed to help them. Others said they would stop using the app if it was too difficult, despite believing that their drinking was an issue they needed to address. Participants had formed expectations about how elements of the app should work based on their experience of using other apps and were disappointed when the app failed to meet these expectations (for example, users expected a calendar to appear when a date was tapped). Elements that were straightforward and intuitive, such as adding drinks, were praised.

Guidance was sought when using many other areas of the app, particularly when using modules that required input but came without instruction, for example, setting goals, adding drinks, creating action plans, or using the identity section. Participants often hesitated before entering information, partly because they were unsure what was required of them, partly because they felt the accuracy of their entries was important and did not know if mistakes could be corrected, and partly because they wanted more help from the app about what entries were appropriate (for example, some participants wanted to know whether the goals they had set were realistic). Participants were frustrated when the app prevented them from completing tasks, such as saving an action plan, without clear indication about what they were doing wrong. Instructions provided on how to play the game were thought overly complex and difficult to follow.

The visual appearance of the app played an important role in its perceived ease of use. Visually unattractive screens were off-putting to participants, who often expressed a desire for more graphic ways of presenting information. Participants found icons more pleasing and more memorable than text links and requested they be used more frequently. Some of the graphs did not make unintuitive sense at first and participants suggested better ways to be found of displaying these data. Screens that were clean and simple were praised and held in contrast to those that were busy and esthetically dull. Many participants appreciated the consistent design of the app, but the green color used throughout was not universally liked.

The range of modules available was overwhelming for some participants who wanted a leaner and more condensed app. Screens full of text, or text that appeared complex to read and understand, were off-putting to participants who wanted to keep their reading to a minimum.

When a minority of participants did not understand what was asked of them, or did not know how to use the technology, their tendency was to blame themselves, and their shortcomings rather than the app for its poor design.

Participants thought the app potentially useful overall but did not understand the benefit of using some of the individual modules, especially the cognitive bias re-training game and the identity change sections, where the relationship between use of the module and reducing alcohol consumption was unclear. The effectiveness of the game was particularly doubted; many participants were unsure of its purpose, or thought it simplistic and unlikely to work. Participants were unlikely to use modules they could see no obvious benefit to and expressed a desire for more information about why a module had been included and how it was theorized to work.

Participants were often unsure if they had successfully completed a task and expressed a desire for visual or audible confirmation at the point of task completion, for example, when a goal had been set. Participants often requested more positive reinforcement from the app and were appreciative when it congratulated them for actions. The sound that was played when participants recorded a drink was particularly appreciated as it was felt to be a reward for their achievements and helped establish a positive relationship with the app.

Some participants felt the app delivered information in a straightforward and non-judgmental tone. Others took the opposite view and considered the information to be judgmental or preaching, a tone they strongly disliked and which made use of the app feel dissatisfying. The feeling of being judged was often expressed when participants received feedback about their levels of drinking which contrasted with their perception of their consumption, for example, when they received their AUDIT score or were given normative feedback (where participants were shown how their drinking compares to other people in the UK). Participants who greatly underestimated their levels of drinking compared to others found the comparison shocking and thought the app was placing them in a category of drinkers to which they felt they did not belong. It was notable that once participants saw one module of the app as judgmental, they tended to see other modules as judgmental too.

Participants disliked when text was perceived as overly formal or impersonal. They wanted the app to have a friendly, humorous, tongue-in-cheek, and light-hearted tone of voice, despite the serious nature of the subject. A too formal tone was perceived as judgmental and off-putting. Participants appreciated parts of the app that were more light-hearted and said it helped them feel relaxed and made them want to engage more with the app.

The credibility of the app and the information delivered by it was an issue for a number of participants, particularly those who felt their normative feedback had judged them harshly and who then expressed a distrust of data about other people’s drinking. Participants found that credibility was established by use of the UCL logo on the first screen they saw after installing the app and by referencing of studies within the app. A number of participants said that the academic nature of the app and the fact that their data would be part of a study increased the trustworthiness of the information and their positive views toward the app.

As with the first study, participants wanted an app that engaged them and provided clear reasons to continue using it. However, this study revealed that for many participants, the engaging elements were either missing or not apparent. Participants felt dissuaded from using the app when it adopted a judgmental tone of voice and wanted to know that the time and emotional investment they were making would be worthwhile.

As with the first study, participants wanted a visually appealing app that made minimal demands on them and provided guidance about how to use the modules.

The third theme identified, that of “feeling lost and unsure what to do next,” was identified only among participants in the first study. Participants in the second study reported being able to navigate through the app without great difficulty, perhaps because repeated use resolved their initial confusion. However, users tend not to use new apps repeatedly; more than half of the apps downloaded are used less than five times (100). Therefore, it is not safe to assume that users will resolve issues of initial use without help. The commercial world addresses these problems with a process known as onboarding (101). Onboarding helps users become familiar with a technology and learn how its use might benefit them. It often takes the form of messages that guide users through the various elements on a screen or a stepped guide that walks users through the process of first using the app. Almost all participants in the first study requested a stepped guide be provided to help them first use the app, and many asked that guidance be provided about using elements throughout.

Two subthemes were identified only in the “think aloud” study: “Do not overwhelm me” and “Blame myself, not the app, if it’s too hard to use.” Participants in the “think aloud” study expressed concern that the range of options in the app might present an overwhelming amount of choice, a concern that corresponds with the theory that an excess of choice can inhibit action (102). Given that this subtheme was not identified among experienced users in the interview study, it is possible that people managed issues of overwhelm by using only the modules they found useful, a strategy some participants in the “think aloud” study had indeed proposed adopting. The presence of the “Blame myself, not the app, if it’s too hard to use” theme in only the “think aloud” study may also be explained by the ability of experienced users to solve problems with use. Alternatively, it is possible that the reason that both subthemes were not found in the interview study is because users who experienced these issues had stopped using the app and so did not respond to invitations to participate.

The subtheme “Update me on how I am doing” could only have been identified in the interview study with experienced users (and not the “think aloud” study of first-time users) because feedback about progress requires repeated use of the app. It is unclear why the subtheme “Be friendly and funny” was identified only in the “think aloud” study. However, one person in the interview study commented that the friendliness of the app increased with use, and evidence suggests that use of a system can increase user satisfaction with that system (103).