Productivity and time management apps have been huge on smartphones in the last few years, and as a big fan of any kind of list-making or diary tool, I really enjoy playing with such apps. One of the most interesting ones out there is Lift, an app for goal-setting.
During a class I took last fall at UW, I got a chance to dive in and do some user research on it with a team, so I thought I’d do a little blog post about that.
All members of my team (Ignacio Contreras, Hong Zhou, and Koen De Couck) consider ourselves to be user researchers and not designers, so we only came at the app from that angle. My own interest in this project was to learn about the differences between user and market research.
This wasn’t an actual sponsored project and we didn’t talk to the Lift guys, so we have no idea what the internal strategy at Lift was while we were doing this study. We’d be happy if the folks at Lift decided to use any of our ideas!
This wasn’t an actual sponsored project and we didn’t talk to the Lift guys, so we have no idea what the internal strategy at Lift was while we were doing this study. We’d be happy if the folks at Lift decided to use any of our ideas!
Metrics and Lift Overview
As a team we made a compelling case for this app and sold this project to our instructor by saying our success metrics, if we were actually on assignment for Lift, would be:
- An increase in the number of satisfied users
- A reduction in the # of complaints via in-app feedback
- Increase new user acquisition
Our intention, however, was not to conduct a summative evaluation of the product. A summative evaluation is often done once a product is complete and uses success metrics. Although we pitched metrics to our instructor, our project was formative. We wanted to inform the design of the product and detect and eliminate usability problems.
We approached this task by using the app ourselves and determining the major user tasks. At the time, these were:
- Add a habit to your personalized list of habits
- Check-in after completing a habit
- Set a reminder to engage in a habit
- Check-in retrospectively (e.g., you forgot to check-in and want to do so the day after)
- Remove reminders to engage in a habit
- Add a friend
- Comment on that friend’s activity
Research phase 1: Finding pain points
Our research study consisted of two phases. The first phase included a semi-structured interview and usability testing where we provided our test participants with 7 scenarios, one for each of the major user tasks. This included having each of our 8 users attempt to complete each task while thinking out loud. The technique of an in-depth interview and having the interviewee think aloud (a method with roots in the cognitive psychology approach called Protocol Analysis) is not that different from the interviewing style market researchers use to test their surveys questions. The, perhaps obvious, difference is that here users were focused on completing tasks rather than on understanding word usage and conveyed meaning.
For the qualitative data analysis, we relied on a method frequently used in the social science: grounded theory. This means we did not approach the interviews with a hypothesis in mind. Rather, we used data from the interviews to inform our hypothesis (Strauss & Corbin 1994). Afterwards, we looked for patterns across all participants to provide direction on which tasks of the app could be redesigned and tested in phase two of our research.
In addition to these qualitative findings, our data consisted of completion rates and confidence ratings. Completion rates are binary, a participant either completed it or didn’t. For confidence ratings, we asked users after they attempted to complete a task, how confident they were that they would be able to complete the same task in the future using a 7-point Likert scale.
The results are below. The tasks with the lowest completion/confidence ratings were: checking-in retrospectively, adding a friend, and commenting on that friend’s activity.
Our qualitative findings provided insights on why some of these tasks were difficult. They showed us that navigation issues were the reason why users were not able to complete certain tasks. Lift’s menu bar was confusing and often required a lot of user actions to go back to previous screens and find the option needed.
Ultimately, we chose 6 areas to redesign:
- Home page design (no homepage previously)
- Persistent menu bar/home button
- Pre-identified list of habit categories
- Check-in frequency per habit (e.g., once a day, twice a day, etc)
- Customizable privacy options
- Retroactive check-ins
Research phase 2: Validating our solutions
We mocked-up some solutions to test with users in our second phase of research. We went back to the same participants we talked with in phase one. We showed each one the original design as well as two additional solutions, these are outlined in the table below. We didn’t want our best bad design to win so we gave participants the option to say they did not like any of the solutions we presented to them.
Phase 3: Putting it all together
We ended up with six recommendations. Here are the top three that I felt were the most strongly supported by our research findings. Our full final report can be found here.
Home Page:
Users overwhelmingly preferred a home page with large icons representing each function of the app. Seven out of eight users indicated this solution as their first choice. Users favored this navigation shortcut, which eliminated the need to click back through several screens to access app functions. Example: Previously, after adding a habit, users had to click back through two screens before landing at their list of habits.
“[I] like it….people will be familiar with [this type of design]. It’s an easier way for learning navigation.”
Home Button
Several participants felt the choice of a persistent home button or persistent home menu (with no home page) depended on what the final design of the home page looked like. They said they would prefer a home button if we chose an icon based home page. Therefore, our final recommendation included a home button.
“…depending on what home screen you use, this might be the best. I like this with the icon home page.”
Privacy
The version of Lift we tested had no privacy options. Lift did, however, have a message under settings that indicated that they might in the future. We decided to test several privacy options, one of which was very similar to the option offered by Facebook. This highly customizable option was preferred by six out of eight users. It allowed them to customize at the habit level, allowing them to have a black and a white list.
“[I like this option} because it allows you to hide from some people…by nature some habits might be habits you don’t want to share.”
“If I knew my friend was trying to form the same habit, I might want to let [her] see my check-ins and comment but I wouldn’t want everyone to know.”
Conclusion
Here’s my biggest takeaway: usability researchers focus on the user and making the product better for the user. Market researchers focus on supporting a very specific business goal such increasing market share or getting the customer to spend more. This difference in mindset/goal is a big difference between the two fields. The other differences, which I have written about previously, such as conducting research with a small group of users that aren’t representative in UX or giving users a list of tasks to complete rather than asking about what they like/dislike are still there. It is mindset and how researchers approach tasks that struck me as fundamental difference.
Endnote: Over the past few months, Lift has made many updates to their UI. It has a much more visual feel and a focus on forming habits using plans their coaches have created. They do not, however, have a home screen that includes the features of the app. The app still opens to a user’s individualized list of habits. It also does not have any privacy settings and the note that said they might offer such a setting in the future is now gone. I wonder what’s driving this direction?
Edit: Tony Stubblebine of @liftapp responded on Twitter that the new version does have per-goal privacy.
Insight Chaos 