Planning a Usability Study

A usability study is a method by which the design of a flow, or interaction or even an entire feature can be validated by observing users use a prototype or built out product, and then identifying what works and what doesn’t.

Remote Usability study setup

A good usability study requires planning, and with practice you’ll be deploying and executing them in no time at all.

Each of these steps may be done as exhaustively as you feel comfortable with

Step 1: Identify assumptions that need validation
Here, find out about the product, or feature, and what assumptions have been made in it’s design. It is often a good idea to bucket things into

  • “I know this is true for sure.”
  • “I sort-of know this is true.”
  • “I have no idea if this is true or not.”

Step 2: Map user journeys to identify points of friction
In this phase, I map out the user journey(s) that have to be tested. This is done by using the app or product and exhaustively selecting every single option available. Once we have the journey mapped out, we can identify major points of friction that will likely cause user drop off. These can be tested for validity, or fixed outright if the solution is obvious enough.

Step 3: Understand existing information about the product / feature

Inquiry about users, target market, existing feedback and problem points will guide what insights you need from your usability study. In this phase, I typically collect information from various customer touch points about the app and. This includes but is not limited to customer support, social media channels, product managers, account managers, technical support, and project stakeholders.

Another thing to consider is, who are competitors in the market? What do they offer that we don’t? What do we offer that they don’t? Where is the potential for maximum success as defined in phase 1?

Step 4: Make a list of things that need to be tested or validated.

In this phase, we ask ourselves, what information do we want to find out?

- friction points
- user pain points
- what works for users

How many journeys are we going to test?

If we do not go through with this step properly, it will be like going shopping a grocery store with no idea what dish needs to be prepared for dinner. The final output, as good as it may be, was ultimately only a result of the ingredients we picked in the store without plan, and does not address the basic question about “what do we want to cook?”

We also recruit and schedule sessions with anywhere between 5 to 10 users for each major user journey during this phase.

Step 5: Formulate task based study

The formulation of the usability study involves the following:

  • prepare scripts
  • set up devices / equipment for the study
  • recruit and schedule users to participate
  • arrange for incentives for participants
  • prepare redundancies of the setup if possible
  • conduct a dry run of the session at least once before doing it with users.

Step 6: Conduct study, Record findings

The script is used, and study is conducted with users. Releases and waivers and NDAs are signed. Special attention is paid to metrics to be tracked in each section. These are indicated on the script in appropriate sections. The sessions are recorded using screen capture or audio/video equipment or eye tracking technology as required. A moderator should ideally be present to take notes while the session goes on so that I can focus on conducting the session.

Step 7: Synthesis of findings

The recordings and notes are analyzed. I look for common patterns in user behavior, pain points and make note of suggestions. I typically use a basic affinity mapping and card sorting exercise in this stage. These findings are then aggregated and compiled together in a report along with key actionable insights.

Step 8: Recommended plan of action

This is critical. Usability studies are useless unless accompanied by recommendations (or direction) about how to use the insights gathered from the study. Based on the report in Step 7, I will recommend a plan of action to address the flaws in the product that have been revealed by the usability study.

Every step of this process is worth deep diving into for best practices. I hope this post gives you a good start for planning your own usability studies. Have you conducted usability studies? How did those work out? What do you differently? I’d love to hear from you.