Usability testing is all about getting insights on the functionality of a website, app or product from the people who use it. Testing involves asking users to complete tasks using the product and measuring both how successful they are and their feelings about the experience.

This process helps us to understand things from the perspective of the people we make things for! Seems like a no brainer, right?

User testing or usability testing?

The proper name is Usability Testing but you may have heard of it as user testing. User testing is a bit of a misnomer, we aren’t testing the users, what we want to test is the product!

As straightforward as it might seem, usability testing comes in many forms and there are good reasons for this; there is no single correct usability test that can catch all issues on any product or platform (if there was such a test, we’d have a machine do it and I’d be looking for a new job!).

To get the most out of usability testing, it’s important to understand why it exists, how it’s done and when best to use it.

Why is usability testing important?

Try as we might, it is near impossible to anticipate all of the ways in which our designs are going to be used by target audiences, what problems they might encounter, and what other needs they may have.

Assumptions at the best of times can only get us some of the way, and at the worst of times can hinder more than help.

It’s all a matter of perspective.

Looking at our products through the eyes of our users is a great way to ensure that our design work meets their needs.

Usability testing asks users to show us and tell us about their experiences of our products, while we observe them using the product “live”. In turn, we can gather and respond to insights, solving the problems they encounter through careful test design, thorough data analysis, and designing creative solutions.

Real insights from real users help us to understand how usable a system is and how positive or negative the experience of using a system is for the people using it. Or in official terms we are looking to evaluate both usability and more broadly, the overall user experience.

Running a usability testing session

Whatever you decide to call it, and however it is being conducted, there are few things that you will almost always find in a usability testing session.

People:

A participant who matches your target demographic.
- Remember, if you come up short-handed when recruiting, anyone who doesn’t already know the product inside and out can be a great help!
Two people conducting the user testing (normally researchers), one to run the session, and one to observe.

Activities:

A set of tasks based around key elements of the product is given to the participants to carry out.
Participants will be encouraged to explain their thoughts, feelings, expectations and responses to the product throughout the tasks.

Equipment:

Video and/or audio recording equipment.
The product and a device on which to use it.

The facilitator will explain the session, set the tasks and prompt the user to “think-aloud” while they attempt to complete the tasks.

The observer will ensure recording equipment is set up correctly and take additional notes throughout the study. Around 5 users per ‘user type’ should be enough to gain meaningful feedback on a product (though the debate rages).

After the study the data gets analysed. This involves comparing the data from each session to see what problems were encountered, who they affected, and how severely. There are a bunch of different ways to analyse usability tests.

After analysis is done the problems can be ordered by priority, the highest priority is usually a combination of the problems that were most impactful to the most users. Actionable design recommendations are created and passed on to the relevant teams who can begin to implement them.

When should usability testing be done?

Usability testing is a powerful tool, but knowing when to use it and what to use it for is critical to its success.

Broadly, it can either be used to help develop new designs or test completed ones, depending on which you are doing the process and outputs can vary.

Formative usability testing:

Tests products in alpha and beta phases
Used in iterative processes to make improvements before production
Involves testing early prototypes

Summative usability testing:

Used to evaluate a shipped or nearly shipped product
Often tests against benchmarks
Tests if the product is suitable for release

What can we learn from usability testing?

Answers to specific questions.

Specific questions will often be derived from some preexisting goals relating to what the business wants to achieve. Or they can be more exploratory and relate to understanding a problem in more detail, to then set specific goals.

These can include all sorts of questions!

How does our product compare with those of competitors?
Can people with vision impairments use our site effectively?
Why are we getting low conversion rates but high visitors?

Identify usability problems.

Any aspect of the user interface that causes the system to have reduced usability for the user. This means any aspect where a change would lead to an improvement in usability.

Usability problems include:

Anything that prevents a user from completing a task.
Anything that requires too much time or effort from the user to complete the task.
Anything that frustrates, irritates or has an otherwise negative impact on the users experience.
Anything that causes the user to take an incorrect action.

Usability problems can be disruptive, frustrating and often will drive users away when faced with difficulties that feel greater than the benefits of achieving their goal on your platform.

Gather data around specific metrics.

Metrics are very useful and give us ways of assigning numerical scores to certain aspects of a product’s performance. This is super useful when you want to compare a product with: other products, other iterations of the same product. Classic usability metrics include:

Performance metrics:

Efficiency: how long does it take users to complete tasks?
Effectiveness: what are the success rates of users performing tasks?

Experience metrics:

Self-reported: Trust, pleasure, frustration…
Physiological: Heart rate, pupil dilation…

Issue metrics:

Number of usability problems
Severity of usability problems
Types of usability problems.

What about the different types of usability testing?

Incorporating usability testing into your design process will help bring the relationship between product and user closer, but to maximise these benefits it is important to make sure you are choosing the right methods.

This includes decisions like, lab or “in-the-wild” testing, encouraging users to think-aloud or using a retrospective inquiry, whether your evaluation is formative or summative.

The methods you should be using depends on the questions that you are asking and when you’re asking them.