Since Web Usability begun way back in 2002, we have conducted thousands of usability testing sessions with hundreds of clients and testers. With these years of experience, here are 4 pieces of advice we would offer anyone embarking on usability testing…
1. Set clear research objectives at the start
Clients will often come to us with a long list of issues they wish to address during research. These can be wide ranging, determined by underlying internal politics and all in all a bit woolly.
Why can it be a problem?
Starting a research project without clear objectives guarantees you will not get value from the testing. It can result in the wrong research approach being adopted, the wrong questions being asked and the wrong testers being recruited – all of which can result in inappropriate and meaningless recommendations.
What you can do about it?
Ask yourself the following three questions before starting the research:
- Why are you doing the research - what questions are you trying to answer?
- Which of the issues you are looking to address have the biggest impact on your organisation i.e. what are your priorities?
- What changes can you realistically implement?
2. Recruit the right testers
Large sample sizes aren’t necessary for usability testing (See “how many testers are enough”) but it is important to recruit the right type of people.Why can it be a problem?
Involving expert testers (those who make a living testing websites) or unsuitable testers may give you insight into basic usability issues but will not allow you to understand their attitudes to the usefulness of the content.What can you do about it?
- Agree who your target audiences are at the start of the project. These user groups will be the focus of the research
- Develop a detailed screener to identify these users. This should allow you to find people who are genuinely interested in the site and its content.
- Do not reuse testers on a regular basis as they may ‘learn’ what they want you to say
3. Set the right tasks
To get insight into the effectiveness of the site and value for money from your research project it’s important to get testers to undertake realistic exploration of your site.Why can it be a problem?
Asking testers to complete a list of predetermined tasks will not give you a user’s perspective on the effectiveness of the site i.e. what are their goals and how well does the site meet these. So while you might discover they can complete tasks on your site, they may not be the tasks they actually want to complete. As Jakob Nielson says “you are not the user. This is why it’s a disaster to guess at the users’ needs.”What can you do about it?
- Let the user lead the exploration – ask about what is important to them upfront and let them explore the site in line with this
- If testing a prototype with limited user journeys, ensure these journeys are based on thorough discovery research i.e. make sure they are realistic
- Adopt an open, non-directional approach to facilitation to avoiding leading testers where you want them to go
4. Get organisational buy-in
UX research requires an understanding of user’s behaviours in conjunction with their attitudes. These can be hard to articulate and easy to dismiss.
Why can it be a problem?
Without buy in from your organisation there is little point in doing UX research. Any actions that come from the testing will fall on deaf ears and implementing change will be an uphill battle (See "UX for change")
What you can do about it?
- Engage key stakeholders from the word go - get them involved in setting the clear research objectives
- Force these stakeholders to watch at least some of the research in realtime. Watching just one tester struggle through a website provides undeniable evidence of issues and is a powerful method to change existing mental models.
- Get them to discuss the research – every person who watches a research session can take away very different interpretation. Only by getting them to discuss the outcomes while the evidence is still fresh can you get agreement on the issues and actions
