One of the problems of usability testing websites is trying to get testers to behave as they would in the real world. It is a very ‘unnatural’ process. We recruit people into the lab, pay them money and ask lots of questions about the site and why they have done what they have done. Testers focus on the task they have been given, think about what they are doing, try and give rational explanations for their behaviour, and are not distracted by the kids or their phone. This is not what happens in real life where users click through most of the time not thinking very hard about what they are doing. To use Daniel Kahneman terms, in the lab users do more ‘slow’ thinking whereas in real life they are doing a lot of ‘fast’ thinking.
I was reflecting on this recently while observing some testing on the site of a road rescue organisation. The testers were being asked to look at packages, choose one suitable for them, and then go and buy it. They did this without difficulty. With a relatively small number of key strokes they identified a suitable package and whizzed through the checkout process. So no problem with the site then? Well - no. While they could choose a suitable package and check out, a key question was - would they? For most of the testers the products they were being offered were considerably more expensive than their current ones. They did not volunteer that they would not buy the product because it was too expensive, or attempt to investigate what they were getting that might justify the extra cost. They had been tasked to select and buy a road rescue product so that is what they did – the lab setting was preventing them from saying they wouldn’t do this in practice. Without very careful probing about the product, the price and whether they had enough information to make a buying decision it was very easy to believe that there was no problem with the web site. In fact, key information about the proposition - why it was good and why it justified the price - was missing from the user journey.
So usability testing will tell you ‘can they use it’ – but it often requires considerable research expertise to establish ‘will they use it’.
