We acquired our first eye tracker in 2007 and what a beast it was! The size of a large, old PC monitor, heavy and consequently difficult to take out and about. However, it rapidly became an important tool for us in usability testing. Fortunately, the technology has advanced very rapidly and these days our eye tracker is a pocket-sized device you can take anywhere.
Liveviewer: how we use eyetracking
For us, eyetracking is most useful when used in real time (using ‘Liveviewer’) to enable our clients to get greater insight into how users look at page. Here’s why…
- Time and again, whilst observing usability testing sessions using the think-aloud protocol, we notice that what a person does is quite different from what they say i.e. their behaviours are not consistent with their attitudes.
- By using eye tracking there is far less doubt about which screen elements are being looked at (or not looked at). This can add significantly to the understanding of the usability issues on a website.
- This is particularly valuable when a testing session makes explicit, without explanation, a lot of issues that may be obvious to an experienced usability professional, but a revelation to many clients. For example, we know that users won't read anything but look at headings, bullets and links before they find the actual content they want, but that doesn't stop the vast majority of websites filling their navigation pages with content irrelevant to most users' goals.
- Watching the eye track as the tester uses the site shows the client all this in horrifying explicit detail!
- The red bouncing ball in Liveviewer really helps to engage observers and make them focus on the testing, when it can be very easy to get distracted on other things
- As a facilitator, we find Liveviewer very useful at stimulating probe questions in real time to understand what and why the tester is looking for
Heatmaps: how we don’t use eyetracking
The main visual output from eye tracking that most people are familiar with is a heatmap - a summary of fixations (how long testers look at page elements). We don’t tend to produce heatmaps for the following reasons:
- Heatmaps mean testers have to be given the same tasks. Although, this may work with a prototype with limited user journeys, on a live site we prefer to use a user-lead approach which more accurately reflects what they want to look for.
- To produce meaningful heatmaps you need a large sample size. (Nielsen Norman Group suggest 39 testers). The problem with large sample sizes is the results requires extensive data analysis, which in turn slows down the turnaround time of a project. It also increases the project cost.
- And then there is the problem of how you interpret the data. A number of commentators (e.g. Spool, 2006, Graphpaper.com 2006) have questioned the value of heatmaps, partly because of the practicalities but also because the results can be misinterpreted:
- Heatmaps can show us where someone was looking on a screen, but it can't tell us what they were seeing or thinking whilst looking at it - did they register what they were looking at, did they understand it? And if the tester did not look at something directly, did they see it with their peripheral vision?
- Does a long fixation mean that someone is looking at something because it is interesting to them, or because it's very confusing and they're having to spend time making sense of it?
- Does a busy gaze plot tell us that there are lots of interesting things to look at on a page, or that a user is confused about where to go to achieve their goals?
Eyetracking is a valuable tool in usability testing but use it carefully to avoid misinterpreting results and skewing your research findings.
Read more about how to do usability testing including our facilitator top tips and the biggest challenges of usability testing.
