User test early - quicker, better, cheaper outcomes

We want to undertake projects for clients that will save them money and maximise the success of the project. However, mostly we end up testing websites when they're live; usually there's been no user research or testing beforehand, and all the design and build costs have been incurred. That's not an issue if the site is coming up for redevelopment (when the research will inform the new site), but often these are sites that have only been recently launched (or re-launched), meaning there's no spare budget for revisions. If the site works perfectly and achieves its aims then fine, but regrettably that's rarely the case. Occasionally we get to test sites just before launch, which means that some revisions can be made to increase site efficiency and effectiveness. However, often, the testing surfaces fundamental issues about the site that would ideally require a significant rethink. Either way, there's often limited budget and time to launch a site that delivers a good user experience and good outcomes for the site owner.

Nielsen (2003) says "there is value in fine tuning the details in the user interface, but the impact on the final user experience is not as great as the impact from fundamental changes made early in the design". He goes on to estimate that "the benefits from early usability data are at least ten times bigger than the benefits from late usability data; it is 100 times cheaper to make a change before any code has been written than if the same change has to be made after the code has been completed."

Where possible, we recommend that clients test as early as possible in the development cycle - it will save time, money and angst in the long run.

What can be tested?

Just about anything can be tested - any early stage testing will help inform and improve the finished website. The testing can encompass a full user journey on a site or it may focus on microsites or specific elements of a user journey (e.g. a specific form or the check out process).

Snyder (2003) focuses on paper prototyping, where the ideas are (as it suggests) on paper and the pieces of paper are manipulated by a human acting "like the computer". Our experience is that valuable feedback can be obtained from early stage testing undertaken either on paper or on screen; but, if anything, we prefer the prototype to be screen based as we've noticed differences in the way people look at screens to the way they look at paper. Prototypes can be hand drawn, PowerPoint slides, Visio drawings, anything so long as its not fully coded; these can very easily be made clickable, and populated with dummy content (but not Latin 'ipso lorem') to simulate a user journey through the site.

Although the visuals and content can be draft, the project is likely to be more successful if there's clarity about:

• The site owner's aims for the site
• The target users
• The users' goals that the site will support (this will depend on the site owner's aims)

(See our paper "Website strategy and measuring success" for more thoughts on this.)

Also, getting the key organisational stakeholders involved from the beginning will also help minimise time delays and additional costs - this might include people from marketing or key content writers.

What are the benefits of early testing?

• There are a number of quantitative and qualitative benefits to early user testing:
• It can be set up quickly, easily and cheaply: depending on the number of user groups, valuable feedback can be obtained from 3 testers (See our paper "How many testers are enough")
• It saves time and money across the life of a project because early stage changes can be incorporated quickly before lots of expensive design and coding time has been invested
• Seeing 'real' users interacting with a prototype challenges tacit mental models about how the world works and, therefore, the assumptions underpinning design choices
• Testing also challenges subjective decision making, instead of sitting around and debating the merits of various solutions within the team, it allows the user to make the decision by their behaviours demonstrating which is the most effective route
• Prototype testing will surface user requirements that have not previously been identified, and so inform more generally about what users want
• Testing will get feedback on a 'macro' strategic scale - do users understand the concept.
• And feedback on a more 'micro' tactical scale - focusing on specific elements of the interface
• User feedback at an early stage will result in an improved user experience, both usability and content meeting the users' needs - which will improve user satisfaction on all sites and conversion rates on transactional sites
• User testing may also surface issues outside the user interface e.g. issues about organisation branding and credibility, other offline requirements, that will enable the overall online and offline user experience to be improved
• Testing will enable prioritisation, surfacing what's most important to users and identifying where development effort should be concentrated
• Testing helps to pull together the client and the developer team to help ensure that their understanding of the user goals are aligned, and that they have a clearer joint picture of what 'success' of the finished site will be.
• During the earlier stages of a project, members of the development team will be less attached to design ideas and, therefore, less defensive about making changes; this minimises what Snyder (2003, page 60) calls the 'invested effort' and makes it easier for the team to revise the design
• And because of that, early testing can promote creativity because the team is not tied to one solution
• This can often lead to the development of alternative solutions which can be tested side-by-side to identify what works best for users

Some case studies

Private Sector

A client, that focuses on connecting trades people with people who need work done to their homes, wanted to improve the trades peoples' experiences on the site, specifically their understanding of the overall proposition and the ease of the joining process. The client prepared prototype pages in Visio, and we 'hot spotted' links to make the prototype clickable. Working closely with the client, we then undertook iterative testing and prototype revisions across the duration of one working week:

• On the Monday morning we undertook testing on the first version of the prototype with three representative users, the clients were active observers, capturing issues as they arose. During the afternoon, Web Usability facilitated a discussion to identify revisions
• By the Tuesday afternoon, the client had made revisions to the prototype; Web Usability then undertook a heuristic review to check that the issues identified in the first round of testing had been addressed. If they hadn't, the client fixed them there and then
• On Wednesday, a second round of user testing was undertaken with three testers, followed by the facilitated discussion
• On Thursday the client made further revisions, and Web Usability again provided a check before
• A final round of user testing on the Friday. A few minor issues were identified, and the client felt confident to move the prototype to the build stage

The client found that this rapid iterative testing/revision process enabled them to launch a new site more quickly and with greater certainty of success. We are now undertaking a similar project with the same client on another part of the site.

Public Sector

A  client wanted to launch some animated tutorials to help users get the most from their site;  a massive repository of records and information. We worked with the client and the developers over an eight week period, undertaking three iterations of testing with six testers representing target users in each round. In each iteration, prototype versions of the tutorials were tested - reviewing the visuals, the audio tone of voice and style, and the details of the content, to check that it was understandable by target users. After the first round of testing, as a result of the user evidence, there was a radical rethink of the approach that should be taken to the tutorials, as well as identifying a number of more detailed changes. By the final iteration, the client and developers were confident they had a format that would help communicate effectively what users could do on the site, and so become a genuinely useful resource for users.

In both cases, these clients have found that testing at an early stage has enabled them to embark with greater confidence on the expensive and lengthy coding and building stages of development.

References

Snyder, C. (2003) Paper Prototyping: the fast and easy way to design and refine user interfaces, Morgan Kaufmann Publishers
Nielsen, J., Alertbox, April 2003 www.useit.com/alertbox/20030414.html