While user testing is an important and necessary part of the product lifecycle (don’t ever let anyone tell you otherwise!) it’s not a catch all. Users tend to do better on a site during usability testing than they would over normal circumstances. Which means… that your site is actually worse than your users say it is! *gasp*
Why do users do better in testing?
Users have a higher rate of task completion during testing for a few reasons:
1. They feel pressured to finish the task when they would ordinarily just abandon the site.
2. They know they’re being observed so they want to do it right for their own sake. No one likes to ‘fail’, even less so when people are watching (even though the user is not what’s being tested).
3. They’re reading everything much more closely than they normally would.
In one instance of testing, the app we were looking at had a “yes” and a “no” button but both were white text on a red button. Subconsciously, we all know red means “stop” or “no”, so we assumed the red “yes” button would trip up the users. None of the six users we tested specifically mentioned the red buttons during testing, but we noticed that they were taking longer to complete than they should have.
When we probed on the longer completion time the users all answered along the lines of “I was reading everything” and “I wanted to do it right”. The two red buttons were definitely confusing the users but we didn’t find out until we specifically asked them about it after the test.
What do I do about it?
If user testing doesn’t catch everything what can UX Researchers do to uncover more usability issues like the example above?
Conducting usability testing and user validation testing early and often in the product lifecycle will provide more opportunities to uncover usability challenges. Testing iteratively will allow you to fix major usability issues and uncover new ones.
Do lots of different research.
Contrary to popular belief, user testing is not the only kind of user research! Shocking, I know. A number of different research methods can be used at all different stages and for different purposes.
Look at your metrics.
Once your product goes live watch your metrics closely for drop offs during a certain task. This is a red flag that there is an issue at that step in the task.
Get an outside opinion.
Designers inevitably get attached to their work and it can be very hard to notice subtle usability issues. An outside opinion in the form of a Heuristic Evaluation from someone trained in user psychology should go a long way in filling in the gaps.
User testing is not a catch all, but it is a good start. If you’ve got limited resources testing is a great option.