Heuristic Evaluation

You’ve got a prototype for a product that you think is stellar. You’ve interviewed some users, mocked up a design, and you think you’re ready to start building it. You’re confident your design will work, but you want to get another set of eyes on it first. Enter the heuristic evaluation.


A heuristic evaluation is when an independent designer reviews your work against Neilson’s 10 Usability Heuristics. If you’re not familiar with these give them a read, they’re a classic for the field of UX. It should be an independent usability expert (often a senior designer) who has not worked on the project yet, has no internal politics to contend with, and has domain and industry expertise. For example, if you’re evaluating an e-commerce website the designer should be familiar with consumer purchase behaviour, e-commerce design, and retail operations.

The output or deliverable is a list of contextual usability errors and how to correct them. This can range from a debrief call and notes or sketches on the prototype to a full detailed report. A heuristic should be done to test or validate the design before production, but it is not a replacement for usability testing. I repeat, it is not a replacement for usability testing.

The Case For Heuristic Evaluations

Specific Elements

Heuristic evaluations are useful when you’re trying to focus your attention on a specific issue or element of the design. Other testing methods allow the participant to roam free or with guided moderation, so there is no guarantee they’ll give feedback on the element you were hoping to review.

Mentorship

Teaching and mentoring are a great opportunity to conduct heuristic evaluations. The junior designer will create a design for a product and the senior designer will conduct an informal side-by-side heuristic evaluation. The prototype gets an expert review and the junior designer gets to learn more about how Neilson’s 10 play out in context before usability testing. Everyone wins!

Budget and Time Constraints

Heuristic evaluations are generally faster and cheaper than usability testing. There are less logistics involved and you don’t need the same lead time to come up with a moderation guide and research plan. That said, a heuristic evaluation should never replace usability testing and instead be done before usability testing to catch the more obvious usability challenges. This frees up time in testing with actual users to focus on the nuances of the product and the less obvious usability challenges – you know, the thing that gives usability testing its value.

The Case Against Heuristic Evaluations

Bias

The two fundamental problems with heuristic evaluations are that it’s based on one person’s opinion and that there is no user input. With a single person conducting an evaluation there is an inherent bias based on that persons experience with interfaces. This can be mitigated by having more than one person involved in the evaluation but then we approach design review territory.

Context

Neilson’s 10 do not consider the context in which this product and its users live. It takes a ‘checklist’ approach to usability that does not always apply perfectly to every situation. An experienced usability expert will realize this and ask for context for everything they’re evaluating, and they’ll need to make decisions and recommendations that may differ from the 10 heuristics. If the context is not properly communicated the recommendations offered by the evaluator may not fit. This is the risk of not involving real users.

Lack of User Input

Real users understand the context of the product because it’s built for them – and their personal situation. Failing to test in ‘real life’ and all its complexity will undoubtedly leave gaps in your product. It’s the equivalent of a lab test versus a field test.

Usability Testing and Heuristic Evaluations

A heuristic is best done before usability testing to catch the obvious usability challenges. If budget and time only allow you to do one – do usability testing.

With usability testing you’re uncovering actual user challenges rather than assumed ones. You can follow all of the design ‘rules’ and still have challenges you can only catch with testing. That’s the beauty of usability testing, you always learn something. Usability does not take that much more time or effort than a heuristic evaluation and the value you realize is far greater.

TL;DR

Heuristics are great for flexing your critical design muscles and practicing user empathy. They’re also great for mentoring a junior designer and learning the practical applications of Neilson’s 10. Heuristic evaluations are useful and they have a place in the design process, but usability testing is your best bet for uncovering real issues and it’s the best bang for your buck. For all their pros and cons the best scenario for a heuristic evaluation is to do it before your first round of usability testing so you can focus on the nuances of the product with the limited time you have with the user.

Share itShare on FacebookEmail this to someoneShare on Google+Share on LinkedInTweet about this on Twitter