A heuristic evaluation is a usability analysis of a site that you perform yourself. Essentially, you have a checklist (comprising the heuristics) that you run through – and then you take the findings from that checklist and translate them into actionable, revenue-generating design decisions.

Here are some typical things you do in a heuristic evaluation:

• Perform a checkout with a real credit card. (Refund yourself later, of course.)
• Search the site for products that fit a specific criteria.
• Add upsell products to your cart.
• Tweak sliders for a custom plan selector.
• Run through these same tasks in a variety of browsers, on a variety of different device types, under a variety of network conditions.

Most team members never bother doing these things after initial setup. And a store’s checkout form can aggressively leak revenue with random, pernicious bugs that exist only in contexts that you never bothered testing. Taking the time to run through common tasks can surface all sorts of conversion-killing bugs.

Heuristic evaluations are one of those things that you can do with a budget of $0 and no executive support. You can take the results of these, share them internally, and make a strong economic case for higher-touch optimization efforts.

While it’s awesome that you can do this sort of thing yourself, there remains no substitute for similar sorts of tests with real-world, paying customers. Usability testing & interviewing happen in conjunction with heuristic evaluations, not instead of them. But heuristic evaluations are great for making the case for research-driven optimization.

You run a heuristic evaluation in 3 steps:

1. Develop a checklist. Struggling to build one yourself? Great, the whole checklist I use with roadmap clients is below. It’s intended for SaaS businesses, but almost all of it is repurposable for ecommerce as well.
2. Run through each item on the checklist, and outline what’s working and what could stand to be improved. Ideally, you’ll be tying each one of these to specific revenue figures: for example, if you can show, in Google Analytics, that mobile is converting 30% worse than desktop, then that’s a strong business case for closely examining the usability of the mobile site, in a variety of devices and contexts.
3. Write up a report that prioritizes various fixes. ConversionXL has three different frameworks that you can use for your own purposes. I use Trello boards for shuffling around various issues.

Develop Your Own Checklist

First, break down each type of page in your funnel

For most stores, this means you have:

• Home
• Collection
• Product Detail
• Cart
• Checkout
• Search results
• Thank you/order status

Then, break out areas in each page

Areas are general sections that correspond to a given focus on your store. In Shopify, this roughly corresponds to individual sections, especially on your home & product detail pages. Some of these, especially navigation, are repeated across your store. Each of these areas contains one or many elements, which we’ll talk about in a minute.

For example, on your home page, you probably have:

• Desktop nav
• Mobile nav
• Masthead
• Description
• Cross-selling
• Mailing list callouts
• Popovers

And on your product detail page, you probably have:

• Desktop nav
• Mobile nav
• Image carousel
• Buy box
• Description
• Cross-selling
• Reviews

You get the idea.

Next, identify each element in each area

Then, you’ll go into each area and audit the elements in all of them. For a home page masthead, that includes:

• The background image
• The headline
• The call to action button
• Call to action text

That’s a simple example. Your product detail page’s buy box includes:

• Product name
• Review average & count
• Price
• ATC button
• Quantity selector
• Variant selector(s)
• Availability
• A short description, maybe

Our checklist

While I strongly encourage you to take a different approach than our own, here, for reference, is the exact heuristic evaluation that we run for all of our roadmap clients:

Business & Positioning

• Is a positioning statement articulated upfront?
• Is it capable of attracting attention immediately?
• Does it solve a concrete pain?
• Does it have a specific customer in mind?
• Does it focus on their needs?
• How much of the text is focused on I/me/we, versus you/your/yours?

The Funnel

• Is it clearly defined?
• What pages does it comprise?
• Can we eliminate any pages?
• Are there any alternate entry points (e.g. landing pages)?
• Are people making side tours (e.g. to features pages)? Are these people more or less likely to convert?

The Home Page

• Is there social proof?
• Is there a chatroll? Does the chatroll have a clearly defined purpose?
• Is there a rotator/carousel? Does it fit each of the Baymard Institute guidelines for carousels (http://baymard.com/blog/homepage-carousel)?
• Is there press proof beyond just logos indicating that coverage happened? Is the press proof for well-respected outlets (NYT, WSJ, FT, etc), or is it for outlets that act as press release factories (TechCrunch, TNW, etc)?
• What does the masthead look like? Full-bleed? Stock photography or a real person? Does it lead with a headline?
• Is the headline specific, attention-grabbing, and focused on alleviating the customer’s specific expensive problem?
• Is there a subheader? What does it focus on next?
• What links are in the header navigation? How does engagement vary among them? Can any of them be safely moved into the footer?
• Is the CTA in the masthead?
• Is the CTA the final element before the footer?
• Is the footer clean & organized?
• Do the home page’s modules focus on the benefits that the customer will get from the product, or do they focus on the product’s features?
• Are the customer’s objections audited & addressed?
• Is there a video? Where is it placed? What behavior does the video have on page load?

Calls to Action

• How many of them are there?
• Does each call to action focus explicitly on revenue generation?
• Is there a clear delineation between primary & secondary calls to action?
• How consistently treated are the calls to action?
• Are the calls to action reworked from a theme default?
• For modal calls to action, is there a “no” option?
• How many modals are there? What behavior do they take (exit intent, time-delimited, etc)?
• What call to action does each modal support?
• Is a freebie given for email-based calls to action?
• Are the calls to action intrusive or “sleazy”-looking?

Pricing

• How clear is the pricing?
• How easy is it to get to the pricing page?
• How likely is it that people beeline for pricing?
• Is there any upsell in the pricing section? Any “a la carte” options?
• Is there any high-touch, “concierge”-based pricing?
• Is the pricing organized into plan columns?
• If so, do the columns have plan names, a most popular plan, a focus on differentiation between each plan, etc?
• What does the objection-busting FAQ look like?
• Is there a split between annual & monthly plans? Which one is the default? What happens when you swap between them?

Sign Up

• Is this page different in any way from the login screen?
• Is this page different in any way from the default-styled signup form for your given framework?
• Are objections still addressed here? Is there any “follow-through”?
• What happens if errors occur?

The Checkout Process

• Does credit card number, zip code (optional, for USA only), CVV/CVC/CID, and expiration date show up with the numeric keypad on mobile?
• Is autocorrect turned off for name, address, and city?
• Is the country selector geolocated?
• Does the email keyboard appear for the email field on mobile?
• How is the country & state selectors’ behaviors?
• Does the logo of the credit card issuer (Visa/MC/Amex/Discover) appear when you begin to type it in?
• Does CVV/CVC/CID show in a diagram on an example credit card after its corresponding type is selected? (Each one has a different acronym, and placement & validation is different for Amex.)
• Is the usability of the credit card form in line with industry standards (cf. creditcardjs.com)?
• Does the padlock appear in the browser bar, indicating HTTPS?
• Are you punted to a third party checkout solution (2CO, Shopify, MoonClerk, PayPal)? Is this in conjunction with a more conventional checkout? What’s the customer breakdown between each checkout flow? Is conversion helped or harmed by the third-party checkout provider?
• Are there multiple ways to check out for multiple different credit card issuers (e.g. Amex through PayPal, Visa/MC through Authorize.net)?
• Do you accept payment by check or wire transfer? Under what conditions? How do you handle recurring payments?

Per-Browser Issues

Vet GA for all browsers. Are any browsers not converting well? Assess why by loading the site in that particular browser, determining page load time relative to the mean (which you can get in GA), and aggressively bugfixing for it.

Per-Device Issues

Vet GA for all devices. Are any devices not converting well? Assess why by getting ahold of an example device, and loading the site in it.

Mobile Issues

• Does the home page render well on mobile?
• Is there a significant conversion difference between smartphone, tablet, and desktop? Figure out why and what you can do to fix it.
• Does the page show a “hamburger menu”? If so, what does that look like when opened?
• How does pricing look on mobile?
• How does the page load on 3G?
• Is page load time severely impacted for mobile devices, and if so, what can we do to make it render at parity with desktop/wireline connections?
• Are people ready to make a purchasing decision on mobile? If not, can we create a funnel that directs them towards some other action?
• Is the checkout form responsive?
• What happens if the internet connection is lost mid-signup? What happens to onboarding?

How do I run through these?

For every single element on a page, you need to exhaustively describe what makes that element perform as well as it possibly can. I evaluate enough to comprehensively answer the question. That might involve a lot of further investigation, more questions, and other points to evaluate.

Where heuristic evaluation falls short

Heuristic evaluation isn’t appropriate in all cases. What does it not work for?

Understanding what your customers are doing

A heuristic evaluation is subjective to your own experience, and your customers aren’t creating your heuristic checklists. As a result, you need different research methods to understand how customers are behaving on your live store.

Instead, use: heat & scroll maps, analytics.

Determining how your customers feel

Again, you have no idea how your customers respond to your messaging, copy, branding, or overall ethos as a company. The only way to accomplish this is by asking them.

Instead, use: interviews.

Discovering whether your customers have different abilities from you

You know too much about your store. You know too much about how technology works. You know too much about your products. Your customers know as much as they care to know. If you want to understand where your store’s usability truly falls down, then you need to watch your customers go through tasks themselves, not yourself. This will uncover many issues that your evaluations are failing to.

Instead, use: usability tests, post-purchase surveys.

Learning what problems your customers want solved

The purchasing process is holistic, continuing past conversion and on to delivery & use. You care about making sure customers buy. Your customers care about whether your product looks like the photos, whether it fits well, whether it does what it says it’ll do. What objections can you address before customers convert? When do those objections appear? No heuristic evaluation is capable of answering this.

Instead, use: customer support inquiry analysis, post-purchase surveys.

← Back to the Blog Check Out Draft’s Store →