Behavior recordings are videos of your prospective customers’ cursors or taps as they navigate through a page on your website. They’re one of the most deceptively tough research methods to pull off well. Here’s why:
- You gather hundreds of them, making analysis time-consuming.
- Most of them are similar, making it mind-numbing.
- It’s easy to focus on outliers.
- Most of it might not even be conversion-minded traffic: people loading a page and closing the tab, existing customers beelining for the login page or customer support, etc.
That all sounds really hard! So, what can you get from behavior recordings?
- Tremendous insight into specific conversion-killing usability issues.
- Understanding of cross-device design constraints, especially on smartphones with non-standard screen sizes.
- A greater sense of what features get ignored on a page – one that you can’t get with analytics or heat maps.
- Far better presentation of these issues than you would typically get with a large, complicated data display from Google Analytics. I know you don’t need a cool, sexy way to show something, but this sure does help when it comes to convincing other team members!
ConversionXL has some broad-stroke strategies for how to analyze behavior recordings, which they call “session replay videos.” Here’s how I do it:
Setup: Install Hotjar
Yes, you need a new bit of software to do this right. Hotjar does behavior recordings better than anybody right now: it tackles cross-device issues handsomely, and unlike VWO it actually has the feature turned on by default. Hotjar is mostly known as a heat map tool, but you’ll be installing it for behavior recordings.
Here is how you use Hotjar to gather behavior recordings:
- Sign up for Hotjar.
- Install its tracking snippet on your site.
- When logged into Hotjar, go to “Recordings” under the “Analysis” heading in your left sidebar.
- Hit the “Record Visitors” button. Gather as many as you’d like – I usually do only 100, but if you have the resources to analyze 2,000 recordings, by all means. Enter the page URL. Hit record.
- You now have recordings.
This should take you, like, ten minutes.
Synthesis: Farm It Out
Behavior recordings are a widespread, significant undertaking that should involve as many people in your organization as possible. Got other 9 team members? Watching 10 recordings is way easier than watching 100.
But then you run into a big issue: when 10 people watch recordings, you’re dealing with 10 different sets of opinions, 9 of which are probably not design experts. So, how do you tackle this? By providing a consistent set of things to look at… with a heuristic evaluation! Here’s our tutorial on running a heuristic evaluation, and here’s the exact heuristic evaluation checklist that I use with my clients.
Essentially, you’re providing a series of usability requirements for a fellow team member – so you can get back a set of data that’s as consistently analyzable as humanly possible. The #1 goal is to figure out usability issues, especially those that are systematic and widespread. If three people struggle to fill out a form, that is a problem that you need to fix. If one person struggles, perhaps consider exploring it.
The analysis you get back from a team member should be as systematic as possible. They should isolate the specific usability issue, the place it occurs, the device(s) that it occurs on, and the quantity of recordings that the issue occurs in.
Analysis involves identifying all issues, along with their corresponding frequencies. You’ll probably find two or three major issues, along with a (very) long tail of other issues. Dump all of these into a Trello board, and make the big issues a high priority. Here’s the Trello board I use with my clients.
There are a bunch of what-ifs here:
- What if people magnify the issues found in a single behavior recording? It’s hard to pay attention to behavior recordings, and inexperienced researchers can easily zoom in on one problem. On the one hand, maybe the problem really is significant – but significant problems are usually widespread. I recommend taking their suggestions seriously and investigating whatever they’re saying, while recognizing that it’s entirely possible that the problem won’t be addressed as quickly as they may want.
- What if people provide insights in the wrong format? You have two options: either try to munge their write-up into something usable, or throw it away and watch their battery of videos yourself. Remember: your goal is always to take action, not polish data. Customers don’t care about your data!
- How do you incorporate any asides that people might be writing about? People may be offering information on background that they think about as they watch their set of recordings. That’s useful, but only peripherally so. I recommend taking most asides with a grain of salt unless they’re demonstrably backed up by research with paying customers.
Use this in conjunction with quantitative methods
Behavior recordings aren’t the only thing you should be doing, of course. You should also be running usability tests on your funnel, especially with a site like UserTesting; and heat & scroll maps, such as with Hotjar or VWO. Behavior recordings are just one part of a multi-pronged research strategy, and while a bit advanced, they can be very powerful!
Most stores get CRO wrong.
They plan CRO activities at the wrong time, or they don’t know how to research new tests. Our free CRO roadmap draws on our years of experience to make CRO easy for you to implement for your store.
Sign up below to get your Ecommerce CRO Roadmap today: