Getting Great Case Studies
Why would you want to write up a case study? They’re time-consuming, and they always seem to be last on your todo list.
Well, you might be a consultant, wanting to prove that you’re a safe bet. Case studies are the coin of any consultant’s realm: they show you’ve worked for other clients before (and hence are seasoned), that your work has an economic ROI (which is rarely conveyed in the tech industry), and that you’re preferable to competitors that may not have as much under their belt. Case studies are also a terrific way to raise your rates, moving slowly towards the end game of value-based pricing.
You might be in-house, wondering whether case studies are a good idea. They act as effective postmortems, though – allowing you to premortem future decisions with the research you gather from test findings. More importantly, they allow you to gain significant political currency by showing that optimization has a clear ROI. Those who are skeptical of testing may need to be convinced, especially after several null results.
Remember that there’s no way for you to overcommunicate test findings. Think of a case study as a more formal, high-fidelity approach to the usual test analysis that you’re doing. Your biggest wins – and most surprising results writ large – should be polished up and shared formally whenever possible.
The parts of a conversion-focused case study
Here are some of the parts of a good case study. While every good case study has all of these parts, I would caution against plugging the following bits in like they’re lines in a database. Good case studies are written – and they’re conversationally-phrased, like a persuasive story.
With that in mind, all case studies need the following:
The business’s real, actual name
No, you can’t NDA a case study. “A large SaaS” is not the business’s name. “A Fortune 500 company” is not the business’s name. Naming the company is essential to ensuring the case study will be trusted and acted upon – even if the reader doesn’t know the company in question.
If you’re a consultant, never accept work on NDA, and contractually mandate the ability to write case studies.
The underlying research
All test ideas should be backed with research, of course, so you should be doing what you can to articulate your research process to the reader. Here’s what you should always try to mention when discussing research:
- What research method you chose, and what led you to choose it
- What results it provided
- How you synthesized those results into a revenue-generating, testable design decision – and how that decision reflects the research
In essence, you’re discussing the before (the problem), during (the research), and after (the design decision) of the research activity.
The control & the variant
Presumably you’re A/B testing this change, so you should probably include screenshots of the control and the variant.
Also elaborate on the change. What’s changed? Why? What hasn’t? Why?
The economic upside
You need a metric, and that metric needs to be tied to the economic health of the business. Any conversion-focused work exists to either increase revenue or decrease costs. If you say “engagement increased”, that’s not a good case study. There needs to be a number, and the number needs to begin with (or imply) a dollar sign.
Here are good examples of economic upsides:
- ARPU increased by 69%.
- Conversions from trial increased by 23%.
- We added $42,000 to MRR – and that’s increasing as the business grows.
- We cut 2 products from our line, dropping manufacturing costs by 33%.
Here are some bad examples of “economic upsides,” in that there is no actual mention of an economic upside:
- Clicks to the signup page increased. (Okay, what about the conversion rate? What about revenue-generating transactions?)
- We drove more traffic to the landing page. (Was the traffic wallet-out? Did the bounce rate spike? What happened to the conversion rate?)
- More people clicked around more often. (Where did they click? Did they buy? Were they existing customers?)
The business ramifications
Revenue-generating design decisions never happen in a vacuum. They also have an impact on:
- Team enthusiasm for conversion-focused research activities
- Customer happiness
- Ongoing testing activity
- The team’s culture (especially with respect to hiring & overall growth)
A good case study talks about as many knock-on effects as possible. If a single change to a website results in the ability to hire 10 new people (which has happened to my clients before), that doesn’t have direct connection to the case study at hand, but it sure matters.
A testimonial (if you’re a consultant)
Good testimonials involve the specific name, position, company, and quotation from the client – with a well-shot photo to match. The client should do what they can to outline the ROI upside.
Want to see how this works in practice? Here’s a slew of testimonial quotes at Baymard Institute, most of which check these boxes (although lacking photos of real faces).
Good places to share case studies
So, you have a case study. Hooray! Where should you post it?
- On your consultancy’s site – with a call to action to apply
- On your blog, in order to establish evergreen content and share it on social media
- On your mailing list – and refer back to it (by linking the blog post) whenever possible
For in-house folks
- On your project management tool of choice
- On Slack
- Direct emails to decision makers & any skeptical stakeholders, with an entreaty to meet to discuss further
Sharing case studies is vital towards ensuring they get read, of course – but it’s also important towards ensuring that you maintain your credibility as a tester and researcher.
Some good examples of case studies
- Copy Hackers are terrific at writing persuasive case studies in the form of long-form stories. Here’s one about SweatBlock.
- Here's a good case study from ConversionXL about mobile UX.
- Amy Hoy wrote a good story about how she bumped her conversion rate by 2.4x.
And these are just a few, of course.
Most stores get CRO wrong.
They plan CRO activities at the wrong time, or they don’t know how to research new tests. Our free CRO roadmap draws on our years of experience to make CRO easy for you to implement for your store.
Sign up below to get your Ecommerce CRO Roadmap today: