The A/B Testing Manual
Capture 5% more customers. Then do it again.
You have a product. It’s seen some success. You’re proud of what you’ve built. Your business is growing at a decent clip.
But right this very second, there’s a perfect customer on your sales page. They’re perfectly in your target market, and they want a lot of what you have. If you landed them as a customer, it’d be a massive win.
They skim your copy. They aren’t persuaded. They close your tab. They go with a competitor. In less than a minute, you’ve lost them.
Lost Business is Very Hard to Get Back
That minute hurts. Why? Two reasons:
- Lost business is difficult to undo. Customers become locked in, making for a high switching cost. People buy products and immediately forget about them – or worse, they become brand-allegiant to your competitor. Your competitors are doing everything they can to capture the revenue that they know they need to survive – and they’re eating your lunch in the process.
- Lost business is invisible. You don’t even realize it’s happening. It doesn’t look like churn, or customer returns; there isn’t as much immediacy to it. But it happens – it’s happening right now – and it’s a very real issue.
It’s absolutely imperative that you do everything you can to win the customers you want. You need to survive in an increasingly competitive market.
Fortunately, there’s a way to address this: by researching what motivates new customers, and testing the economic impact of how you communicate to them. And you can start today.
The A/B Testing Manual
How? With a new course that teaches you everything you need to know about capturing lost business. It’s called The A/B Testing Manual, but it’s a lot more than a how-to guide for launching A/B tests. It’ll teach you how to plan, execute, and analyze a company-wide strategy that researches your customers, analyzes their needs, creates a resonant offering, and generates at least 5% more annual revenue for your business. Research-driven A/B tests improve your brand strategy, company values, and your product itself.
It’s not enough to simply want to test. It’s not even enough to know how to test. Research-driven testing requires a long, hard look at the way you make design decisions in your organization. And this video course will help your business do what it takes to focus on optimization, now and in the future.
So, why is testing important?
- Testing makes you money. How much? Try $300,000,000. Want a more modest example? Here’s a $4,000 MRR boost – which happened by adding a single button. Small changes can have outsize impact: for example, changing single words can boost a site’s click-through rate by 161%. It’s no longer the case that these are outliers, either: well-researched, incremental tests have insanely high success rates, because they capture the lost business that you’re currently leaking.
- Testing means your design decisions are no longer speculative. Design decisions can’t be validated until they’re put in front of paying customers. A/B testing lets you understand the economic impact of specific design decisions, allowing you to proceed as deliberately and carefully as you do in your other business affairs.
- Testing defangs your internal debates. Smart people hold strong opinions – making them likely to resort to infighting, backstabbing, and changes by arbitrary fiat. This sort of toxic culture is actively killing your ability to attract more customers. Research-driven A/B testing provides a solid, actionable process for understanding what works, what doesn’t, and what to do next – without resorting to the highest paid person’s opinion.
- Testing turns your company’s values outward. It forces you to ask: What does the customer want here? What are they motivated by? And how can I help them achieve their goals?
Never forget: focusing on your customers brings you more customers. Are you focused on helping your customers, or are you focusing on what your coworkers want?
Why You Can’t Do This Alone
Even if you’re already convinced about testing, there are a million ways to do it wrong. A/B testing is tricky on numerous fronts:
- There’s no clear place to start. You get a framework and sign up – what then? With an overwhelming amount of contradictory information out there, it’s hard to know what to do first. I’ll tell you exactly how to get started, because your framework’s designers won’t.
- Running A/B testing frameworks is hard. They’re heavyweight, powerful, complex apps. When you execute a test, it can be riddled with bugs. I’ve seen pretty much all of them. I can help you run tests that are more likely to work the way you intend.
- Deciding what to test is hard. You can test a call to action that you heard “performs well”, or you can research precisely what call to action is most likely to resonate with your customers’ values. Which one do you think is more likely to capture untapped sources of revenue?
- Changing your company’s internal processes is hard. It’s not enough to simply run a test and act on it. Your entire culture needs to embrace optimization from the top. I’ll run down all the ways that your organization can undermine a thoughtful, well-researched testing strategy from the get-go – and you’ll ditch all the broken, toxic habits that have hurt your team.
Fortunately, there’s a way past each one of these issues. And there’s a better way to run revenue-generating A/B tests in general. Wouldn’t it be great to have a resource on your side that’s been proven over several years of in-the-trenches work?
What You’ll Get
The A/B Testing Manual is the best possible way to get started. With 5+ hours of strategy-focused video & over 150 pages of tactics, it will teach you:
- How to get your site in order before you launch a single test.
- Why A/B testing alone is never enough.
- Why there’s no “one secret trick” that always works in testing.
- The relationship between design research & A/B testing.
- How to improve your tests’ success rate.
- Why it’s important to test per platform, especially on mobile.
- The first steps you need to take in order to move away from the default template you probably used when starting your business.
- The right way to analyze test results – and how to use them as a way to plan more tests.
You’ll unlearn all the bad lessons that “CRO gurus” and “growth hackers” spout to give A/B testing a bad name. And you’ll learn how to do honest, ethically sound A/B testing in an way that gets more customers and keeps them happy. If you’re doing this right, you’ll cover the cost of the highest-value option in a single day.
The A/B Testing Manual has only been out for a short while, but people already love it. In the words of ecommerce developer Franz Sauerstein:
Before purchasing The A/B Testing Manual, I dug really deep into Nick's publications, because I was unsure if he could deliver on his promises. It quickly became clear that he is regarded as one of the absolute experts in his field and has a very bright track record. After reading The A/B Testing Manual, I overhauled my own testing approach, updated my templates and processes, and was able to not only work more efficiently, but also deliver better results.
You Now Have 4 Options
- Get the full course. First, you’ll receive a book that shows you all the tactics you need to build A/B tests in your organization. Here’s a free chapter if you’re curious.
But that’s not even the half of it. My friend Patrick McKenzie and I recorded a 5-hour video course, filled with over a dozen screencasts, that provides a huge deep dive on exactly how to enact a solid testing strategy.
Patrick is broadly considered among the best A/B testers in the industry, having pioneered significant portions of the practice for companies around the world. He’s captured tens of millions of dollars in revenue for his clients. You’ll probably make back the cost of this course after watching for 30 minutes.
You’ll also get exclusive interviews with Visual Website Optimizer CEO Paras Chopra, freelance educator Brennan Dunn, copywriting expert Joanna Wiebe, and Test Triggers founder Josh Frank. They’ll share their valuable experiences with testing, copywriting, and generating revenue for businesses just like yours.
Finally, you’ll get a comprehensive toolkit for A/B testing, which will include:
- All of the checklists I use with my Draft Revise clients, to ensure that tests are running on time and are bug-free.
- The initial heuristic evaluation that I perform on your cart & payment funnel.
- The exact process I go through when writing a Revise Express report.
- Pretty much every way an A/B test can break that I know, and what you can do to fix it.
- A free month of Revise Weekly, a series of actionable, valuable lessons on A/B testing & design research. (This is worth $49 alone!) Zero risk in following along; cancel anytime.
If you follow it right, The A/B Testing Manual will have a few nice outcomes for you:
- Clarity in making design decisions, through knowing the economic impact of each one.
- Significantly reduced risk in making changes that don’t fare well for your business.
- The peace of mind that comes from certainty in the changes that you make every day.
- At least 5% more top-line annual revenue.
Most importantly, though, you’ll execute the right strategy for focusing your business around optimization – without having to fumble in the dark first. Inaction has a cost: if you don’t get this, how much in time & resources will you waste through testing by trial & error? $2,000? $5,000? Or even more?
Okay, Fine, Here’s a Lesson for Free
Don’t think that these outcomes are reasonable for your business? Been burned by testing before? Watch this free lesson from the full course, about exactly what you need to do to create revenue-generating test hypotheses. You’ll learn all sorts of actionable steps about how to get more out of testing than you put in.
What? Oh, right, the button.
Need a license for your entire team? We’ve got you. Download & share forever.
You can also pay for a team license via ACH or check – email us.
- Get the book, worksheets, and interviews. If you’ve already handled all of the politics in your organization – or if you’re okay with leading strategy yourself – then the book and worksheets will provide all of the tactics you need to run successful tests. This will also come with the full course’s expert interviews, as well as a free month of Revise Weekly. You’ll be on your own when it comes to strategy.
- Get the full course & Revise Express. Revise Express is a one-hour video teardown of your business’s funnel. In addition to everything you get with the videos, I’ll run through a 35-point checklist and provide specific, actionable steps for your specific business to begin capturing more customers and generating more revenue.
I’ve sold Revise Express on its own for three years, and now I’m bundling it here to create the maximally valuable package for businesses of any size. This is the easiest way to work with me on a consulting basis, and it’s the only way to get Revise Express from this point forward. Also, remember that Revise Express clients get first crack at Draft Revise slots, before they open to the public.
Your Revise Express report will be recorded based on my availability, and you need to book a confirmed time upon checkout. If you’re working on an impending redesign, I would strongly caution against buying Revise Express before you’re fully, 100%, no-take-backsies ready to launch your site to paying customers. Otherwise, I’ll be forced to analyze your old site.
Here are a couple of people that have enjoyed their Revise Express reports in the past:
[Revise Express] gives us things to try without having to come up with things to try. I’m sure for all of us, that’s going to save a lot of time.
— Gopal Patel, GetFeedback
I love the changes we made to the site. It was worth it to just be able to put the design of the homepage to rest in my mind and have an expert opinion go over all the details. I was blown away by it… 10/10 would buy again.
— Robert Williams, Workshop
Here are Revise Express’s terms & conditions, if you’re curious.
If you order right now, you’ll get your report in approximately 6 weeks. Firm deadlines will be arranged within a day after your purchase.
- Do nothing. Keep losing customers. Deal with the same internecine squabbling. Continue fearing the truth about how your business really works. Piss off your shareholders and business partners. Save $50.
Who I Am
I’m Nick Disabato, a designer & writer from Chicago. I’ve previously worked for great clients like The Wirecutter, New Music USA, Double Your Freelancing, and Chicago Magazine. I look like that drawing to the right, there. You can read a longer bio if you’re curious.
In my three years running Draft Revise for dozens of customers – planning, executing, and analyzing hundreds of A/B tests in the process – I’ve learned everything I can about optimizing your site so it captures more wallet-out customers, in your target market, who pay you and like it.
For example, DNSimple is a DNS provider and domain registrar. In the first month alone, I improved new account signups by 18 percent. In CEO Anthony Eden’s words:
When I heard about Draft Revise, I was excited but skeptical. It turns out that Nick really knows his craft and was able to do significant incremental improvements, constantly testing, validating and improving our conversion rates. I look forward to continuing to work with him to make DNSimple even better.
Nick is seriously a wizard. I can give him a half-evolved idea I want to test, and he interprets it perfectly. He’ll do all the work, analyze the findings, and send us a simplified report.
Thomas Fuchs of Freckle liked my work, too:
We hired Nick to work on A/B testing our marketing pages, but over the course of our work together it evolved into much more than that. Nick was instrumental in redesigning our onboarding flow, tutorials, and exit intents – and revenue and customer happiness have both improved as a result.
And finally, Profit.ly got fantastic results from Draft Revise. In the words of cofounder Mike Mosseri:
Nick helped take Profitly’s conversion rate and user experience to the next level. Through methodical and thoughtful testing, he advanced us while always aligning our business and users’ needs. The outcomes of the tests almost always paid for Nick’s time through the uplift they provided. Draft Revise is an exceptional service and should be considered by all businesses as an additional tool to differentiate themselves.
The A/B Testing Manual is my fourth book. The first three are Cadence & Slang, a very small book about interaction design; Draft Evidence, a series of essays, interviews, and other work from 2011 to 2016; and The Independent Consulting Manual, a resource I co-authored with 12 others about crafting a durable solo practice.
Who Patrick Is
He began three businesses – Bingo Card Creator, Appointment Reminder, and most recently Starfighter – and has become well-known & respected for his expertise in understanding how software businesses can most effectively sell their wares.
His blog posts have been widely shared, read, and acted upon by millions of people, especially on topics like salary negotiation and programming career advice. He’s written extensively about A/B testing, and favorably critiqued Draft Revise’s first marketing page – three days after it first launched in 2013.
Jason Cohen, founder & CEO of WPEngine, writes:
Patrick's advice on starting a drip campaign for WPEngine was an epic win for us – it permanently moved the needle on signups after just a week of work. And it's easy to measure and therefore to improve.
In one of his recent A/B tests, he increased signups to his business by 50% by switching to a multi-stage signup workflow over a single-page signup.
You can read more about Patrick here. He lives with his wife & daughter in Tokyo.
Start Capturing More Customers
The best thing you can do for your business right now is to optimize the way you attract & secure paying customers. If you have enough traffic to get a statistically significant A/B test result every month, you are definitely not doing enough to accomplish this.
Fortunately, The A/B Testing Manual is the most comprehensive and authoritative resource that exists for helping you execute successful tests, take comprehensive action, and profit. Stop designing by speculation – and start acting like the most successful companies in the world today.
Here are some buttons. Please click one of them.
Questions You Might Have
Want a refund? I’m terribly sorry, but all sales of The A/B Testing Manual are final. I provide the advice; it’s up to you to act on it.
Already purchased and want to upgrade? Hooray! You’ll only pay the difference, because we don’t believe you should ever pay twice for the same thing. Email us and we’ll take care of it.
What format is the book? Pure, glorious PDF. No print, no ePub, no Kindle. There are no plans to issue ePub or Kindle versions for the foreseeable future.
Still have other questions? That is reasonable! I would love to hear from you; I’ll get back to you as soon as I can.
Thanks so much for reading, and I hope you have a great day.
PS: Want a book that’s just about design? Check out Cadence & Slang®, my first book. With over 3,500 sold to date, it’s now in its second edition.
PPS: This was long; thank you for reading it. Your reward is this incredible video of dozens of Saint Bernards on an island near Vancouver. LOOK AT THEM. I LOVE THEM.