We all want to know how to improve the number of conversions we get on our websites. A lot of the time, we try to do that by rearranging the layout or switch up page elements. Then we go to our dashboards, see if the number went up or down. You’ve probably spotted the flaw in this. This is circular reasoning, a change in customer behavior can’t be proven by “looking back at data, trying to decide whether or not it was some sort of change that we made” As conversion expert, Matt Gershoff puts it. What’s the right way to do this? It’s simple, reverse the order of events and start off with your hunch about what you should change, run an experiment on your customers, proving or disproving if the thing you believe causes their behaviour actually has that effect.
This discipline is called CRO, which stands for Conversion Rate Optimization. There’s someone who’s superbly qualified to talk about this and I’m lucky to have known her for the past few years.
Deborah unknowingly ran her first optimization study in school at the age of 8, when she put her classmates through a science experiment where they looked at pieces of construction paper tacked on a bristol board. Little Deborah grew up to earn a master’s of science degree, specializing in eye tracking technology. Today, Deborah applies her specialized skillset to Conversion Rate Optimization (CRO). She founded a well known resource website where she has published hundreds of client A/B test case studies. She also has a certificate in graphic design, giving her the blend of left and right-brain thinking that’s just right for working in CRO.
Some things to listen for:
- Marketers who run paid search & paid social will want to listen to what she has to say about mixing traffic from various channels together.
- She explains what minimum traffic constraints we’re under for A/B tests, and why with small volumes we’re better to put 2-pages into a test where only 1 thing has changed, versus testing multiple pages or multiple variables at the same time
- She has some tips on how we can maintain objectivity as we run our tests and as we present the results to our leadership.
People/Products/Concepts Mentioned in Show
Deborah’s Five Step Process:
1. Practice Know Your Audience (KYA) and research the performance of pages via site analytics and heatmaps like CrazyEgg and HotJar.
2. Form a SMART experiment hypothesis. Here is a single-sentence version: “Because I observed and received feedback on [what is causing users to convert at X rate], I believe that [the change to be tested] For [targeted segment or all users] will result in [specific lift in conversion rate].”
Ensure you have sufficient traffic / time for the experiment, using one of these calculators:
3. Pick a Tool to run the experiment. Here are some common ones:
4. Run the experiment. Here’s a screenshot from one run using Google’s free tool
5. Implement the winning page and monitor for expected results.
How A/B pages are commonly named: “A” is your Control (original version) and “B’ is your Variant (includes change you’re testing). Here is a more in-depth Conversion Rate Optimization glossary
Note the different frameworks that tools use: Frequentist vs Bayesian frameworks. Depending on the statistical framework used, the timeframe needed to give results will change.
Connect with Deborah on LinkedIn and Twitter
Deborah’s websites include GuessTheTest.com and Convert Experts
Episode Reboot
Run an A/A Test, here is Deborah’s article on why you should and what to consider.