Presented by Chris Dayley
The December 16th Utah DMC webinar this week featured Chris Dayley. Chris is a digital marketing entrepreneur, speaker, and neuromarketer who gets excited about helping businesses improve website profitability using psychology-based testing and analytics. His agency, Smart CRO, is a full-service A/B testing agency that helps companies see double-digit lifts in conversion rates.
Chris shared his extensive marketing knowledge with us by identifying common split testing mistakes along with a good set of principles to follow when setting up solid A/B tests for your website. Keep reading to catch a recap of the event below!
Split Testing Principles: Start With a Good Business Question
To begin the event, Chris kicked it off by talking about the significance that business questions can have on split testing outcomes. He spoke about the importance of starting your split testing process by asking a valid business question. He defines a business question as an open-ended question that is something you want to learn about your audience. Another critical principle Chris talked about was that you must start your split testing with a business question. By doing this, he guarantees that you will see good results with A/B testing.
Chris then talked about forming hypotheses and what to look for or ask yourself when split-testing. His takeaways are below:
- Each variation should isolate one concept or variable (should give a specific answer to the business question)
- Each hypothesis should give you specific learning- ask yourself if this variation wins, what will I now know about my audience
- Should always have multiple hypotheses
- Should include a negative variation (wherever possible) - test the opposite of what you think will work
- The more extreme the change, the more extreme the result - start with big changes and move your way into nuances once you find something that works
Analysis: Outcomes of a Split Test
There are winners, and then there are losers. But when it comes to split testing, there are three outcomes; winners, losers, and no impact. Winners and losers are both equally valuable because they both provide key insights that we will explain below. However, having no impact is the worst outcome of split testing. Chris breaks down each outcome and how he comes to that conclusion.
When analyzing a successful split test, you want to ask yourself three questions.
- First: Why do I think this change resonated better with my audience?
- Second: How large was the impact? If more than 10% run a follow-up test to dig deeper
- Third: Is there anywhere else I can test/apply this learning to my audience?
Losers are just as valuable of an outcome as winners are when it comes to split testing. Here are three questions that Chris mentioned to ask yourself when analyzing a losing split test outcome.
- Why do I think this change did not resonate/upset my audience?
- How large was the negative impact? (if more than 10% decrease, run a follow-up test to dig deeper)
- Is there anywhere else I can test/apply this learning to my audience?
The last outcome of a split test is no impact, which is the worst possible outcome. Chris defines this outcome as having a 3% impact (greater or lesser) on the total outcome.
The main takeaway to remember when split testing is to start with an effective, open-ended business question. By doing this, Chris guarantees that we will see success with any A/B test. Another critical thing to remember is always having a negative variation or the opposite hypothesis of what you think will work. Here are five action items to take to get started conducting successful split tests:
- Think of something you want to learn about your audience
- Write a business question (open-ended)
- Think of 3-5 hypotheses you could test
- Build and run your split test
- Analyze your results and take action on significant winners or losers
These five action items will guarantee that your split tests take a big step up and help you to see some serious results for your company. If you couldn’t join us for this event, be sure to join us at the next Utah DMC to learn more tips and tricks of the trade from some of the best and brightest in digital marketing.
If you couldn’t make this event live, you can see the replay below. Towards the end of the video, there was a great Q&A session where Chris answered viewer submitted questions as well.