The Utah DMC Blog

Principles of Split Testing for Marketers with Chris Dayley - December 16, 2020 [Event Recap + Replay]

Utah DMC Chris Dayley - Dec 16 2020

Presented by Chris Dayley

The December 16th Utah DMC webinar this week featured Chris Dayley. Chris is a digital marketing entrepreneur, speaker, and neuromarketer who gets excited about helping businesses improve website profitability using psychology-based testing and analytics. His agency, Smart CRO, is a full-service A/B testing agency that helps companies see double-digit lifts in conversion rates.

Chris shared his extensive marketing knowledge with us by identifying common split testing mistakes along with a good set of principles to follow when setting up solid A/B tests for your website. Keep reading to catch a recap of the event below!

Utah DMC Darin Berntson - AJ Wilcox - Chris Dayley

Split Testing Principles: Start With a Good Business Question

To begin the event, Chris kicked it off by talking about the significance that business questions can have on split testing outcomes. He spoke about the importance of starting your split testing process by asking a valid business question. He defines a business question as an open-ended question that is something you want to learn about your audience. Another critical principle Chris talked about was that you must start your split testing with a business question. By doing this, he guarantees that you will see good results with A/B testing.

Chris Dayley - Business Questions

Hypotheses

Chris then talked about forming hypotheses and what to look for or ask yourself when split-testing. His takeaways are below:

  • Each variation should isolate one concept or variable (should give a specific answer to the business question)
  • Each hypothesis should give you specific learning- ask yourself if this variation wins, what will I now know about my audience
  • Should always have multiple hypotheses
  • Should include a negative variation (wherever possible) - test the opposite of what you think will work
  • The more extreme the change, the more extreme the result - start with big changes and move your way into nuances once you find something that works

Chris Dayley - Outlinin Test HypothesesAnalysis: Outcomes of a Split Test

There are winners, and then there are losers. But when it comes to split testing, there are three outcomes; winners, losers, and no impact. Winners and losers are both equally valuable because they both provide key insights that we will explain below. However, having no impact is the worst outcome of split testing. Chris breaks down each outcome and how he comes to that conclusion.

Chris Dayley - 3 outcomes of a split test

Winners

When analyzing a successful split test, you want to ask yourself three questions.

  • First: Why do I think this change resonated better with my audience?
  • Second: How large was the impact? If more than 10% run a follow-up test to dig deeper
  • Third: Is there anywhere else I can test/apply this learning to my audience?

Losers

Losers are just as valuable of an outcome as winners are when it comes to split testing. Here are three questions that Chris mentioned to ask yourself when analyzing a losing split test outcome.

  • Why do I think this change did not resonate/upset my audience?
  • How large was the negative impact? (if more than 10% decrease, run a follow-up test to dig deeper)
  • Is there anywhere else I can test/apply this learning to my audience?

No Impact

The last outcome of a split test is no impact, which is the worst possible outcome. Chris defines this outcome as having a 3% impact (greater or lesser) on the total outcome.

Chris Dayley - Test Questions

Conclusion

The main takeaway to remember when split testing is to start with an effective, open-ended business question. By doing this, Chris guarantees that we will see success with any A/B test. Another critical thing to remember is always having a negative variation or the opposite hypothesis of what you think will work. Here are five action items to take to get started conducting successful split tests:

  1. Think of something you want to learn about your audience
  2. Write a business question (open-ended)
  3. Think of 3-5 hypotheses you could test
  4. Build and run your split test
  5. Analyze your results and take action on significant winners or losers

These five action items will guarantee that your split tests take a big step up and help you to see some serious results for your company. If you couldn’t join us for this event, be sure to join us at the next Utah DMC to learn more tips and tricks of the trade from some of the best and brightest in digital marketing.


Be sure to connect with Chris on Twitter, LinkedIn or his website.

If you couldn’t make this event live, you can see the replay below. Towards the end of the video, there was a great Q&A session where Chris answered viewer submitted questions as well. 

To read more of our recaps and professional insight, explore our blog, and be sure to check out our DMC Online Events page to see future and past webinars from our amazing community.


Want More Detail? Enjoy The Replay Below!

Topics: Event Recaps CRO