<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=763991110377089&amp;ev=PageView&amp;noscript=1">

Split Testing - Knowing is Half the Battle, Part 1

Aug 13, 2010 / by Josh Summerhays

Footloose stops a nose a bleedI get energized when clients and my SEO colleagues catch the vision of split testing changes to a website. The potential for conversion rate increases is huge, and it’s also great fun!

It’s not all rainbows and fluffy bunnies, however. There are challenges that can sabotage otherwise promising split testing efforts. Awareness of these challenges will help you avoid some of the common pitfalls.

Here's Part 1 in a four-part series:

 

Your website needs robust traffic to support a reasonable testing cycle.

If your website is brand new, it’s likely that you don’t have enough traffic to make split testing worthwhile. The more time it takes to complete a split test, the less valid your test result is likely to be because external factors can influence results.

These external factors are things like shifts in market demand (seasonality is a common example of this) and external media factors.

Want to know how long your test might take to reach a valid conclusion? Here’s how to figure it out:

  1. Figure out how many test combinations you want to run. If you want to test one variation, that’s two combinations because you always test against the current version.
  2. Calculate how much traffic per day passes through the test page. More traffic means the test will take less time, everything else being equal.
  3. What percentage of traffic do want to include in the test? Exposing only a fraction of your traffic to the test will reduce the risk of your test negatively impacting revenue, but also increases the amount of time needed for your test.
  4. What is the current conversion rate of this page? Whatever your success metric is, you need to know the performance benchmark you’re trying to beat.
  5. What is your expected rate of improvement? The improvement needs to be high enough to have a measurable impact on your business. If a 10% improvement doesn’t offset the cost of executing the test, you need to aim higher.

Google provides a free test duration calculator that does the complex math for you. Just punch in the numbers to see how many days your test will likely have to run. If you need to decrease the duration, you can choose to run fewer test combinations or increase the percentage of traffic included in the test.

Topics: google website optimizer Conversion Rates Blog cro split testing Conversion Optimization Conversion

Josh Summerhays

Written by Josh Summerhays

Josh has been involved with search and online marketing since 2006. He regards conversion optimization as his 1b to a day with the family at Disneyland. Aside from his role as a conversion optimizer, Josh has extensive experience in PPC marketing. He currently lives in Eagle Mountain, Utah with his wife and two children.

Let us improve your online marketing results

We have increased traffic, leads, and sales for well-known companies—including Dell, Mrs Fields Cookies, Hotels.com and H&R Block.

Plus for hundreds of local smaller companies like dentists, plumbers, dermatologists, etc.

Find out how to work with us  

Subscribe to Email Updates

Lists by Topic

see all

Posts by Topic

see all