How To Set Up A/B Split Tests On WordPress (Using Content Experiments)

How We’ve Increased Conversions By 228% Using A Simple A/B Split Test (And You Can Too)

David Hehenberger February 26, 2014

Update – 3/15/14: I’ve included the result of my A/B test at the bottom of this post.

When running A/B split tests, you’re randomly serving two variations of a webpage (A and B) to your visitors to determine which variation makes you more money.

A/B testing makes you money.

A/B testing makes you money.

Still, most businesses don’t test because they don’t know where to start. Keep reading and you’ll learn what to test and how to implement your test in WordPress in under 10 minutes.

“How much traffic do I need to A/B test?”

Running A/B tests is worth it, even if you don’t get tons of traffic. 3000 experiment visits are often sufficient to detect a small change in conversion rates. In the experiment I’m sharing with you in this post, 1000 visits were enough. You can plug your traffic stats into this handy split test duration calculator.

Step 1: Forming a Hypothesis

Every sensible split test starts with a hypothesis. I like the CHANGE : EFFECT framework by Marketing Experiments:

abtesting-forming-a-hypothesis

Some ideas on what to A/B test

Testing your headline

Your headline is the first thing every visitor sees. It should address the main pain your product is solving. This simple headline test increased sales by 89.97%.

Testing your pricing tables & pages

Testing your pricing can be a big win. Try raising your prices by $10, 10% or 20% and see what happens.

The next thing to test on your pricing table is the copy. Are you confusing your users with mentioning dozens of features? Pricing tables exist to make it easy to compare your pricing plans, not to list every single feature and benefit of your product (that’s what your salespage is for).

EPTP-Pricing

Other elements worth testing on your pricing page:

(Shameless plug: if you’re looking for an easy way to create beautiful & highly converting pricing tables for your site, check out my WordPress plugin.)

Removing / adding elements

Try to remove / add elements. Try different types of testimonials or trust symbols. Try adding a FAQ area.

Call to action buttons

If you are offering a free trial, “Start Free Trial” tends to work well. For emails opt-ins, pretty much anything works better than the default “Submit”. For one-time purchases, “Add To Cart” works well.

You might also want to test your call to action color and button size, but these things tend to have less of an impact.

My Hypothesis

My Easy Pricing Tables live demo is the 2nd most visited page on EasyPricingTables.com. My hypothesis: Changing the copy of its footer call to action will get more people click to through to my salespage and thus increase revenue.

Control (Variation A)

variationa

Treatment (Variation B)

variationb

Step 2: Building your variation B

Once you’ve decided on what to test, it’s time to build your variation B. I’ve simply created a new Page in WordPress called “Variation B”, copied & pasted the HTML from Variation A and made the changes.

Depending on what you want to test, tools like Optimizely or Visual Website Optimizer can be useful, but they are overkill if you’re making simple changes to your WordPress site’s copy.

Step 3: Deploying your test using Google Analytics Content Experiments

Once you’ve built and published your variation B, it’s time to deploy your test using Google Analytics Content Experiments. (You’ll have to have Google Analytics installed on your site.)

Step 1: Log into Google Analytics and open your site’s profile.

Step 2: In the left-hand sidebar, go to Behavior -> Experiments. experiments

Step 3: Click “Create Experiment”

Step 4: Enter your URL and click “Start Experimenting” start-experimenting

Step 5: Fill out the “Create a new experiment”-form

objective

Step 6: Enter the URLs of your Original Page and Variation 1, then click next. configure

Step 7: Click Manually insert the code and copy and paste the code.

Then, install the Google Content Experiments WordPress plugin.

If you’re using a theme built on Genesis, Infinity, Thematic or PageLines you’re good to go. Otherwise, you’ll need to add a code snipet to your theme: 1. Within your theme, locate the file where the <head> tag is inserted (probably something like header.php) 2. Place <?php do_action( 'wpe_gce_head' ); ?> right after <head> 3. Save the file

Then, go to the original version of your page and insert your code using the “Google Content Experiments settings” metabox. Save your changes. metabox

Step 8: Click “Next Step” and if the code validation was successful click “Start Experiment”.

start

Step 4: Measuring Results

Google will automatically email you if there is anything important going on with your experiment. Even if it seems like one variation is clearly outperforming the other early one, you should run the experiment in 7 day intervals (7 days, 14 days, 21 days, etc…) to account for different buying behavior on different days of the week.

Update: My Results

Screen Shot 2014-03-15 at 11.09.45 AM

After running the experiment for 14 days, my Variation 1 beat the Original by 228% with 95.8% confidence.

Experiments-Results-Google_Analytics

Again, here’s my winning variation:

winner

I was pretty sure that my Variation 1 (also referred to as “Treatment” or “Variation B” earlier in this post) would win, but I’m surprised I saw a massive 228% lift from a small copy change. This shows how powerful A/B testing can be.

#####

Wrapping up…

Above approach works well for testing entire pages, but not for testing single elements like opt-in widgets. I’m currently working on a WordPress plugin that easily allows you to test your opt-in forms. If you’d like to be notified when this plugin launches simply sign up to my email list.

What about you?

Have you experimented with A/B testing in the past? How did it go? If you haven’t done any A/B testing, why not? Share your thoughts in the comments below…

You may be interested
in these articles