by Kalena Jordan

We all know that the most effective Pay Per Click advertising campaigns use landing pages that are matched perfectly to your target search keywords and designed to follow through with the idea or theme that your PPC ad has hinted at.

But how do you determine the effectiveness of those landing pages? How do you know what design or page features will trigger a better response in your audience and lead to more conversions? The answer is that you don’t, unless you test.

Benefits of Landing Page Testing

Whether they are a part of a PPC campaign or not, there are countless benefits to testing your web site pages, including:

• Improve the effectiveness of landing pages
• Increase conversions / sales
• Attract more leads / sign-ups
• Increase time spent on your site by visitors
• Reduce the Cost Per Acquisition of new customers
• Eliminate guesswork. Improve your site design via information from your site’s end users
• Avoid staff disputes – let your customers decide what design elements should be changed

Google Website Optimizer

The Website Optimizer is a tool that allows marketers and webmasters to test variations of pages on site visitors automatically, to see which pages or variations of pages perform the best (i.e. lead to the most conversions).

In April 2007, Google took their Website Optimizer tool out of BETA and made it available to the general public. I had been wanting to use Google Website Optimizer to test our landing pages on Search Engine College for some time and I finally found the time to trial it in October this year. After what we learned from our experiments, I wish we’d implemented it months ago!

Website Optimizer helps you study the effects of different content on your users and identify what users respond to best so you can alter your web site accordingly. You can test any kind of site elements from individual copy blocks and images to complete page layouts. Perhaps the best thing about Website Optimizer is that you can test ANY page on your site, including landing pages you have designed for other PPC programs like Yahoo or pages designed for non-PPC purposes.

Google Website Optimizer allows you to perform 2 different types of tests:

1) A/B Split Testing
2) Multivariate Testing

You can view a 5 min overview of Website Optimizer here.

A/B Split Testing:

Through the use of code added to the “A” (original) page, Google is able to serve the A/B variations (there can be many more variations than just the “B” page) to site visitors and then provide results of which page was most “successful”, commonly through reporting which of the A/B pages lead traffic to a “results” page.

A/B Testing compares the performance of entirely different versions of a page. Google suggests using it if:

- your page traffic is fairly low (i.e. less than 1,000 page views per week)

- you want to move sections around or change the overall look of the page

In Figure 1, you can see an A/B Testing experiment being set up in Website Optimizer.

Setting Up A/B Experiments in Website Optimizer

To set up an A/B testing experiment in Google Website Optimizer, you first need to prepare three things:

1)    Your “original” web page
2)    Your variation/s of this original
3)    Your conversion page (e.g. the “thank you for subscribing/purchasing” page)

In the example you see in Figure 1, we set up an experiment on SearchEngineCollege.com consisting of our original page (/add-me.shtml) and a single variation (/add-me2.shtml), with our conversion page being /seo-starter-course-sample-download.shtml.

Next, you need to add some javascript to each of these pages to enable Google to track your experiment. Then it’s simply a matter of uploading all your test pages and having Google validate your URLs to confirm you’ve set up your experiment correctly.

Multivariate Testing:

Testing can be made not only with A/B pages, but with different possible versions of a single page.

This allows you to trial different types of layouts and page text to see which combinations lead to the highest conversions on your site.

Multivariate Testing compares the performance of content variations in multiple locations on a page. Google suggests using it if:

• your page traffic is high (i.e. more than 1,000 page views per week)
• you want to try multiple content changes in different parts of the page simultaneously

Setting Up Multivariate Experiments in Website Optimizer

To set up a Multivariate testing experiment in Google Website Optimizer, you need to do the following:

1)    Choose the web page you wish to test.

2)    Decide with your marketing/technical teams which page sections you wish to test e.g. headline, image, call-to-action, copy etc.

3)    Add the JavaScript code to your page’s source code. This includes the Control Script, the Tracking Script and the Page Section Script.

4)    Identify your conversion page and add the Conversion Script to that page’s source code.

5)    Upload your revised test and conversion pages.

6)    Validate your pages. If you’ve set up your experiment correctly, you will see a confirmation message.

7)    Create the code variations for each page section you are testing (see Figure 3).

8)    Review and launch your experiment.

Tracking Your Experiment

Once your experiment is launched, Website Optimizer will serve up your original page and over time, switch it out with page/section variations included in your experiment. During the testing phase, Website Optimizer will display a report showing the progress of the experiment and the number of conversions each page variation has achieved. It will also attempt to estimate the winning page combination based on the number of conversions each page variation achieves.

Once the tool has gathered enough page impressions from your experiment to produce meaningful data, it will display a report. Depending on what type of experiment you ran, there are two kinds of reports: a combination report and a page section report. Each column in the reports provides a different summary of the performance of combinations, page sections and variations.

Combination Report

A Combination Report (see Figure 4) will show the performance results for all of the page combinations made from the page variations you created for your experiment. By seeing how well a particular combination performs in comparison with the original and the other combinations, you can choose the most successful one to improve your business.

The Chance to Beat Original column shows the likelihood, expressed as a probability, that a particular combination will be more successful than your original content. It is very possible that there can be more than one combination which has a good chance to beat the original. When this number goes above 95% or below 5%, the corresponding bar will be all green or all red, respectively. You can see this in Figure 4, where the Optimizer has determined that Page Variation 1 has a 99.5% chance of outperforming our original page configuration and is therefore showing a green bar.

The Estimated Conversion Rate range provides an at-a-glance summary of overall experiment performance. View this column to see how well each combination is performing relative to your original content.

The Observed Improvement column displays the percent improvement over the original combination. Because this percentage is a ratio of the conversion rate of a combination to the conversion rate of the original column, it will often vary widely. Google suggests that you only concentrate on the observed improvement when a large amount of data has been collected and it can be considered more reliable.

Conversions/Visits represents the raw data of how many conversions and page views a particular combination generated.

Page Section Report

While the combination report looks at your content performance as whole combinations, the page section report focuses on which variations of each page section performed best. The tricky part is in choosing variations that work effectively as a winning combination. Why is it tricky? Because page section variations that perform the best in isolation may not work as well in combination with each other.

The Relevance Rating shows how much impact a particular page section has on your experiment. For example, if your headline page section showed a relevance rating of 0, you’d know that the headlines you used did not significantly distinguish themselves. Alternatively, a relevance rating of 5 for your image page section would show that there were one or more images which significantly differentiated themselves from the others, and that the images page section is important for conversions.

The other Page Section Report columns contain similar data to that shown in the Combinations Report, except the figures are only relevant to the variations of a single page section and not to how each variation performs relative to the original variation in that section.

Suggestions for Testing

Whether you conduct your own experiments to test pages for effectiveness, or use testing tools such as Website Optimizer, there are a few things to keep in mind that will help you get the most from your experiments:

1)     Test a page that gets a lot of traffic

There’s no point testing a page that nobody visits. Test a page that brings you a lot of traffic so your experiment has some meaning and the changes have real impact.

2) Test a few things at a time

Test various web site elements at the same time and see how they impact each other. But remember to give your experiment more time and/or more traffic in order to receive meaningful data.

3) Pick a logical conversion goal

For your first test, pick a common conversion goal that will be easy to measure, e.g. purchasing a particular item or signing up for a newsletter.

4) Be brave

Make your experiment obvious! Try a hard-hitting headline versus a blander one. Compare a “buy now” button in a brighter color, try moving a link above the fold to see if it attracts more clicks.

5) Learn from your experiments

Once you have the results of the more obvious experiments, redesign your page/s to incorporate the most popular elements. Then you can try more subtle changes, continually tweaking a page element until you strike a combination that outperforms all others.

6) Take your site traffic into account

For experiments to result in meaningful data, sites with low traffic and/or a large number of variations should run longer experiments in order to obtain a higher percentage of overall traffic.

What Did We Learn?

We were amazed by the results of some of our experiments. We expected our landing pages that incorporated images would convert better than the versions without images. Wrong! Website Optimizer proved that our landing pages with more text and fewer images actually converted much better.

Next we discovered that our landing page which included our navigation menu converted better than one without, despite the persistent belief that landing pages should contain fewer links. Most interesting of all was that while we confirmed that positioning our Call to Action button above the fold resulted in higher conversions, using a red (stop) button converted much better than a green (go) button.

So the lesson here is that assumption is a dangerous thing. You need to test your landing page combinations and not assume that you know what your visitors will respond to.