Multivariate Testing With Google Website Optimizer

Testing variations in your design and copy and measuring which variations lead to higher conversions is an important part of doing business online. A/B split tests are relatively easy to set up on your own, but multivariate testing can be much harder or too expensive for small business owners to implement. Google will make it easier for anyone in their AdWords program by offering their new Website Optimizer tool.

As with all things Google, Website Optimizer is currently in beta. It is only being offered to AdWords customers at the moment and Google will not even guarantee invitations to all who apply. If you’re interested you’ll likely want to sign up soon. I’m sure in time Optimizer will be released to a wider audience, though it may always require an AdWords account.

A/B And Multivariate Testing

With A/B testing you create two distinct pages and direct some people to one page and some people to another. Generally you would only change one part of a page such as a page heading to determine which heading will lead to a greater rate of conversion. The conversion being measured could be signing up for a newsletter, entry into your shopping cart, or simply clicking a link to another page. A/B tests are relatively easy to set up. You create the two pages, changing one thing on the page, and have some tracking in place to measure the actions of those who land on each page.

Where A/B split testing will measure the difference of making a specific change, multivariate testing allows you to test many variations and combinations of variations. Multivariate testing lets you get more fine grained results with your testing. You might test two different headings along with two different ‘buy now’ buttons. Or you might test different page copy and try a different location for an image or a different color for a graphic. Instead of A/B testing it’s more like A/B/C/D/E/F/G testing and can get complex pretty fast. Because of the many variations and combinations involved it can be much more difficult to set up. You could conceivably set up an A/B test for every possible combination of page elements you want to test, but the number of test you would need to run would add up quickly.

Enter Website Optimizer

With Website Optimizer, Google does most of the hard work and you can focus more on what you want to test and measure. With code provided by Google, you break up your landing page into various sections for testing. Each section would then be a variable you can test, such as page headings, images, a section of text, or just about anything you want.

You’ll still want to do a certain amount of preparation in deciding what you should test for and what changes to make, but Google will collect the data and report back to you. You can find an overview of Website Optimizer showing what some of the reports will look like and there’s a nice technical overview of how Optimizer works along with some general information about multivariate testing.

This isn’t meant to impy that all your testing will be easy. You still need to consider what you want to test and you still need to create the different versions for testing. And depending on what you want to measure your test or ‘experiment’ as they are called in Website Optimizer can still be very complicated to run. Google is helping, but you’re still running the test.

What’s In It For Google?

For the moment it’s tied to AdWords so it should help them sell more accounts. Whether or not Website Optimizer will always be tied to AdWords only Google knows, but since you typically set up this kind of testing for landing pages to an ad it’s likely it will stay linked with AdWords. Helping advertisers generate higher ROI on their ads would also be an advantage to advertising with Google over another search engine. Search advertising is getting more competitive so Google may simply be trying to retain their lead in this space.

In addition on the AdSense side more optimized pages and ads could lead to more clicks on those ads. That’s good for publishers and also good for Google. Google has a lot of incentive in helping people get clicks on their ads. Helping us certainly helps them as well.

This could also be the start of Google implementing a pay per action advertising model. Google would need to find if they can accurately track successful actions in order for a pay per action model to work. Google has previously confirmed that they are working on a CPA network as well as testing cost per action in AdSense. Website Optimizer could be a part of a test of that network’s readiness. It’s possible after a few months we’ll see Optimizer open to more site owners followed up soon after with the launch of a CPA ad program.

As with everything Google they like to collect data. The more data they have the happier they are. Website Optimizer will provide them with another level of information on top of what they already collect through search queries, AdWords, Analytics, etc.

Google’s Website Optimizer looks pretty good from what I can see. Multivariate testing is something beyond the means of most small business owners and this will now put it in their hands for free. Free only in the sense that you won’t be charged for the testing, but not so free in that you will need to sign up for AdWords first and you still need to run the tests. There is also the consideration of allowing Google too much access to information about you site. Do you want them to see your traffic and conversion rate and also be able to base how much you pay them per ad click? It’s something to consider. Still for most websites run by individuals and smaller business, Website Optimizer promises to be an easy entry into multivariate testing.

Download a free sample from my book, Design Fundamentals.


  1. This is probably one of the most useful tools google has ever produced for webmasters.
    I was looking through thier demo videoo and it does look very easy to use and looks like it has the potential to increase everyones ROI.

    I havn’t had the chance to use it yet, But i will certainly be giving it a go.

  2. I think it’s going to be pretty useful too and if used well should increase ROI for websites as well as helping Google’s bank account grow a bit larger. Being able to test and measure more accurately is one of the things that sets apart business online and offline. Small businesses aren’t going to have the money to just throw marketing and advertising out there in the hopes that something works. While you can test things offline it doesn’t compare with how much more you can know about your visitors online. Testing if done properly and well can help small business owners to not only optimize their websites, but also their budgets as it’s much easier to find out what is and isn’t working.

    I never did get to watch the video. It wouldn’t load for me yesterday after the first couple of slides for whatever reason. Thanks, though, for reminding me to take another look.

  3. Actually, I was thinking they introduced something to optimize the landing pages for the right keyword placement 😛

    But anyway, that’s just what I was looking for to test how effective my headlines and body copy are. It looks, though, that I’ll have to use a lot of Javascript for the testing, so this may distort the results a little.

    Good job, overall, though.

    It looks natural to use analytics when the site and business is established. You not only have the budget, but the necessary traffic for testing as well.

    P.S. I watched the video and it says:
    – use several different page sections (headling, body, image)
    – track the effectiveness of various page sections
    – track the effectiveness of certain page elements

Leave a Reply

Your email address will not be published.