Online Testing and Optimization Solutions: Quick guide to Google Website Optimizer

I just flew back from the Googleplex and boy is my brain tired. It’s exciting to see the growth of Google Website Optimizer and the online testing and optimization community in general.

Tasked with writing a blog post about my trip, at first, I just wanted to focus on the Googleplex itself (yes, it’s that cool). Then I thought, a fawning post about Google Website Optimizer might be apropos.

But given the number of questions we receive about online testing and optimization tools, I’m going to take a third, more aspirational path. Today I’ll give you some info about Google Website Optimizer, but this will just be the first in a series of posts aimed at helping you answer that “Which testing tool?” question we often receive.

Of course, I can’t answer that question for you per se (the real answer is, “It depends”), but hopefully I can provide enough info over this series of posts to help you make an informed decision.

And here’s my promise to the vendors out there, even if you didn’t invite me to your “-plex” and ply me with delicious locally grown organic food, I still may write about your tool (and if the FTC is reading, Google did, in fact, serve up quite a bounty).

Now all full disclosures aside, here’s a quick look at Google Website Optimizer…

The biggest benefits for Google Website Optimizer are the obvious, it’s free and it’s Google.

But beyond the cost, Google is committed to making quality products and is continuously working on enhancements. Keep in mind that Google doesn’t offer much support, if any, so you’ll need to put in some time researching if you come across any obstacles. Here’s a quick overview…

Key benefits

• Combine with Google Analytics for ecommerce product SKU tracking
• Robust tracking with Google Analytics and Adwords
• Testing a landing page setup is simple – just add a couple lines of JavaScript code and setup the test in Google Website Optimizer ‘s  interface (you will need the page URLs as well as the conversion page)
• Overall reliability of Google’s servers (we’ve never heard of an outage of Google Website Optimizer servers)
• Open application programming interface (API) – you can draw data from the tool into your own environment without having to log into Google Analytics or Google Website Optimizer
• Large base of marketers have had exposure to the tool and, at a minimum, have some sort of expertise on the tool
• Google name brings credibility – May help get buy-in from others that might be unfamiliar with the tool but recognize the Google name
• With multivariate testing, once the JavaScript switching code is in place, you can add and rotate in elements without IT needing to alter the code for each test
• Active product development (constantly rolling out updates and bug fixes)

But keep in mind…

• Google does not deal directly with clients for support – while Google does not offer support, they do have an exclusive group of Certified Partners that are screened by Google and qualified to offer support with any Google Website Optimizer question or issue you may encounter
• Security of data is a concern – Google has access to all data
• Google Website Optimizer only provides aggregated data – you need to add Google Analytics to get reports on daily data
• If you only use Google Website Optimizer, tracking is limited – Google Analytics needs to be added to get tracking beyond visits and conversion metric
• For both Google Website Optimizer and Google Analytics, data is not real time – it can take a few hours (this can elongate quality assurance, or QA, time)
• Have to get creative when working with secondary conversions – i.e. use other tools like Google Analytics to measure secondary conversions/clicks like newsletter sign-up in a sales process
• Cannot add additional metrics collections points within the tool – for example, segmentation of conversion rates, elements that users interact with other than the conversion point (you will need to leverage Google Analytics for this additional tracking)

What types of elements can you test?

• A/B and multivariate testing

How does it validate?

• Reports – chance to beat original, chance to beat all
• Method – Google Website Optimizer uses statistical significance of the difference in sample means among the treatments based upon a Gaussian distribution presumption

Cost

• FREE

Support

• This is important enough to mention again – Google does not deal directly with clients for support, although there are Certified Partners that may be able to help you
• Step-by-step instructions and FAQ’s
• Google Website Optimizer blog
• No contact person (unless you have one through AdWords)
• Forums – lots of great questions answered
• Instructional info available on Google Website Optimizer site

Technology / Development

• For Google Website Optimizer implementation, will need to add JavaScript code – or have Apache Subversion (SVN) access to add JavaScript code
• Google Analytics /Google Website Optimizer moved to AJAX calls – speeds up loading to end user
• A/B testing – has to be hosted on your server (you only have to add hosted page URLs to tool)
• Multivariate testing – has to be hosted on Google’s server

Reporting

• Alone Google Website Optimizer only reports aggregate data
• With Google Analytics it can track daily

We’d love to hear some customer reviews as well. Use the comments to share your experience with Google Website Optimizer. And stay tuned to this blog as we provide quick guides for more online testing solutions to help you choose a platform that is best for your individual situation.

Related Resources

Marketing Testing and Optimization: How to begin testing and drive towards triple-digit ROI gains

MarketingExperiments methodology

Fundamentals of Online Testing training course