Marketing Testing and Optimization: How to begin testing and drive towards triple-digit ROI gains


So you’ve sat through our webinars or taken our online certification courses. You’ve identified what to test but haven’t yet thought about how you are going to test. How do you get started on the path to triple-digit returns?


When it comes to the “how to test” question, what most people usually want to know is “which tool should I use.” But before you delve into testing platforms, you should really ask yourself a few questions:

  • Do I have IT or technical support to build out the new page/path to test?
    • If not, who can I contract to build out the HMTL of the page?
  • Do I have FTP or Apache Subversion (SVN) access to modify the page code so that I can add the testing platform code?
    • If not, where does this fit into IT’s project queue?


Once you’ve documented the logistics to build out the test, now it’s time to choose the testing platform that is best for your business. There are quite a few options out there right now. To help you pick the right tool, I put together the following list with Jessica McGraw, our technical manager of research partnerships…

Google Website Optimizer

The biggest benefit is quite obvious – it’s free! So what do you get for nothing?

Let’s start with support. There is a lot of instructional information about GWO available on the GWO site, including step-by-step directions and FAQs. This makes it easy for just about anyone to implement. And it is fairly straightforward and easy to use with little IT involvement necessary.

Yet it has the capabilities to allow you to conduct everything from simple A/B split tests to more complex multivariate tests (testing different combinations of elements).

Paired with Google Analytics e-commerce tracking, you’re able to report at the transaction and product level.

It can take a few hours for data to begin showing in the reports.

Omniture Test&Target

Test&Target also offers A/B and multivariate testing along with transaction- and product-level reports, but it does so with some really impressive in-depth reporting. For example, you have access to not only aggregate data, but daily data as well. This gives you the ability to spot cyclical changes within your test data and validate tests quicker.

You also gain the flexibility to target content to different groups of people based on defined segments. If you don’t know which segment should get which content, you can find out by setting up a test with different segments and watching the results of each segment individually to determine your new content. Test&Target also includes a feature where offers can be automatically targeted to site visitors based on several different variables including observed visitors’ on-site behavior.

In terms of support, you are not on your own. Your contract likely provides access to an account rep along with support hours to gain some insight into reports and help with complex test setup.

You data will likely appear in reports in mere minutes. And you can integrate reporting into a Site Catalyst account (Omniture’s web analytics product).

And while Omniture is a paid tool, the cost difference compared to a free tool might not be clear cut, because your company may already have a contract with Omniture. If it does, find out the additional cost to use Test&Target.


Unbounce bills itself as having “self-served, hosted landing pages for marketers with A/B testing.” It allows you to not only design the page, but build it and set it up all within the same tool.

This is an easy-to-use service which requires little to no IT involvement. You even get a

WYSIWYG (what you see is what you get) landing page editor, so you can build landing pages with no web designer or without even having any HTML experience.

The tradeoff is that you can only run simple A/B split tests.


If your site uses a content management system (CMS) like Drupal, there is likely a split testing plug-in that is available. Drupal itself is free and open source, and if it is your CMS and you use it for testing, you are assured instant compatibility with no need to pay for extra testing platforms.

As with other open-source solutions, “free” usually comes at a cost. Not surprisingly, Drupal requires a lot of IT involvement. Although, new platform updates are coming that will require less from IT.

On the flip side, since Drupal is open source, you can add any and all reporting features your particular tests require.

Build your own

You can always build your own split testing tool. Well it’s free, right? And by free, I mean that there are no outside costs or need to engage a consultant on product questions.

And you certainly have flexibility to build your own reports. You also have the support of in-house staff with the full knowledge of tool capabilities and ability to add extra features as IT has time.

And there’s the rub…”as IT has time.” You’ll have to gauge exactly what those words mean for your particular organization.


This last section usually doesn’t fall under “how to test” for many marketers, but rather “why I really can’t test even though I’d like to.” In fact, in a poll we conducted in a recent web clinic, time was identified as our audience’s biggest challenge to begin the implementation of the testing-optimization cycle in their organization.

This might seem like an insurmountable challenge up front, (since most marketers feel like they can barely keep up with their deadlines as it is). But as with any excuse (I have no time to exercise, eat right, learn Swahili, etc), it can be overcome by simply planning ahead and getting a firm understanding of the resources you will need.

When building your marketing campaign timelines, keep in mind that depending on what direction/solution you take with testing… you’ll need to build in extra time for:

  • Reviewing your site analytics to identify your biggest opportunity and the best elements to test
  • Working with a designer to modify your current design and incorporate the test variables
  • Fitting into the development queue (if not using Unbounce). In most cases, marketing is the lowest priority in the development world, so ensure that you have your place in the queue before setting an expected launch date

Related Resources

Technology Blind Spots: How human insight revealed a hidden (and almost missed) 31% gain

A/B Split Testing — How to use A/B Split Testing to Increase Conversion Rates, Challenge Assumptions and Solve Problems

Cost of Delay: How to win approval for your test and test schedule

What to test (and how) to increase your ROI today

How Can You Make Your Web Site Smarter?

Are You Listening Fast Enough? Using A/B and Multivariate Testing to Accelerate Conversion Improvements

You might also like
  1. Kim Ann King says

    Hi Gina and Jessica,

    Good article on how to get started in testing. I want MarketingExperiments’ readers to know that there are other testing tools available, such as SiteSpect’s non-intrusive web and mobile optimization platform. Testing tools generally take one of three approaches: 1) no-tag/no script testing apps (like SiteSpect 2) Javascript-based testing apps or 3) Server-side software plug-in apps. The bottom line is readers should choose which approach is right for them and then select the tool based on their needs. Hope that’s helpful!

    Kim Ann King

  2. Gina Townsend says

    Hi Kim,

    Thanks for your comment. I’m not sure if you saw my LinkedIn discussion, but I’m actually in the process of preparing a post that will provide a more in-depth look into each of the above testing tools.

    I’m planning to evaluate other tools that are suggested as well, hence I will review SiteSpect and may email with specific questions. I have your email address and will be in touch.


  3. Rick Perreault says

    Hello Gina and Jessica, thank you very much for the mention! Please feel free to contact me if you have any questions for your in-depth post.


  4. IntelligentMVT says

    I am so glad that you included, “Reviewing your site analytics to identify your biggest opportunity and the best elements to test”. I feel this is the crux of the matter, and is overlooked by many people who advocate testing for the sake of it, or to make more money for themselves or their clients. Also, this is the only post I have seen that encourages building your own testing. It’s a big undertaking, especially if you want to work with customer segmentation, but it’s not impossible – I know because I have done it! You could also quite easily do this with multiple landing pages and custom google analytics tagging. basically there a ton of ways forward. is a great place to start and is completely impartial.

  5. Russell William says

    I’m very glad after reading this article. It is great opportunity for visitors to know how to select testing tools and their implementation. This website has some monetary benefit for this site readers. That’s great job thanks for that sharing.

  6. Han says

    Very useful article Gina! I’m doing this research myself and still didn’t know about the last two. However, I did research on ion’s LiveBall feature. Do you know it? Anything to say about it?

    1. Daniel Burstein says

      Thanks for the comment Han,
      Gina wrote about Liveball here — Online Testing and Optimization Solutions: Quick guide to LiveBall

Leave A Reply

Your email address will not be published.