Deciding what to test can be a difficult decision for marketers. Simply setting a test up and running it can be hard enough, but planning a test so that you can actually learn something adds another layer of complexity to the process.
At this year’s Optimization Summit 2012, marketers from HubSpot gained the input of 280 event attendees to determine what to test on a lead generation landing page for the HubSpot All-in-One Marketing Software.
The result was a 6% increase in lead generation rate.
Unfortunately, because of the level of traffic we were expecting for the test, the audience at the Summit had to make a difficult decision. A decision that faces every marketing department that engages in optimization and testing.
Should they run two treatments against the control, and possibly get a deeper learning? Or just run one treatment against the control, and have a higher probability of running a valid test.
The Optimization Summit audience decided to cut the treatment provided by the MarketingExperiments blog audience. Because the conversion rates were so high on the page, there was a significant risk that sample distortion effect would have occurred and the test would not have been valid. To mitigate that risk, the audience decided to run two treatments, the control and the audience-chosen treatment.
However, because we didn’t run the treatment selected by the MarketingExperiments blog audience, we’re giving Sherrie Mersdorf, the marketer who submitted the page, the prize she would have gotten if her treatment had won – a free MarketingExperiments Landing Page Optimization Online Course plus a free copy (both PDF and printed copy) of the MarketingSherpa 2011 Landing Page Optimization Benchmark Report.
Related Resources:
A/B Testing: How a landing page test yielded a 6% increase in leads
Landing Page Optimization: Test ideas for a B2B lead capture page
Lead Capture Optimized: 201% increase in captured leads with clearer value proposition
