There’s a myth being propagated across the Internet as we speak …
It’s the myth of “one variable at a time” testing.
And it’s mostly bunk.
Well, at least in the way most marketers think about testing “one variable at a time.” Usually the result of testing one thing at a time is a page that looks like this:
This page was submitted by Jerry Everett, a marketer from our Web clinic audience. When our researchers provided live optimization suggestions, Jerry asked us to review his page in depth. We thought it would be the perfect opportunity to illustrate this point.
While Jerry didn’t necessarily test only one thing at a time to get to this page, he did a redesign based on specific variables in the page design. He was focused on the individual headline, the bullet points, the button, etc.
It looks as though it may have been designed by a committee (even if it wasn’t).
The problem with Jerry’s (and most marketers’) approach to landing page treatment design stems from a misunderstanding of the term “variable.”
Usually, marketers think of variables as single elements on a webpage, e.g., a headline, a button, a layout, etc.
When we view variables as physical elements on a page, we miss the bigger picture.
In reality, variables do not occur on a webpage; they occur in the mind of the customer. When a customer views your landing page, they don’t see particular elements on a page. They see a means to an end.
And, the elements they see to get them to that end are not necessarily related to individual headlines, buttons and layouts.
Part of our jobs as evidence-based marketers is to name the variables in the mind of the customer.
To help us do that, we developed the Conversion Sequence heuristic.
Each of the elements in the MarketingExperiments Conversion Sequence heuristic tends to correspond with variables in the mind of the customer.
When we look at those variables, and adjust them accordingly, we can come up with effective treatments. Here is what our researchers developed when looking at the page with customer-centric variables in mind:
Stay tuned to the MarketingExperiments blog for part 2 of this post where we’ll dig into why we made the changes we did.
In the meantime, help Jerry and his team out. What do you think will work better in the current wireframe? What do you think won’t work at all? Let us know in the comments.