Throughout the year, I talk to a lot of people who do A/B and multivariate testing to increase conversion rates. Even in very large companies with well-known brands, it’s not uncommon to hear stories of many tests without a lot of lifts.
Often, when they first started testing, there were easy wins. Now, not so much. These companies have all the tools necessary to test but lack one or more key ingredients necessary for sustained success. In this blog post, I will share one of those key ingredients: testing progression.
What to do after you’ve plucked the low-hanging fruit
Getting great results is not easy to sustain. There are many factors that go into successful test strategies, starting with having a product or service with a compelling value proposition. All the strategy and tactics in the world will not prop up mediocre products and services.
If you do have a great product or service, the right approach to conversion rate optimization will help you realize the full potential of your solutions, especially after you have capitalized on the “low-hanging fruit” leadership is forever hungry for.
The best testing strategies are not a collection of random, disconnected tests. They are also not the product of blindly copying what your competitors do.
Rather, tests build on each other, ideally in a series of logical “if-then“ statements.
For example, the below flowchart shows a series of “if-then” statements that can be filled in based on the cost per acquisition (CPA) test results.
There are two dimensions to the sequence:
- Where to test in the customer journey
- The number of elements you test in a test sequence
How to determine where to test in the customer journey
Where in the customer journey you start to experiment will depend on what’s wrong with the experience.
Tests earlier in the customer journey have the advantage of generally being easier to do.
On the other hand, increases in conversion earlier in the funnel do not always translate into downstream conversions. For example, if you are promising people a free trip to Paris, you’ll get many clicks until they find out they have to buy a million-dollar house to qualify.
Tests at the end of the journey often require more IT/development resources. However, if you believe your company’s customer experience is costing you conversions, there are four reasons to start there:
- Those are the most motivated customers
- Small increases in conversion can often make a big difference from a financial standpoint
- You can transfer some of the insights to the earlier stages of the funnel and possibly to other purchase processes or lead forms
- You might also discover segmentation strategies from those who are not converting that will help you win higher conversions with different groups of visitors
How to decide the number of elements to test
The second dimension of creating a testing sequence involves the number of elements to test.
When there is sufficient traffic to warrant more complex test sequences, you should consider a progression that goes from many elements in your test design to specific elements. By “many elements,” I mean a radical redesign.
Start with a radical redesign …
As opposed to single-factor tests that pit, say, one headline against another (and we’ll get to those in a minute — they do have a place in your testing strategy), radical redesigns involve significant changes between the treatment and the control — headline, body copy, images, call-to-action … you name it.
For this approach to work, it is important to not just make random, radical changes. Each change you make should support a hypothesis that leads to further understanding of the customer.
This approach allows Marketing to think outside the box and go for significant increases in conversion. Once you have a win, you can isolate the elements within the treatment that are helping and those that need improvement, gradually growing more and more specific in terms of the insights you glean.
… then test variable clusters …
Once you’re identified a key discovery about customer behavior from the radical redesign, it’s time to get more granular with your testing.
From a radical redesign you would then test variable clusters from the winning treatment.
For example, a call-to-action (CTA) button has a cluster of variables that include the:
- shape
- size
- color
- message
Testing different versions of a CTA button is a simple example of a variable cluster test.
Many marketers go wrong by testing unrelated elements. For example, testing the number of types of fields and the CTA button.
Such tests make it hard to know what impact the fields had relative to the CTA button. In fact, it may be that the button drives an improvement but the changes to the fields decrease conversion, making you wrongly include that both elements played a role in the lack of improvement.
While multivariable tests like this may be appropriate at the variable cluster stage (as long as all changes that you’re testing are tightly aligned around a single hypothesis), at this stage in the testing you want to focus on single variable clusters to test to understand how best to express (though messaging and images) the greater lessons you learned about the customer from the radical redesign tests.
… finally test single factors
From variable clusters, if you have sufficient traffic, you can move into single factor tests. Single-factor tests allow you to isolate the specific variables that motivate your customers to say “yes.”
In the CTA example, you may want to test just color or just shape or just text in a single variable test, assuming you have sufficient traffic volume and the cost of testing has a reasonable chance at a payback.
Testing your way from insight to action
If you follow the process I’ve outlined, you are going from the highly generalized to the very specific, gaining wisdom about what really appeals to your customer (or a segment of your customers). From there, you can begin applying your new-found wisdom to other parts of your funnel, other areas of your website, other online channels — and even to your offline channels.
These two approaches — deciding where in the customer journey to test and moving from testing many elements to discrete elements — are fundamental building blocks to more sophisticated testing strategies.
By looking at a group of tests as a whole, you’ll learn more about your customers and give yourself the best chance of maximizing conversion.
Related Resources
MarketingSherpa Summit 2016 — At the Bellagio in Las Vegas, February 22-24
Online Testing Course: Learn a proven methodology for executing effective and valid experiments for increased marketing ROI (from MECLABS Institute, MarketingExperiments’ parent research organization)
Negative Lifts: How a 24% loss produced a 141% increase in conversion
Lead Gen: 17% lift in lead capture by including more details in email [Email Summit 2014 live test]
