As marketers and researchers, we regularly deal with the risk and reward cycle. It is what makes our work simultaneously interesting and scary. A recent conversation really got me thinking, and it may hit home for you, as well.
While I was recommending a series of landing page experiences, one of our Research Partners mentioned they were very close to “plan,” and they could not have this test put them behind. I heard genuine fear in the executive’s voice. This brought the risk that we deal with to the forefront, in a very tangible way. Though, in the MECLABS Conversion Sequence heuristic we consider anxiety, this was the first time I heard the fear.
So, how should we deal with this reality of the corporate marketing world? With five straightforward steps for rigorous test preparation.
Step 1. Design a test to impact and measure the KPI that will affect your business’ bottom line
We have many partners who want to test for clickthrough rate on a banner or landing page. This can be a poor KPI since it is an interim step in a sales funnel. We often find an experience with a lower CTR ultimately results in a higher conversion rate.
This is especially true if you are running PPC ads or banners, and are measuring that initial click. These can simply be curiosity clicks.
Step 2. Determine what you and your team feel are the value points
Step 3. Determine the correct number of experiences
There are two major considerations:
- The lift you wish to detect. The smaller the difference, the more data points are required.
- Daily traffic available for the test and the expected/desired percent of lift.
Here at MECLABS, we require a minimum of seven days of data to certify a test – 14 days is better, providing two complete weekly cycles.
Step 4. Design the experiences with enough differences to elicit a significant change
Changing a button color generally will not elicit a significant change in visitor behavior. However, the offer, headline message, page layout or shopping cart experience can make huge changes in overall conversion.
Step 5. Watch the data
To reduce risk, I recommend removing an experience that is losing before it reaches a 95% level of confidence to mitigate any losses. The chance a lagging experience will end up suddenly winning is rather remote.
However, we must reach a confidence level of 95% before recommending a new experience should be adopted as the new control.
With good planning, design and execution of a test, along with careful monitoring of the results, you will consistently achieve better performance of your online efforts. Risk is limited, but the benefit is maximized for your business.
Related Resources:
[Video] Landing Page Optimization: Tips when making a radical redesign decision
Online Testing: 6 test ideas to optimize the value of testimonials on your site
A/B Testing: Split tests are meaningless without the proper sample size
Great tips, I just starting my own internet business and I have been researching, and testing is on the top of the list. So thank you for you information.