At Web Optimization Summit 2014, Ryan Hutchings, Director of Marketing, VacationRoost, shared the nuts and bolts behind putting together a foundational testing process.
In today’s MarketingExperiments Blog post, I wanted to walk through Ryan’s five steps you can use to create a small testing program in your organization.
Step #1. Decide what to test
When deciding what to test, the trick, according to Ryan, is prioritization.
There are lots of things to test in a conversion funnel, but limits of time and resources are important to factor in when putting together a test plan.
One of the tools Ryan uses to help his team prioritize smaller testing efforts is a spreadsheet of test ideas from across the organization.
The items highlighted in the screenshot above are columns that list test ideas and their prospective confidence levels that the team thinks will produce a lift.
“This helps us prioritize,” Ryan explained. “It gives us a starting point.”
Step #2. Identify a target conversion goal
Ryan explained that the next step is to identify target conversion goals. To help do that, the VacationRoost team sets ideal ranges for their KPIs.
During his session, he used bounce rates as one example of where KPIs can help you set some target conversion goals and identify some testing opportunities.
“Bounce rate is a good example and a good starting point for a lot of people when talking about individual landing page optimization,” Ryan said.
One additional small mention to add is the disclaimer that the illustration is only an example. When it comes to bounce rates, 37% (represented in the image above) is just to visualize the importance of setting standards, and is not inherently an industry goal.
Step #3. Create a hypothesis
Ryan explained that his team uses the MECLABS, parent company of MarketingExperiments, Conversion Heuristic to help them turn test ideas into testable hypotheses. Using a repeatable methodology helps the team vet testing ideas and keeps testing focused.
“Everything is based on the heuristic, and that’s all we use, Ryan said.
Step #4. Build wireframes, develop the treatments and launch the test
If you’re going use a methodology to help identify testing opportunities, you should also consider how that methodology can help you build a treatment to test against your control.
Ryan explained how the Conversion Heuristic is also used in developing treatment designs to help keep testing centered on the specific variables they want to explore.
One example he shared in his session was a PPC landing page in which VacationRoost wanted to test the impact of quality seals on delivering the value proposition.
“As you can see, these are two totally different pages as you’re looking at it,” Ryan explained, “and when we look at it, we say, ‘OK, what do we want to impact?’”
Step #5. Analyze and calculate ROI
Analyzing and validating your results is an important step to help you discover what works, or perhaps what doesn’t for your customers. In Ryan’s case, the team saw a really nice lift.
However, demonstrating the return on your testing efforts is arguably one of the hardest things for marketers to illustrate to executives, clients or shareholders.
One strategy Ryan’s team uses to help clearly communicate their gains (or losses) from testing is an ROI spreadsheet. He explained that being able to clearly show how the testing efforts impacted the bottom line was key to building a successful testing program.
This leaves one big question: How do you even approach something like this?
According Ryan, one of the best places to start building your ROI model is by becoming deeply tuned into how the relationship between metrics and sales revenue works in your organization.
Ryan also stressed that you have to know how to translate customers moving through the funnel into dollar amounts.
“You have to have a very intimate knowledge with all the metrics and levers that impact sales in your organization,” Ryan said. “If you, as a marketer, don’t know what those numbers and levers are, it’s going to be impossible to build out an ROI model that’s going to be fairly accurate that you feel confident in.”
Putting it all together
Every testing program has to start somewhere.
Taking the time you need to build out the foundation means you will create a process that works better for your testing needs, your company’s bottom line and, ultimately, for the customers you serve.
You may also like
A/B Testing: The value of choice in decision-making [More from the blogs]
Value Proposition: 4 considerations to identify the ideal channel for testing [More from the blogs]