Marketing Strategy: 4 steps to developing an effective and strategic test

Twenty four conversations, countless emails and three-and-a-half Red Bulls later, I walked away from my last coaching clinic session at MarketingSherpa Email Summit 2013.

I realized a lot of the questions I was asked were deeper than:

• Which subject line?
• What should I track?
• Long copy or short?

Instead, at the root of many questions was a thirst for advice on strategic thinking and test planning.

A common problem I see among marketers is  a lack of sound strategy behind testing. We are asking the wrong questions. We’re asking, “What headline should we test?” when we need to begin by asking, “Why is the current headline underperforming?” We need to ask “Why?” before “What?”

As an optimization manager, one of my responsibilities is guiding my coworkers through planning an effective test series for our Research Partners. Each time I go through this exercise, there is a thought process I follow, which is what today’s MarketingExperiments blog post will teach.

But before we begin, I want to emphasize the most important point of this post – the reason I follow this process. If you leave this page with nothing else, please remember:

We test solutions to problems, not ideas.

Okay, now we can get down to the nitty-gritty. Let’s talk about the thought process you should follow to craft a strategy-centered test.

 

STEP #1:  STATE YOUR GOAL

Start by asking yourself, “What am I trying to accomplish? What is the objective of this test?”

 

This will lay the foundation for your test, and you will need to continuously refer to your goal. It helps me to write the goal on the whiteboard to remind everyone what we are working towards. This way, we all stay focused.

An example of your goal could be to:

Increase the average order value of customer transactions.

 

STEP #2: IDENTIFY THE PROBLEM – The “Why”

Now that you have identified your goal, it’s time to start thinking about how to achieve that goal. But before you can craft a successful solution, you need to understand the problem. Ask yourself, “Why?”

If your goal is to increase average order value, you may begin by asking:

Why are users only spending an average of $50 on my site?

Once you have asked “Why?” you should then form your hypothesis by formulating possible answers to your question – the “Because.”

 

Users are only spending an average of $50 because:

     A. We place the most emphasis on the product that costs $50.

     B. The only product that is relevant to our customers is the one that costs $50.

     C. They are unaware of the extra benefits of our more expensive products.

As you can see, you can quickly identify multiple possible problems. A trap many people fall into at this point is trying to solve all of these problems at once. This will put you on the fast track to failure with no tangible learnings from your test. Choose one problem to solve at a time.

Sometimes, the biggest problem may not be feasibly solved within your project scope (possibly Problem B). You must choose the problem to solve by weighing the benefits of solving the problem with the costs of testing and implementing the solution.

Let’s move forward with Problem A – We place most emphasis on the $50 product.

STEP #3: IDENTIFY A TESTABLE SOLUTION – The “How”

Once you have identified the problem you will solve, begin brainstorming possible solutions of how to solve that problem.

The issue that I see people run into here is coming up with ideas and not solutions. There is a difference. A solution is a strategy, whereas an idea is a tactical implementation (which we will get to).

For example, solutions to our Problem A (emphasis is primarily on the $50 product) could be:

A. Direct the eye-path more effectively toward more expensive products by use of shape and color.

B. Increase the amount of emphasis placed on more expensive products by changing the layout.

Whereas ideas may be:

A. Place a green arrow pointing at the more expensive product.

B. Use a three-column layout with one wider column dedicated to a more expensive product.

 

See the difference? Solutions can usually be implemented in more than one way, and “ideas” (in the way I’m using the term for this post) typically have less flexibility.

To choose which solution to solve, we typically try to choose something that has the deepest, most transferable learnings. Solution A (shape and color) will most likely tell us “Green or Blue,” which is a very shallow learning. Solution B (layout) may tell us if giving more real estate to a product increases purchases of that product – a somewhat deeper learning that could be leveraged in other situations.

Let’s go with Solution B – Increase the amount of emphasis on more expensive products by changing the layout.

 

STEP 4: TACTICAL IMPLEMENTATION – The “What”

 

Once you have decided which solution you will focus on, you can focus on what you will do to make that solution come to life.

This is where you decide what the actual test treatments will look like – the ideas.

Too often we jump right into the tactical and end up having to back into a strategy and hypothesis. But if you consider those three predecessor steps, your ideas will be related to a solution rooted in a strategy.

So, to summarize how we went through the example:

Goal: Increase the average order value of your customers.

Problem: We place the most emphasis on the product that costs $50.

Solution: Increase the amount of emphasis placed on more expensive products by changing the layout.

Tactical Implementation / Treatments:

While this MarketingExperiments blog post was directly related to testing, this process applies to many decision-making and problem solving situations – how to run a productive meeting, how to convince your boss on your idea, etc. Instead of jumping to a decision, take a step back and consider your goal and possible ways to accomplish the goal. And always remember to test solutions to problems, not ideas. 

Test Interpretation: How over-measuring helped us discover a hidden 198% increase in leads

Marketing Analytics: 20% of marketers lack data

Email Marketing: 3 questions every marketer should ask about negative lifts

Marketing Analytics: 4 techniques to discuss with your data analysts