Marketing Testing and Optimization: The value of being wrong

“I didn’t fail. I just found ten thousand ways that didn’t work.” – Thomas Edison

The scientific process is a funny thing. Sometimes the biggest discoveries come from being wrong. Or unintentionally.

For example, Susan Freinkel, author of “Plastic: A Toxic Love Story,” explained how scientists first discovered that plastic was not inert…

But we’ve known since the early ’70s that DEHP leaches out of vinyl, and the way that we know is that there were a pair of scientists at that time who were doing some experiments with rat livers. It doesn’t really matter what they were trying to do.

But they kept finding this weird, strange compound that was fouling up their experiments, and when they set out to figure out what it was, they discovered it was DEHP. And they were very surprised because everybody had assumed that this is, you know, an inert material.

So how do you make sure that you gain value from each test you run, even when the changes you work hard on making lose to your original? How do you ensure that a loss is, as I like to call it, a “negative lift.”

Join us for our next Web clinic (educational funding provided by HubSpot) this Wednesday at 4 p.m. EDT when our Managing Director, Flint McGlaughlin, will share a loss from a test we ran…and how we ultimately used the learnings from that test to drive an increase in conversion – Negative Lifts: How we turned a 25% loss into a 141% increase in conversion.

But before we share our process, we wanted to hear how your marketing peers handle a loss. What happens when you run a test and don’t get a lift? How do you use a loss to improve your marketing? Here are a few of our favorite responses…

Constantly re-try experiments

We may be less scientific in our approach than Marketing Experiments, but we feel it’s always very important to figure out why a loss occurred.

Typically when we see a no-impact or negative-impact result, it’s not because the marketing channel was poor or the concept was wrong. It’s more a matter of execution and getting the details right. Perhaps the copywriting was not quite right. Perhaps the call to action was misleading. Or maybe we sent out an email blast at the wrong time.

What’s important for us is to not give up on a strategy after just one attempt. We constantly re-try experiments several times before we put them to rest.

– Sean McVey, digital marketing consultant, Hinge

Anyone who does direct marketing fails

If you don’t you have either not been at it long enough, or you just aren’t measuring what you are doing.

It is sort of like falling off a motorcycle…eventually it will happen.

I would say once we understand why we failed, don’t do it ever again.

That being said, people are fickle. What worked for me 20 years ago, did not work for me 10 years ago, but if I modify it slightly and change the medium, it works today. What works in the Midwest of the U.S. does not work in New York and what works in New York, sometimes might not work in Los Angeles.

You need to understand why something failed, modify to fix the perceived problem, and reTEST.

That is the fun thing about this business. It always changes. And market segmentation, demographics and offer can be a winner in one location and an utter failure somewhere else.

We only truly fail when we do not learn from our mistakes.

– Ben Baker, President, CMYK Solutions Inc.

Follow these steps

There can be so many things you can learn from a campaign irrespective of the success or failure. For me, I always try to see what information we have collected and what might have went wrong. Most of the time taking a step back and detaching yourself from the campaign makes you think differently and you get a different perspective of your total campaign.

I would follow these steps to see what we have got…

• Were we able to reach our target audience with our marketing message?
• Was my marketing message was rightly taken or rightly conveyed? Famous example –  Volkswagen named the sedan version of Golf the Jetta. However, the letter “J” doesn’t exist in the Italian alphabet, so Jetta is pronounced “Ietta”, which means Misfortune…”
• Did you put enough focus on the main message? Sometimes you put so much attention on small things that your audience misses the main thing.
• Did we read the research data correctly? Many times you build your campaign on correlation data which is not actual data.

And the idea should be to evaluate and re-evaluate the data that you have collected. And when you feel confident that you have the answer to the failure….then begin the campaign with all the possible points which you now see but may have missed the first time around

And last but not least…TEST, TEST and TEST. The best way to do this is to run your campaign on a small scale to see the initial impact.

Sourav Sharma, Sr. Manager Internet Marketing & Social Media, SAM S.A.

Related Resources:

Negative Lifts: How we turned a 25% loss into a 141% increase in conversion – Wednesday, August 3^rd, 4 p.m. EDT (Educational funding provided by HubSpot)

Online Testing: 3 takeaways to get the most out of your results

Marketing Optimization: How to determine the proper sample size

Evidence-based Marketing: Why you need more than just numbers to truly drive ROI