Marketer’s Intuition

We recently released the audio recording of our clinic on this topic. You can listen to a recording of this clinic here:

Marketing Intuition

There is no doubt that intuition and moments of great insight can have a profound impact on your marketing efforts.

We all know of business leaders who have achieved extraordinary success for their companies by following their instincts and capitalizing on moments of great insight.

However, when we get down to the details of how best to communicate our value proposition in words and design, intuition appears to be a great deal less reliable.

• Can your intuition and experience tell you which advertisement headline will result in the most click-throughs?
• Can intuition tell you which email subject line will result in the highest open rates?
• Can intuition tell you which web page design will give you the best ROI?

In this brief we will walk you through some of our own test results, and also share the results of your own intuition when it comes to identifying the best-performing copy and design.

Test 1: Multivariate Testing

In our recent clinic on multivariate testing, we looked at JoAnn.com, a website serving millions of arts and crafts enthusiasts. The company used Offermatica to set up multivariate tests in a number of site areas. After one round of testing, they registered the following improvements:

What You Need To UNDERSTAND: This company experienced significant growth due to its multivariate testing, including an overall revenue per visitor improvement of 209%.

But the most memorable lesson was this: the offer that the marketing team thought would be the least viable (“buy two sewing machines and save 10 percent”) actually generated the highest return. “People were pulling their friends together and we sold enough … machines to outperform single purchases,” said Linsly Donnelly, JoAnn.com’s chief operating officer.

Many of us, as marketers, would have thought, “Who needs two sewing machines?” But the close-knit nature of the crafts community proved to create significant increased returns through word of mouth.

KEY POINT: Even the smartest marketers are often proven wrong by testing. Intuition is no substitute for well-designed experiments.

Test 2: Offer Page

In an optimization effort for one of our test sites, we believed that customers visiting the site with an 800×600 or 1024×768 resolution monitor were not finding the relevant sales language for the primary site product unless they scrolled down that page.

We set up an A/B/C split to test this hypothesis:

• Page A was the original page.
• Page B featured slightly shortened data and used a “click here” anchor text to take visitors down the page. This page showed the order process on a 1024×768 resolution monitor and on an 800×600 resolution monitor it displayed the offer copy for the primary product.
• Page C was a radical redesign in which the order process was partially viewable on 800×600 and higher. It used two columns to make more information available “above the fold.”

Here were the actual pages tested:

• Page A:
  http://www.meclabs.com/resources/PageA.gif
• Page B:
  http://www.meclabs.com/resources/PageB.gif
• Pace C:
  http://www.meclabs.com/resources/PageC.gif

Before revealing our test results, we polled the audience of one of our web conferences. We asked them to vote for which page they thought would perform the best. Here is a breakdown of their votes:

What You Need To UNDERSTAND: More than half of our live audience predicted that page C would perform the best.

In addition, many of the members of our team believed that Page C would perform the best. Our experts and many of the seasoned marketers on the phone all seemed to believe that Page C would out-pull the other two pages.

But here are the results of our testing:

What You Need To UNDERSTAND: Page B outperformed the original page by 15.57%. Page C was a dismal failure.

KEY POINT:What seems “intuitive” to most marketers is not always revealed as the best page after testing.

In this test, our hypothesis about important sales language appearing higher on the page proved correct. However, the two-column approach of Page C was ineffective.

Test 3: PPC Headline Copy

In our next test, we tested four PPC headlines on Google AdWords. We polled an audience of 50,000+ marketing professionals, asking them to select which headline they thought would generate the highest click-through rate (CTR).

These ads were tested on the following selection of keywords:

• B.C.
• Foxtrot
• Ziggy
• boondocks comic
• calvin & hobbes
• calvin hobbes
• cartoon comic
• comic art
• comic page
• comic store
• comic strip
• comics
• doonesbury
• free comic
• funny comic
• garfield
• newspaper comic
• online comic
• sunday comic strip
• web comic
• webcomic
• yahoo comic
• “Andy Capp”
• “BC Comic”
• “Dog eat Doug”
• “For better or for worse”
• “bloom county”

Here are the results of the survey:

What You Need To UNDERSTAND: The majority of our audience chose the second headline, “DailyComics Comics,” as the most likely to achieve the highest click-through rate.

Here are the actual test results:

What You Need To UNDERSTAND: The second headline, “DailyComics Comics,” actually did perform somewhat better than the others.

In this case, our marketing audience was able to predict the headline with the greatest success.

KEY POINT: Intuition proved sufficient to select the most DESCRIPTIVE headline, which performed the best out of our selected group.

Test 4: Email Subject Line Copy

What You Need To UNDERSTAND: The majority of marketers who responded to our poll predicted that the third subject line would perform the best.

What You Need To UNDERSTAND: The second subject line generated the greatest Open Rate, while the first generated the highest click-through rate.

Interestingly, our survey respondents favored the worst performing subject line. Why did almost 50% of respondents choose the worst line? Probably because it appears to be the “strongest” line, pushing and challenging the reader. However, the data demonstrates that the less pushy, more passive lines generated higher open rates and click-through rates.

This is an excellent example of a result that is counter-intuitive. Hence the need for rigorous testing.

The following guidelines will be helpful in determining the best applications of marketing intuition:

1. Intuition can be effective in determining what needs to be tested. It can provide the insights you need to decide which general direction to explore.
2. Intuition may fail you when you may attempt to predict, which of those tests may prove successful. Depending on intuition to select one of four possible email subject lines, for instance, it not a reliable way to maximize your chances of achieving the best open rate.
3. Intuition can be used, sometimes, to dig deeper down in to your test results. Sometimes it can help you glean an insight as to why the market behaved in a particular way.
4. Intuition typically fails miserably when applied to initial test results. In the beginning stages of the tests the conclusions may be seriously flawed. You have to wait until you have a large enough sample. The danger is particularly real when provisional results happen to match your own intuitive opinion.
5. A mistake many marketers make is to take the initial findings of the test, make an intuitive determination as to which is best, and stop the experiment. More often than not, this leads to a bad decision.
6. Intuition is often limited, in that it depends on previous experiences. It depends on pattern recognition. Because of this you may make natural assumptions that are contrary to absolute changes in the context, or to changing conditions in the marketplace.

Intuition and testing are not mutually exclusive. Far from it. They go together. Being dependent on intuition alone leaves you flying blind, crossing your fingers and hoping for the best. At the same time, testing without being open to intuitive leaps tends to constrict breakthrough thinking.

Here is an example.

Bridges.com publishes newsletters for career counselors and students. Most of their newsletters are fairly conservative. Then one day a new and very young employee suggested writing a newsletter to the 18-24 age group in the voice of an alien from outer space. They even had a cartoon image of the alien as part of the masthead. This newsletter grew very rapidly and became hugely successful for them.

The idea came from an intuitive leap. Testing of the regular newsletter lineup would never have yielded the idea of developing a career counseling newsletter written in the voice of an alien.

However, from the first day, they tested elements of the alien newsletter, first against a very small group of subscribers.

Both intuition and testing have their place. It’s a matter of embracing both and knowing the right time to apply them.

As Henri Poincaré stated: “It is through science that we prove, but through intuition that we discover.

RELATED MEC REPORTS:

• PPC Ad Copy Tested
• Avoiding Unprofitable PPC Campaigns
• 90-Day PPC Plan
• Landing Pages Tested
• Order Forms Tested
• Conversion Rate Optimization
• A/B Split Testing
• Price Testing
• Subscription Revenue Tested
• Offer Pricing Tested
• Abandoned Shopping Cart Recovery
• Shipping Charges Tested
• Customer Ratings
• Website Copy

Credits:

Editor — Flint McGlaughlin

Writers — Brian Alt
Nick Usborne

Contributors — Jalali Hartman
Jimmy Ellis

HTML Designer — Cliff Rainer