Email Marketing: Learn from 3 A/B test results to set a firm foundation for your next campaign


When I was a teenager, I once had to help my father build a large concrete slab in front of our home.

For me, it was nerve-wracking as I spent hours bearing the brunt of Florida’s merciless summer heat amidst measuring, cutting, framing and leveling a patch of yard that would probably only be used as an occasional place to stash our family’s boat overnight.

I was excited when we were almost finished with the project, when the real fun arrived in the form of a large metal meshed roll of rebar that had to be set in place before a single drop of concrete was poured.

This process would likely take another day to finish and would guarantee my weekend away from school was spent.

So, when I asked my father why we were adding the rebar when we really didn’t need it in my view, he explained the bigger picture with one simple phrase …

“Whatever good things we build end up building us, too.”

I didn’t really appreciate what that meant at the time, but after almost 20 years, a few boats and a lot of cars parked out there … the old grey slab is still going strong.

For me, it’s a reminder that taking a relatively small amount of additional time to build something optimal for your needs instead of settling for adequate can result in successful, long-term ROI gains.

I mention all of this because last week, MarketingExperiments’ sister brand, MarketingSherpa, opened its call for speakers for the MarketingSherpa Email Summit 2014 at the Aria in Las Vegas in the form of Email Awards 2014 (there is no entry fee, and winners are invited to speak at Email Summit). The deadline for proposals is September 8, 2013.

The focus for this year’s Summit is centered on four pillars of email marketing:

  • Build & Cleanse
  • Create & Design
  • Deliver & Automate
  • Connect & Integrate

In today’s MarketingExperiments Blog post, I wanted to share three highly successful email tests featured in MarketingExperiments over the past year or less you can use to aid your testing and strengthen your own email marketing.


Email Test #1. How the right tone in message copy increased lead inquiry 349%

Background: ActiveNetwork is a large event management software provider.

Goal: To increase the amount of free debt consolidation sign-ups without additional traffic.

Primary Research Question: Which email tone will result in a higher rate of lead inquiries?

Approach: A/B single factor split


Treatment #1



For the first treatment, our research analysts hypothesized the email recipients were not converting because the tone of the email’s message was not engaging enough, and so ActiveNetwork changed the tone to take a more direct sales approach.



Treatment #2



For the second treatment, our analysts hypothesized email recipients were also not converting because of high anxiety due to perceived phone spam and high-pressure sales language.






What you need to know  

By addressing customer anxiety through the tone of the email copy, the marketers at ActiveNetwork saw a 349% increase in total lead inquiries.

To learn more about this email test, you can watch the full free “Email Copywriting Clinic,” to see Flint McGlaughlin, Managing Director, MECLABS, share a patented methodology you can use to craft email copy quickly and effectively.


Email Test #2. How email relevance increased revenue 318%   

Background: An e-retailer catering primarily to teen shoppers, its house email list topped 300,000 addresses, yet marketers felt they were not maximizing the list’s potential.

Goal: To increase email revenue by increasing the relevance of their emails.

Primary Research Question: Can we increase email revenue by increasing the relevance of emails?

Approach: Sequential, multifactorial experiment




The research team noted the standard email capture was located on the homepage and the left-side navigation bar appearing throughout the site.

Although this was prominent placement allowing the company to capture thousands of email addresses, the response from customers to new product emails was less than optimal.






For the treatment, researchers placed an email alert button specific to the products added to the site.







By relating the email capture offer to the specific product and timing expectations of potential subscribers, this company was able to increase revenue by 318% and conversions by 53.3%.


What you need to know 

Adding the email alert to the product pages helped the marketing team…

  • Segment the list to include only customers interested in the specific product
  • Relate the email capture directly to the content on the current page
  • Deliver only product-specific emails to this list to improve clickthrough rate and conversion rate
  • Create the expectation that customer would receive an email for timing new arrivals related to their selected brands.

To learn more about this email test, you can watch the full, free Web clinic, “When Should You Send an Email,” to learn more from Flint about five factors you can use to determine the optimal time to send email to your customers.


Email Test #3. How determining the most effective point of value increased clickthrough rate 17%

Background: This company provides technology and product supply to the oil and gas industry. For this experiment, it was making a specific segment (drilling engineers) of its opt-in list aware of an upcoming conference.

Goal: To determine the most effective point of value.

Primary Research Question: Which value category (“overcoming challenges” or “generating results”) will generate the most response?

Approach: A/B single-factor split


Version A  



For version A, researchers crafted an email that focused on overcoming challenges.




Version B



For version B, researchers crafted an email that focused on generating response.







What you need to know

The email message focused on overcoming challenges outperformed the email focused on results, leading the researchers to conclude for this segment, there is more value to customers in obtaining the solution to problems.

To learn more about this email test, you can watch the full Web clinic, “How You Can Use Email to Discover the Essence of Your Value Proposition (in 5 Simple Steps),” to see Austin McCraw, Senior Editorial Analyst, MECLABS, present a simple five-step method for using email campaigns as a testing ground to quickly discover what customers find most captivating about your offers.


Related Resources:

[Video] Email Marketing: Use website content to increase list size

Email Optimization: A single word change results in a 90% lift in sign-ups

Copywriting: How long (or short) should your copy be?

You might also like
  1. Julius Minor says

    Great post!.. You always have to test and tweak until you find that winning campaign.. Even then should still test and tweak.

  2. Abdul Rafay says

    Marvelous Post 🙂

Leave A Reply

Your email address will not be published.