I could have made the title of this blog post, “Subject Line Testing: How we achieved a 48% lift from a split test.”
That is the challenge you face with your subject lines, as well. Should you make a promise? Or should you intrigue your audience? This is a perfect opportunity for you to do a little A/B testing and to see what resonates most … which we did recently for a MarketingExperiments Web clinic invite.
This was a single variable, A/B/C split test, with the only variable being the subject line. One “promise” subject line was tested against two “intrigue” subject lines.
“We created two subject lines that were intrigue based. One was a question, and one was more of an intriguing statement,” said Paul Cheney, Editorial Analyst, MECLABS. “We wanted to narrow down what makes people more intrigued. Is it more of a question or more of a statement?”
Subject Line Treatment A (promise) – How to Effectively Use Color on a Website
Subject Line Treatment B (intrigue) – Are Your Website Colors Hurting Conversion?
Subject Line Treatment C (intrigue) – 5 Horrifying Mistakes Designers Make With Color
The email body remained the same for all three treatments. The only changed was the subject line.
The two “intrigue” subject lines generated more clicks on the call-to-action than the “promise” subject line. Treatment B did not perform well enough to meet our goal of a 95% level of confidence. However, Treatment C generated 47.9% more clicks on the call-to-action than Treatment A, with a level of confidence over 99%. So one thing we learned from this test is that “intrigue” outperformed “promise,” which is consistent with an earlier subject line value prop test.
That earlier test is why the team chose to have two forms of an intriguing subject line — to see which form of intrigue was most successful.
“For the experiment, we ran the intrigue winner to make sure our learnings were still valid,” Paul said.
“What we found out in this case was that the intriguing statement performed better than the question. It may not be due to the question format, it may due to the content of the subject matter, but regardless, it is a compelling research question to keep testing and narrowing down on,” Paul added.
“The next thing we want to learn is whether or not it was the content of the subject line or the format of question versus sentence that generated the additional lift,” Paul concluded.
MarketingSherpa 2013 Email Marketing Benchmark Survey – Receive a free Special Report, CMO Perspectives on Email Deliverability, for completing this survey