Test Your Marketing Intuition: Pier 1 Imports email design

38

How much will companies spend on email marketing this year? According to Forrester, that number is well over one billion dollars. And still email designs are being sent out without any clue as to how well they perform. It is not uncommon to see the “most beautiful” email messages that follow all the “best-practice” guidelines and have a committee of “design experts” backing them underperform – as if spending more than a billion dollars wasn’t enough!

So we want to see if you can tell the difference. We ran an experiment with three top-of-the-line agency-designed email messages. We want to know if you can spot the email design that performed best. (A prize for all the winners this time)

Background: This email test ran for Pier 1 Imports, which I will assume most of you know is a large B2C company selling home products. This email in particular was a seasonal promotion going to a segment of their house list. There are three agency-designed email messages (Treatments 1-3) being tested against Pier 1’s baseline version (Control).

Test Design: This was a simple A/B/C/D multi-factorial test. While we also measured open rate and conversion rate, the objective was to increase the clickthrough rate. Here are the page versions (click to zoom in):

Control                                              Treatment 1

Control Treatment 1

Treatment 2                                      Treatment 3

Treatment 2 Treatment 3

Results: So now that you understand the experiment background and have seen the treatments, can you spot which email performed the best? Before we reveal the results, here’s a chance to test your own marketing intuition and be regarded as a world-renowned marketing leader!

Related:  When is a Click More Important than a Purchase?

1. Which email generated the highest clickthrough?

  • Control
  • Treatment 1
  • Treatment 2
  • Treatment 3

UPDATE: Surprise! The Control was the winner. Each of the agency-designed treatments underperformed the original (one of which decreased clickthrough by 52%). Congratulations to Ben, the only correct response we received before we announced the results on yesterday’s web clinic. You can follow Ben on twitter at @findingforrest. Also, subscribe to the MarketingExperiments Journal to be notified when the web clinic replay and research brief are available so you can see the correct answer, the results of the control and treatments, and how these experiments can help you shape your own marketing campaigns.

You might also like

38 Comments

  1. Patrick - eKommerse says

    Cool, competition! 😀

    I’m guessing Treatment 3 will do well:
    – Lots of whitespace, clean separation of elements
    – All other designs our quite crowded, with no clear call to actions
    – The mentioning of prices will also help customers to assess whether the products are suited for them.

    Interested to know which design performed best!
    @patricksteenks

  2. Darragh says

    Treatment 1 I reckon. The ‘Shop Seating and Cushions’ call to action is much clearer on this than on the others. In fact it would have possibly been better to have said ‘Click here to Shop Seating….’ thus telling the reader exactly what to do.

  3. Claudia Viana says

    I love this task! : )
    ok, I think the third treatment is the most convincent.
    Is clean, show the products very clear, and show and ambience very real and comfortable.
    The squirrel is nice but grab to much attention…
    The only thing I would add is people enjoying the place.
    I’m from Uruguay and and I ‘m learning so much from you.

  4. Matt says

    Going with Treatment 2.

    matt_deboard on Twitter

  5. Crystal Kruszyna says

    Treatment 1

    @Crystal1407

  6. Elyse says

    @iconoinvestor

    Treatment #3!

  7. Kevin says

    I would say #3 because it is easier to read (more white space) and the messaging is more focused. In addition, their calls to action are clear and easily found.

  8. Ben says

    Control – @findingforrest

  9. Chris says

    It’d be nice if the images were larger so we could read the CTA button text easier

  10. Erick says

    Treatment 3

  11. Ber says

    Treatment 2 – @bbernrd

  12. Jerry Freeman says

    Treatment #3. @JerryKFreeman

  13. BradleyT says

    Treatment #2 because the squirrel asks users to click on a chair.

  14. Elke says

    Treatment #2
    @elkebovers

  15. Tom says

    Treatment 3.

  16. scpynkeal says

    treatment 2 because i like the squirrel.
    @scpynkeal

  17. Weston says

    Treatment 2
    @westontt

  18. yos says

    I saw treatment 3. It has in my mind the clearest call to action and the value prop. is the most obvious

  19. goaliemom31 says

    Control. Product is more clearly visible, color is appealing. Inviting scene invites click-through.

  20. Jeroen B says

    Treatment 1
    @Jeroen_Bouserie

  21. Alex Fisken says

    I think option #2 will be the best for CTR because it has two clear calls to action are above the fold (rather than the confusing “Hunker Down” in v1), especially as it pertains to an email preview window. It also lists the 25% discount and has pricing with slash-throughs for each item. Also, it has text links for the global navigation that should help with CTR if these specific products are not of interest to the end user. With regards to the product shots, showing them on a white background is better than showing them out of context around a cartoon campfire (I do like v4 showing them in a real-life scenario the best).

  22. Dave says

    I think “Control”

  23. indie_preneur says

    Yowza, thats a tough pill to swallow for the agencies…interesting to say the least. Great contest tho.

  24. Tim Watson says

    Great stuff. Having run many experiments and posed the question of which one won to experts its no surprise to me that few people got this right. In all honesty I probably wouldn’t either, probably would have gone for #2.

    You have to test to know ‘which’. There is no other way.

    Now knowing that #1 (control) was the winner the ‘why’ question?

    The control
    – Doesn’t have prices
    – Doesn’t feature particular products directly
    – The ‘out of the woods’ orange box is visually strong
    I’m sure Eyetracking would should it as hot spot.
    This is helping lead customers into the desired action, click to website.

    The main objective of #1 is to get people to click to find out more about the products, rather than sell the product already within the email. The other treatments give me more options, more direct product promotion and attempt to get further down the sales cycle in the email itself.

    For #1 you have to click to find out more. This could be pushing up the click rate over the others.

    It would be interesting to know if the conversion rates followed click rates exactly to support or reject this idea. Could it be control got most clicks but not in relative terms the same increase in conversion?

    Keep up the good work.

Leave A Reply

Your email address will not be published.