3 A/B Testing Case Studies from Smart Brand-Side Marketers


When you’re embarking on a new campaign, or a launching a new CMS, there are a million things that can go wrong. To which I say: GOOD.

Bring it on. That’s what testing is all about, right? Letting the things that will go wrong, go wrong – but tracking it every step of the way.

That, my friends, is the difference between a mistake and a “learning.”

See that positive spin?

As the Managing Editor of MarketingSherpa (MarketingExperiment’s sister site) I have covered hundreds of marketing campaigns by companies, and generally they are focused more on high-level inspirational stories. However, many of them have a lot of specific ideas for testing, and I’ve put together a collection featuring three of those case studies here.

Case Study #1. Extra Space Storage


In this MarketingSherpa Reader’s Choice nominated campaign, the team at Extra Space Storage knew they were missing opportunities with customers using a business-first email program. So, they decided to tear it down to the studs, and rebuild it as a customer-first email program.

The first place they started with was to scrutinize the previous system for lost opportunities – the first of which was that it was completely transactional, and not mobile-friendly.

“It’s a natural progression [that] customers are contacting us on the web. They don’t want our manager calling them to talk about their reservation,” said Jennifer Stamper, Interactive Marketing Manager, Extra Space Storage. “They’d prefer to get an email. They want to be spoken to in the channel that they’ve contacted us in.”

She and her team tied all of the available data together to come up with four different personas to represent the customer base. They then took those four personas and worked through the purchasing patterns of each, to figure out what would convert each of them best.

After utilizing a lot of behavioral research, data, and even utilizing a Myers-Briggs personality-type indicator, they were able to come up with template aspects that spoke to each specific customer group.

“Sixty percent of our customers have actually never used storage before. We did a lot of research around what questions do they have. What information do we need to provide them before they show up at the store? We really focused on answering those questions before the customer even realized they had those questions,” she said.

Read the full case study to see how through three iterations, Stamper and her team reworked templates, personalized sends and ran email frequency and content tests to drive a 50% jump in attributed conversion rate for email – a first in company history.


Case Study #2. BookPal

1-asA better B2B shopping experience is what drove BookPal, a B2B ecommerce firm specializing in bulk book sale, to redesign its website. The idea driving the entire effort boiled down to: while BookPal sells to schools and businesses, its customers are individual consumers.

That’s the principle that drove this entire campaign, starting with an in-depth website audit. The objective of which was discovering what BookPal was doing right and uniquely in the space, and what areas needed greater attention.

“We went through a fairly substantial needs analysis looking at what our existing website technology provided, where the shortcomings were, and then ultimately looking at the unique nature of our B2B business and our school education and corporate customers, what we did uniquely online that we could do better, where could we enhance that B2B experience,” said Tony DiCostanzo, President, BookPal.

Enhancing that experience included integrating B2C practices like social media, and tracking via AdWords, while also reconfiguring the website based on analytics data, to remove a lot of the existing friction blocking the path to conversion.

Read the full case study to show how by creating a simple, persona-focused experience, BookPal was able to drive order volume up 211% in three years.

Case study #3. Skyjet



This Reader’s Choice nominated case study, by private jet charter provider Skyjet, focuses almost entirely on value proposition.

In early 2015, Jonathan Levey, Digital Marketing Manager, Skyjet and his team identified an opportunity to leverage mobile based on an impressive trend in Skyjet web analytics.

“The mobile traffic on Skyjet jumped 50% in Q1 2015, compared to Q4 2014, and quote requests increased 177% on mobile devices,” he said.

Furthermore, the average mobile user spent two minutes and 14 seconds on Skyjet.com, which is 29% longer than the typical desktop user.

Skyjet realized the opportunity in supplying its niche group of customers with the value of an app featuring on-demand booking, with the ability of its system to instantly calculate and process an itinerary, evaluate different variables to give customers a price within seconds.

The team is also constantly testing features of the app, such as push notifications, and iterating it on a monthly basis to ensure customers have the best possible experience.

Read the full case study to see how the app’s real-time pricing tool now processes roughly 1,500 user itineraries on a weekly basis, and has more than 22,000 downloads since its launch.

You might also like:

Homepage Optimization: 5 Marketing blind spots that inhibit conversion (and how you can correct them)

Content Marketing Generated a 5,100% ROI for Health Care Innovator Optum [via MarketingSherpa]

Make Your Content Useful: How a simple UI change created 74% more page views

You might also like

Leave A Reply

Your email address will not be published.