You could describe me as a drooling, babbling, incoherent person who can’t find his mouth with a fork and drags himself along the floor to get from point A to point B.
But (except for a mistake or two in college), I’ve changed significantly since I was a six-month-old baby. And so have your customers.
You test. You learn. You know. But …
A/B or multivariate tests are just a snapshot of customer behavior at one moment in time
So to really understand your customer behavior, you must continually test and not think you know the answer because something worked in one test a few years ago.
To wit, let me tell you the story of Musicnotes.com, the world’s largest ecommerce sheet music retailer and publisher, with more than 4.5 million customers around the globe.
“Actually introducing trustmarks was causing negative impact on our conversion,” Bill Aicher, Chief Marketing Officer, Musicnotes.com, told me when I interviewed him at a Media Center run by MarketingSherpa (sister publishing company to MarketingExperiments) at IRCE.
Watch the video interview below, or read on, to learn why trustmarks might have been reducing conversion for this ecommerce site, and the ultimately successful approach the marketing team discovered by not stopping at one test, but continually challenging what they thought they already knew with further tests.
Why were trustmarks reducing conversion for Musicnotes.com? Bill went on to tell me, “Our theory came down to the fact that we’re introducing a question of trust when that issue hadn’t existed in people’s mind.”
Testing to discover how to best use trustmarks
However, Bill and his team knew that while trustmarks didn’t work in that particular situation at that particular time, customers’ anxiety about entering information and making purchases was growing.
So they ran some follow-up tests at a different place in the funnel to discover how best to use trustmarks to reduce (instead of introduce) customer anxiety.
For example, they tested with a seal from a vendor that promised guaranteed shopping and identity theft protection. “We actually didn’t see much of an impact, so we didn’t go forward with that,” Bill said. This time there wasn’t a negative result, but it wasn’t positive either.
But customers kept changing
Over time, identity theft and credit card fraud appeared more and more in the news and if felt like the zeitgeist was changing.
So Bill and his team wondered if their customer had changed as well. They tested yet again, putting a trustmark below the “Add to Cart” button and also a pervasively scrolling trustmark in the lower corner of the screen.
“Actually, it turned out it was different. We ended up seeing a 5% lift by showing that [trustmark],” Bill said. “I was honestly surprised to see it, because it was against a lot of our prior learnings.”
In addition to the conversion lift, the seal also helped with customer feedback.
For every order, customers receive an email from the trustmark company that says, “Hey, you’ve also gotten this free safe shopping guarantee courtesy of Musicnotes.com” that goes over the details of the program and gives them an automatic way to respond and say “thank you” to Musicnotes.com.
“We actually saw a lot of positive feedback,” Bill said. While the default response for customers is sending back an email that said simply said “thanks,” Bill said that “over two-thirds of the feedback we’re getting from customers is explicitly positive.”
Customers were actually writing emails like, “Oh, I saw in the news the other day that 97% of customer data isn’t safe” and “I feel so much safer. Thank you for doing this for me. I really appreciate this.”
“Which is pretty interesting. People don’t usually take the time to do that kind of stuff,” Bill said. “When my customer support team told me we had 3,000 responses over the last couple months … I was really surprised at that.”
About one-third was a default “thanks” response, and “less than 1% have been negative,” Bill said. “And mostly it was people thinking that they had to pay for something … which wasn’t the case so we just dealt with those issues ourselves.”
When should you challenge the discoveries you’ve already made through testing?
Of course, resources are limited. So you can’t continually test everything all the time. You want to learn from each test to build a customer theory. Just make sure it is a living, breathing customer theory that can evolve over time.
A few indications that you may want to run follow-up testing to challenge the paradigm include:
- There is a long-term, macro-shift to consumer preference (e.g., the growth in popularity of organic food in relation to processed foods)
- There is a “shock to the system” that has consumers question their traditional habits (e.g., the documentary “Super Size Me”)
- Your marketplace shifts thanks to the addition of a disruptive competitor (e.g., Tesla for autos or Apple for phones)
- Customer service starts getting feedback indicating new customer preferences
- You expand your products and services into new demographics or segments
- You keep the same customers but they go through significant changes over time (such as aging)
- You keep the same demographic of customer, so you must continually replenish with new generations in the same age group that may have different priorities (e.g., Gen X versus Millennials)
- The technology of how your customers interact with your buying process changes (e.g., smartphones)
All of these changes indicate a need to keep your finger on the pulse of conditions that can impact your customer. And keep an open mind to testing.
“We’re a company that tries to look at things [in] a constantly testing environment,” Bill said. “Things change all the time. The landscape is always changing.”
You can follow Daniel Burstein, Director of Editorial Content, MECLABS Institute, @DanielBurstein.
You might also like
Customer Anxiety: One element of the MECLABS Conversion Heuristic explained
Watch 42 more ecommerce-related interviews from the MarketingSherpa Media Center at IRCE 2015
MECLABS Online Testing online course – Learn a proven methodology for executing effective and valid experiments for increased marketing ROI
MarketingSherpa Summit 2016 – At the Bellagio in Las Vegas, February 22-24