Ask MarketingExperiments: How do qualitative research, design thinking, and design sprints relate to A/B testing?

(This article was originally published in the MarketingExperiments email newsletter.)

We frequently receive questions from our email subscribers asking marketing advice. Instead of hiding those answers in one-to-one email communication, we occasionally publish edited excerpts of some of them here as well as our sister publication, MarketingSherpa, so they can help other readers as well. If you have any questions, let us know.

Dear MarketingExperiments: Hi. I read a lot of your articles and see your videos. Your research techniques are based on A/B testing. What’s your opinion about design sprint and design thinking process and qualitative-based research.

Thanks.

Dear Reader: Thank you for your question. A/B testing is not mutually exclusive from the other techniques you mentioned. In fact, they are often paired together by practitioners who are either aware of specific methodologies incorporated by these techniques, or are simply engaging in many of the tactics in those methodologies without specifically following (or even being aware of) them.

Here is some information to answer your question paired with some related resources so you can dig further.

Qualitative-based research

Qualitative research can be valuable to help you build a theory about your customer. You should then form hypotheses to test your theories. That’s where A/B testing comes in. The qualitative research can also help you interpret those test results.

So, for example, you might assume customers care more about the environmental aspects of a product than the luxury aspects of a product based on your qualitative research. You can run an A/B test to determine if you are correct. The qualitative research can then help you answer why customers have the preference. For example, if they care about the environmental aspects for a deeper reason — maybe they are concerned about getting cancer from chemicals — it can lead you to explore other ways to serve those customer motivations with your messaging and your products.

Some sources you might consider for your qualitative research include:

• Social media — How to use social media to help discover why customers buy from you
• Online reviews — How to start discovering your value proposition with Yelp
• Focus groups — Procter & Gamble Reveals How a New Focus Group Tactic Solved Four Flaws in Their R&D Process
• Surveys — 6 Tips for Creating an Effective Survey
• Customer and employee interviews — Every Conversion Copywriter Should Be Skilled at Interviewing
• Usability testing — Usability Testing: 5 tips for augmenting A/B testing

You can build a database of your qualitative research discoveries to help you track your findings and prioritize what you would like to test.

While the MECLABS methodology employs quantitative methods to study objective evidence in pursuit of a robust Customer Theory, those quantitative methods are enhanced by qualitative methods. Quantitative and qualitative methods are not at odds with each other, rather they work better in tandem.

Design thinking process

Design thinking is a methodology that focuses on the user. A methodology like design thinking can go hand-in-hand with customer-first marketing.

Key elements of design thinking include empathizing with the prospective customer or user, defining the problem, ideating a solution, prototyping that solution and then testing it.

Again, we see where A/B testing can fit into this methodology.

Design thinking is similar to the MECLABS Conversion Sequence Heuristic — a patented methodology to help you empathize with the customer, define the problem and come up with solutions. And ultimately you want to test to see how your suppositions (hypotheses) behave in the real world.

Design sprint

A design sprint incorporates many of the ideas we’ve already discussed. It is in many ways a focused application of design thinking in a condensed amount of time.

That focus can be essential to the modern marketer and business person. We frequently find our time and attention splintered by all of the tasks we’re trying to keep up with — from long-term strategy to immediate fire drills. A surge of effort on a precise task has become a popular way of attacking a specific problem or opportunity. It can be personally fulfilling as well, bringing the team closer together and uniting everyone around a common goal with a clear success metric — building something specific to solve a specific problem.

As such, this sprint mindset isn’t unique to design or marketing. In fact, it is especially popular in information technology (IT) where practitioners, for example, use the Scrum project-management system for website redesign and software development.

If you are going to engage in a design sprint, it helps to begin with insights about the challenges and opportunities for the product and/or user experience based on some information.  This is where qualitative research can come in.

It also helps to get an understanding of how the thing you create performs with users and customers — this is where A/B testing comes in. After all, a design sprint can be a good way to build something based on the best internal gut thinking, external opinions and other data. But again, this should lead to a hypothesis that you can ultimately test with real-world behavior.

While we never use the term or official methodology of a design sprint, MECLABS also engages in focused activities with business and marketing teams to get a specific, customer-focused output that can then be tested — our Quick Win Intensives and Value Proposition Workshops.

A common thread

I’ve found that some people in life prefer a specific methodology or canonical approach to follow.

I’m more of a big-picture guy myself — what can I learn from different methodologies, approaches and belief systems? And how can I apply that in my life, in my career and in specific situations?

If we were to step back for a minute, there is a common thread in all of the research tactics we’ve discussed:

• How can you focus on the other (i.e., user or customer)?
• How can you discover what will best serve the other (which can be very different from your own opinion)?
• How can you structure your time, efforts and resources to serve the other?
• How can you know if your suppositions and effort were effective? (What is your single source or truth?)

Thanks again for your question and best of luck in wherever your research journeys take you.

Related Resources

MECLABS Institute Online Testing on-demand certification course — Learn where to test, what to test, and how to turn basic testing data into customer wisdom

Marketing Analytics: 4 techniques to discuss with your data analysts

Heuristic Cheat Sheet: 10 methods for improving your marketing