At MECLABS, we approach marketing as a science, but even scientists can lose sight of testing to find answers and start behaving as if they already have them.
Our Peer Review Sessions are the most notorious moment for this to happen. We all come together in the hope of having an open discussion on the many possibilities of every test and often end up throwing out solutions instead of making sure we’re asking the right questions.
Consequently, this can become a problem, as we are not our own target audience and often too close to our individual projects to serve as a fresh set of eyes.
Usability testing can help you ask the better questions
It is in this situation that usability testing can become an asset. We certainly don’t endorse usability testing as a substitute for A/B testing.
Even better, we suggest you engage in heuristic-based A/B testing to make sure you have a formal methodology around which your behavioral testing is based, to both create more informative A/B tests and to replicate the ability to create these tests across your entire team.
That said, usability testing could be useful as a supplement to Peer Review Sessions or other brainstorming sessions to make sure the target audience for the page has a say.
To help you create hypotheses for your A/B testing, here are five tips for using usability testing:
- You must understand you are not going to receive results that indicate the best design, copy, or page to sell your product.
Usability testing is better for determining if a process makes sense or if the functionality of a page contains a lot of friction or anxiety.
- When writing the tasks and questions for a usability test, always make sure to target questions specifically to help you learn.
For example, when we engage in usability testing we target questions to a specific part of the MECLABS methodology.
We never ask, “Do you like this page?” Rather we would ask the user “Is there anything on this page that confuses you?”
A question like this will highlight friction elements from this user’s perspective.
Additionally we would never ask a user to just move through a process. We would ask them to “show us how you move to the next step and explain what you think will happen next aloud. Did the click match your expectations?” This question will not only call out the friction in the functionality of the page but also indicate possible motivation. What did they expect, what did they actually see?
- Never make a business decision based on usability testing.
Usability testing is generally done with groups that are too small to statistical significance. We are usually looking at five to 10 15-minute videos filled with opinion and occasionally off-topic ranting. This is the place you get additional ideas and identify more scientific questions that will make good tests. It is not a replacement for or an alternative to actual testing. We never make a final recommendation based on results from usability testing.
- Pay attention to demographics.
Some of the Research Partner sites we test on are naturally divided into the groups (age, income level, geographic location, gender and in-market) that will enjoy the products or services our partners offer.
Online usability testing, however, tends to be a paid service and participants who sign up are likely motivated by money, which makes the sample not a true reflection of the site’s user population.
As a result, it is important to indicate the age, gender, nationality, and any other factors that you would prefer to see affect your test.
- Watch the videos yourself.
Many online usability-testing services provide video analysis as part of some of their many packages. If you’re thinking of investing in such a service for your own brainstorming, we would certainly suggest getting a report.
However, no one will ever understand your product and your objectives better than you do and you will always get more from your usability test if you watch at least some of the videos yourself. In addition, they are occasionally entertaining.
I cannot stress enough that usability testing should not be a replacement for real world, behavioral testing with a valid sample size. Rather, it should be used as an aid to the scientific process.
Therefore, if you start to feel that you have stopped asking the right questions about how you can make your site better and are just throwing ideas at the wall, you may have reached a point where usability testing can come to the rescue, bringing fresh eyes and concerns to the table.
Related Resources:
Web Usability: People don’t need many options, they need the right options
Web Usability: When should you avoid navigation?
Web Usability: The Squint Technique and other insights from your peers