I’m pretty sure I dislike pop-ups. Maybe it’s just a gut reaction to some pre-pop-up-blocker memory of my computer being overtaken by hundreds of pop-ups in mere seconds, but whatever it is, I just don’t like them. Immediately, I think of spammy, over-hyped, flashing messages with really bad clip art.
But am I throwing the proverbial baby out with some really obnoxious and annoying bath water? Well, at MarketingExperiments we test even our most strongly held website prejudges. We constantly challenge, test, and many times find holes in commonly-held “best practices.” And so, feeling a bit adventurous, our analysts took the opportunity to test the effectiveness of that frequently offensive marketing tactic – the popup.
The Research Partner we were working with was a B2C company offering counseling services in credit consolidation. The primary objective that they identified for their homepage was to increase the amount of quote requests.
Currently, their homepage featured the quote request form front and center, but we wondered if the current format got lost at all in the design of the homepage. So we tested a couple of designs attempting to bring more attention to the form offer.
The first treatment integrated the form into the most attention-getting element of the homepage – the top banner. The banner was blue and utilized strong imagery. The team’s hypothesis was that if we placed the form in the banner, more visitor attention would be drawn to it.
It is interesting to note that even thought we added the form to the top banner of this homepage, the form still remained in its current location. This gave the visitor two separate places to see the quote request form.
But how much would banner blindness mitigate the attention brought to the offer? Would people just skip over the image? Are banners like this effective for primary objectives? These were all secondary research questions we were seeking to answer with this treatment
In another treatment, we tested using a popup to emphasize the quote request message for first-time visitors. The overall homepage design remained the same, but was grayed-out behind the popup. The popup could be closed by clicking a simple text link in the top-right corner, filling the screen with the normal homepage in full color. Like the first treatment, the form still appeared in its original spot.
The hypothesis was that the popup might eliminate all other distractions focusing the visitor’s attention squarely on the free quote request offer.
But would using a popup cause any additional annoyance or concern? Would narrowly focusing on a single offer hurt the effectiveness of this homepage? Are there any prejudices against pop-ups? These were all secondary research questions we were testing with this treatment.
Will a popup really work? Is the top banner strategy better? Or will the control remain king with its simple front and straightforward approach? Questions like these underscore the importance of testing, for even in our most brilliant moments, we cannot be 100% sure of our intuition. But that’s not to say that some marketer intuition is better than others.
And the winner is…
UPDATE: As some of the blog comments predicted, the winner was Treatment 2 (the pop-up) – by 63%. That’s right, despite all the negative baggage that comes with a pop-up, it appears that in some cases it will perform best. It is also interesting to note that the first treatment, which is a more common approach, underperformed against the control by -2%.
If there is one key principle that can be gained from this test (and there are others), it’s that sometimes best practices don’t work. And sometimes the very tactics you know won’t work, do. That is why it is important to get beyond gut feelings and test. This is not anything you haven’t heard from us before…but too often, common knowledge is not common practice.
What do you think?
Why did the pop-up work in this test? What are some cases in which a pop-up wouldn’t work? Should we start using pop-ups more often or are there still enough negative conations associated with them?