The placement of small data files called “cookies” on a visitor’s computer has long been a key tool in tracking our behavior across a website, keeping us logged into sites, and remembering who we are and what we like when we return.
Cookies are well-intentioned efforts to provide the most relevant content to each user, which is certainly a good thing. However, as far back as the 1990s, the privacy risks from cookies were making headlines.
Learn the ways of the third-party cookie
Suppose you visited a website called “www.lightsaberuniverse.com” which actually contained some banner ads pulled from another server, “www.sithboy.net.” A cookie was placed on your machine that passes a unique ID to “sithboy.net” each time you encounter one of its ads in the future, and you suddenly start seeing ads for lightsabers on all sorts of websites.
Providers of display advertising may serve
ads to many different sites, but use the same tracking cookie across all of them in order to build a profile of your interests (lightsabers, used speeder parts, paternity searches, rebel alliance manifestos, finding a good copilot … aka the “Wookiee cookie”) and general location.
That cookie, which was sent to a server other than the one housing lightsaberuniverse.com, is referred to as a third-party cookie. The profiles built could eventually become quite specific, and the third-party provider could theoretically put them to work in other ways which are beyond your control, such as geolocating possible Jedi threats to the Empire.
Awakening of the rebellion
In order to stop third-party companies from finding Luke Skywalker and Han Solo (or just generally knowing more about you than you want them to), there has been push back against third-party cookies, most prominently by the Safari browser, which does not accept them at all. An upcoming version of the Firefox browser will be set not to accept them by default, although individual users can opt into allowing them. Internet Explorer and Chrome offer the ability to opt out of third-party cookies.
As a marketer, you probably see the obvious implication of the push away from third-party cookies – display ads won’t be able to target beyond knowing what site the visitor is currently on.
In online testing, there is a major implication of the shift away from third-party cookies, which may be harder to see. Splitters, such as Adobe’s Test&Target, usually use a first-party cookie from the host domain, but in order to track across multiple domains or tracking profiles, it offers the option to use a third-party cookie.
If a test is run with third-party cookies, that cookie will not be accepted by Safari users and those people will most likely be defaulted into the control experience, or eliminated from the test altogether. This creates a sampling validity threat to test results, because Mac and iDevice users have been directed disproportionately to one treatment within the test.
From past experience, we know that user group is slightly more affluent, more likely to be on a personal computer than a business machine, and more aesthetically-driven than the general Web audience.
This means our control, where Safari users were all placed, will better reflect the behavioral tendencies of the Safari user group than it should, and the treatments will better reflect the Windows and Linux users than they should. Any comparisons of treatment and control are rendered highly suspect. If all Safari users were eliminated from the test, our results would not be generalizable to the Safari group because of their known behavioral differences.
A new hope
While I have yet to see a good alternative solution for cross-domain tracking during tests, if I’ve learned anything from the events that happened a long time ago in a galaxy far, far away, it is that when danger is present, new heroes tend to emerge. So, I look forward to seeing how online testing platforms respond to this threat marketers face in learning more about their customers through A/B testing.
Photo attribution: Peyri Herrera
Related Resources:
Online Testing: 6 test ideas to optimize the value of testimonials on your site
Social Media Microsite Test: 3 lessons based on a 154% increase in leads
Best blog post ever!