What really sets Adobe Test&Target (powered by Omniture) apart is both its reporting and support. While it can be expensive, what you get is a professional/polished, customizable tool that’s great for in-depth, multi conversion tests. And even better, you have an assigned support rep that can help you quickly overcome any stumbling blocks before or during a test.
To illustrate the level of reporting, with one of our Research Partners, we setup 35 success metrics for just one test. In the second in our blog series about testing solutions, here’s a quick guide to Test&Target…
Key benefits
- Transaction- and product-level reports
- Revenue tracking can be added to any campaign
- “Push winners” functionality – with a single click, pushes winning test scenarios to standard content for all audience segments
- Option for Omniture to host images, HTML or you can use your own server
- Tracking within Test&Target – ability to setup any metrics within tool; can change conversion metric during test (on the fly)
- Reports on a daily level
- Can track pretty much anything you want (links, buttons, etc)
- Behavior targeting – serve landing pages based on user defined segments (traffic sources, etc)
- Test&Target segmentation for reporting – can report on different campaign codes, offer IDs, etc
- Data privacy due to contractual relationship with a company
- Support – includes testing ideas, test implementation, report setup
- Ability to tack on other data-mining tools (i.e., Discover can track on the visitor level)
- Increased capabilities for segmentation and tagging of actions on a page – can get extremely granular in what you track, i.e. different content for different segments at different times
- If you don’t want to send all traffic to the page, you can set a tool to only receive a certain percent of traffic (would have to write custom code for Google Website Optimizer)
But keep in mind…
- Cost of tool and cost for additional support
- IT requirement for setup
What types of elements can you test?
- A/B and multivariate testing
How does it validate?
- Adobe Test&Target provides running status during data collection of statistical level of confidence, and will supply the bounds of the confidence intervals for the difference in sample means between each test treatment and the Control. It us up to the individual managing the experiment to decide what level of confidence they feel comfortable with…we recommend a 95% statistical level of confidence (p=0.05).
- Adobe Test&Target appears to use the presumption of a Gaussian distribution (the Student’s T) rather than the computed variance for a series of intervals for each treatment in estimating the test significance. Under the best conditions, for most testing situations, this should be a perfectly sensible assumption. It is, though, important to know that the presumption has been made so that if the test results appear to fluctuate significantly over time, you can dig more deeply to search for potential sources of fluctuation such as seasonality (in the statistical definition, meaning time-dependent patterns of difference like day/night, weekday/weekend, etc.).
Cost
- Range of costs
- Depends on server calls
Support
- Contract includes access to an account rep
Technology / Development
- Implementation – similar to Google Website Optimizer, couple of lines of JavaScript code
- Functionally, very similar to Google Website Optimizer
Reporting
- Tracking on Test&Target – Has the ability to setup any metrics within the tool; can change conversion metric during test (on the fly)
- Integrates with Adobe SiteCatalyst (powered by Omniture) for Web analytics
We’d love to hear some customer reviews as well. Use the comments to share your experience with Test&Target. And stay tuned to this blog as we provide quick guides for more online testing solutions to help you choose a platform that is best for your individual situation.
Related Resources
Online Testing and Optimization Solutions: Quick guide to Google Website Optimizer
Hi Gina, can you clarify the last bullet under Key Benefits? “If you don’t want to send all traffic to the page, you can set a tool to only receive a certain percent of traffic (would have to write custom code for Google Website Optimizer)”
GWO allows you to expose various percentages of traffic to the variation(s) in a test, so how does T&T differ? thnx
Hi Brendan,
Good question and thanks for asking me to clarify. As I re-read what I wrote, I realize that I could have been more clear. What T&T allows, which is a differentiator for the tool, is that it allows you to target (or include) a specified amount of traffic per test. As an example, say you’re testing a product page and the test is setup in T&T you can specify that you want to include only 50% of the total traffic for this page in the test. For Google Website Optimizer you’d have to write custom code to use only a segment of your traffic, and most likely the custom code would require additional development resources and time.
I hope this answers your question.
Thanks!
I looked into all the optimization platforms we’ve been reviewing, and Test&Target is the only one that has an application for smartphones. It’s available on the iPhone, Android and Blackberry. I believe the app is free for all three.