What to Measure in Multivariate Testing: Defining Your Success Goals (Part 2)

December 8, 2014

In my last article, I defined multivariate testing and how it can optimize your Web marketing, as well as five common errors to avoid. Let’s now look at what to measure in your tests and how to define your criteria for success.

Before you start formulating a test hypothesis, or begin running your tests, the first and most important step is to ensure that you have clearly defined objectives for your Web site. You’ll want to examine your marketing goals in order to determine the appropriate success factors that all of your organization’s stakeholders can agree upon.

Let’s start with typical measureable Web site goals:

  1. Make money: sell product, generate leads and advertising or promotional click-throughs.
  2. Save money: enable users to adopt self-service features and answer product and service questions on their own (such as through online FAQs and documentation).
  3. Create brand awareness and industry visibility.

It’s important for all stakeholders to agree on the goals of your Web site, because when a decision is made to adjust or optimize something on the site, everyone’s needs should be addressed. You’ll want to make sure you are testing the things that truly matter for your organization and balancing performance across all stakeholders.

Once you determine the goals, the next step is sifting through potential key performance indicators (KPIs) to decide which will accurately measure progress toward your specific marketing goals and benchmarks. For example:

  • If your goal is to make money, you’ll want to track those pages and areas of the site that users click on in the conversion funnel, such as the Buy Now button and resulting Thank You pages.
  • If your goal is to save money, track the interactions with both the self-service areas (like FAQs and help content) as well as non-self-service areas (like contact and help ticket generation) of your site.
  • Brand awareness goals can be more difficult to track, but certain KPIs can be proxies for customer loyalty, such as recency and frequency of visits and time spent on site, and the percentage of return visitors. Other behaviors to track could include “send to a friend” or “print page” features that indicate a visitor’s interest in sharing your site with others.

Beware of scenarios where an increase in one desirable KPI can cause a decrease in another (perhaps more valuable) KPI. This cannibalization can sometimes be a Catch-22, so the best practice is simply to track both KPIs to provide increased visibility of user behavior.

This article originally appeared in the June 2008 issue of MarketingProfs. You can read the original version here.