Multivariate Testing, Part 1: An Introduction

As web marketers seek new ways to boost conversion rates and improve their visitors’ site experience, interest in multivariate testing is on a feverish rise. But those unfamiliar with the techniques are often unclear about where to start, or how to ensure success.

In this article, I’ll discuss the following:

• A clear definition of what multivariate testing is and how it differs from another common type of testing: A/B testing.
• How testing provides a foundation for continuous improvement of your Web marketing initiatives.
• An overview of five common mistakes to avoid when planning and running tests.

What Is Multivariate Testing?

Common methods for running controlled experiments on Web sites range from simple A/B testing to sophisticated multivariate testing, also known as multivariable testing.

In A/B testing, one or more new versions of a page or single site element competes against an existing control version. For example, two versions of a headline might compete against an existing headline.

Multivariate testing, on the other hand, is like running many A/B tests concurrently, where there are multiple elements being tested at the same time. For example, two alternate product images, plus two alternate headlines, plus two alternate product copy text, for a total of 27 possible combinations (including the original control versions).

This article originally appeared in the April 2008 issue of MarketingProfs. You can read the original version here.