One of the questions that comes up all the time in web optimization — especially for bigger design changes — is whether you should A/B test or Multivariate test. Each type of A/B testing serves a unique purpose and answers a different question. This blog will give you a quick definition of terms, and the video below walks through real examples of when you would use each.
What is an A/B Test and a Multivariate Test?
A/B Test (Also A/B/n A/B Test): An A/B test involves two or more variations of the same element compared against each other. You might have many variations of that element, but you only A/B test one element of your site.
Multivariate (MVT) A/B Test: A multivariate test compares different combinations of multiple elements. You could have as few as three variations, but those variations measure several separate elements in conjunction with each other.