Optimization Planning 101

By SiteSpect Marketing

March 21, 2013


When it comes to the quality of your site’s user experience, A/B testing and optimization are proven techniques to help you scientifically and continuously improve how you engage, convert, and retain your customers.

Whether you have been A/B testing for years or you are just getting started, running a successful optimization program depends on defining and refining your strategy.

This article will explore ways to create your A/B testing strategy, techniques to identify key optimization opportunities, and best practices to develop a metrics-driven optimization roadmap.

So how do you get started thinking about your optimization program? The answer is simple; start with what matters most — your key performance indicators (KPIs). In turn, those KPIs can be correlated to site factors that can be A/B tested.

As you think about your A/B testing strategy, and start mapping out your plan for 2013, make sure you’ve addressed the following questions:

  • Why do you want to run this A/B test? (What is your hypothesis?)
  • What do successful metrics look like?
  • How do those relate to testable elements of your web or mobile site?
  • Who exactly are you targeting?
  • How much traffic do you expect (in order to estimate time to statistical significance)?
  • What external factors could affect the A/B test (such as marketing campaigns, seasonality)?
  • Under what conditions must the A/B test be stopped?
  • Who will be affected internally (key resources)?
  • Who is your key stakeholder?
  • When do you need results?

So, if you’re in the e-commerce business, then revenue and average order value are quite likely your KPIs. Obviously, you’ll want to track those pages and areas of your site where users click on within the conversion funnel — e.g., the “buy now” button and resulting “thank you” pages. You’ll also want to A/B test micro-conversion events, such as viewing the product detail pages and the elements on them — descriptions, images, or videos, for example.

Once you decide which factors are most important to your KPIs, then you’ll decide what experimental design is best. With A/B testing, you test one factor (such as the “buy now” button) against one or more variations to see which is most persuasive. With multivariate testing, you test multiple factors simultaneously. Evaluating the impact of combinations of factors and variations often reveals significant interaction effects that can have a dramatic impact on your conversion goals.

If you are not sure where to start when selecting testable factors, here is a three-step multivariate testing process to zero in on the factors and variations that matter most:

Test Wide

Test multiple factors with fewer variations to see which factor is most influential in moving the needle.

Test Deep

Take a few of the top performing factors and test multiple variations of them to understand which combination works best.


Take the winning combination and A/B test it against the control, and your results will tell you how much you will be able to improve your KPIs.

But don’t stop there! Optimization is a continuous process, not a “set it and forget it” activity. In an ideal world, you’ll use A/B test results as a source for future A/B testing ideas in addition to your analytics data, key stakeholders, and user research, among other things. You’ll create a process for vetting possible A/B testing ideas against your online goals and using accepted ideas in creating new hypotheses and A/B test campaigns. In this way, an optimization program goes through a define-design-develop-analyze loop as you think about comprehensively A/B testing every part of your funnel, from reach (email and advertising), to acquisition (landing pages, product detail pages), to conversion (forms, shopping carts), to retention (thank you pages, additional offers).

One thing is clear: optimization is critical to your site’s success. I am hopeful this post has given you a few ideas about what you can do to start planning your optimization efforts.

To learn more about SiteSpect, visit our website.



SiteSpect Marketing

SiteSpect Marketing

SiteSpect Marketing focuses on authoring content for the web and social media.

Suggested Posts

Subscribe to our blog: