Getting a Jumpstart on Optimization Planning

This post will explore ways to create your A/B testing strategy, techniques to identify key optimization opportunities, and best practices to develop a metrics-driven optimization roadmap.

So how do you get started thinking about your optimization program? The answer is simple: start with what matters most — your Key Performance Indicators (KPIs). In turn, those KPIs can be correlated to site factors that can be A/B tested.

As you think about your A/B testing strategy, and start mapping out your plan for 2013, make sure you’ve addressed the following questions:

• Why do you want to run this A/B test? (What is your hypothesis?)
• What do successful metrics look like?
• How do those relate to testable elements of your web or mobile site?
• Who exactly are you targeting?
• How much traffic do you expect (in order to estimate time to statistical significance)?
• What external factors could affect the A/B test (such as marketing campaigns, seasonality)?
• Under what conditions must the A/B test be stopped?
• Who will be affected internally (key resources)
• Who is your key stakeholder?
• When do you need results?

So, if you’re in the e-commerce business, then Revenue and Average Order Value are quite likely your KPIs. Obviously you’ll want to track those pages and areas of your site where users click on within the conversion funnel — e.g., the “Buy Now” button and resulting “Thank You” pages. You’ll also want to A/B test micro-conversion events, such as viewing the product detail pages and the elements on them — descriptions, images, or videos, for example.

Once you decide which factors are most important to your KPIs, then you’ll decide what experimental design is best. With A/B testing, you A/B test one factor (such as the “Buy Now” button) against one or more variations to see which is most persuasive. With multivariate testing, you test multiple factors simultaneously. Evaluating the impact of combinations of factors and variations often reveals significant interaction effects that can have a dramatic impact on your conversion goals.

If you are not sure where to start when selecting testable factors, SiteSpect recommends a 3-step multivariate testing process to zero in on the factors and variations that matter most:

1. Test Wide – Test multiple factors with fewer variations to see which factor is most influential in moving the needle.
2. Test Deep – Take a few of the top performing factors and test multiple variations of them to understand which combination works best.
3. Bakeoff – Take the winning combination and A/B test it against the control, and your results will tell you how much you will be able to improve your KPIs.

But don’t stop there! Optimization is a continuous process, not a “set it and forget it” activity. In an ideal world, you’ll use A/B test results as a source for future A/B testing ideas in addition to your analytics data, key stakeholders, and user research, among other things. You’ll create a process for vetting possible A/B testing ideas against your online goals and using accepted ideas in creating new hypotheses and A/B test campaigns. In this way, an optimization program goes through a Define-Design-Develop-Analyze loop as you think about comprehensively A/B testing every part of your funnel, from Reach (email and advertising) to Acquisition (landing pages, product detail pages) to Conversion (forms, shopping carts) to Retention (thank you pages, additional offers).

One thing is clear: optimization is critical to your site’s success. I am hopeful this post has given you a few ideas about what you can do to start planning your 2013 optimization efforts today.

To learn more about SiteSpect, visit our website.