In Focus: Online Split Testing

Cabela’s runs real-time Web tests without burdening its IT department

Adding new functionalities without adding to the IT department’s workload is always a win-win situation. Cabela’s recently implemented an A/B split Web site testing function that runs without burdening its IT department. “With SiteSpect we need virtually no IT involvement,” says Vince Stephens, Internet manager of planning and analytics for Cabela’s.

The A/B testing is used to compare one or more variations of a single site element, such as a promotional offer or product presentation. Different versions of the element are presented to consumers whose response determines the success or failure of that content.

Using that information the retailer can determine which version is most persuasive and yields the best conversion rates. The SiteSpect product chosen by Cabela’s involves no site modifications and new tests can be up and running in a matter of hours.

Cabela’s is using the A/B testing to evaluate many details of its site, beginning with smaller items such as images and font size. “Other companies had some of their biggest wins with some of the smallest changes,” says Stephens.

Initially, though, Cabela’s looked at evaluating more complex site features. “We ran a test with the checkout process but didn’t find any significant differences,” says Stephens. “We have changed our focus to tests that are easier to execute and measure, and offer potential for bigger wins.”

Stephens chooses which functionalities to test based on potential ROI and simplicity. “We have a list of different tests we’d like to do and we ask some questions such as: ‘What is the potential ROI? How complex is the test? Can we run multiple tests at one time? What will be the impact of one test on another?’ Then we prioritize.”

Avoid Extra Steps

One of the best qualities of the system is that the customers are not routed through a third party. “That is extremely powerful,” Stephens says. The SiteSpect system provides results and statistical analyses in real time plus the ability to target testing efforts by segmenting the customer base. Web-based reports, showing how site variations impact customer behavior, are available throughout the test process.

The new system was implemented after six to eight months of load testing and was launched in October of 2005. And with minimal involvement with the IT department, Cabela’s was able to make the decision to implement more easily.

“It was just the right timing for us,” says Stephens. “A/B or split testing is something more companies are doing. We’d seen it demonstrated at trade shows and other online retailers were beginning to have some success with the tools.”