Lisa Frank

A recent blog post on “A/B Testing and the Benefits of an Experimentation Culture,” by Wyatt Jenkins on Harvard Business Review’s website, raises several points that resonate with SiteSpect’s view of optimization.

Jenkins is the VP of Product at Shutterstock. He mentions the need to test everything and at Shutterstock, everything includes not only aesthetics, such as copy and link colors but also back-end functionality, such as the relevance algorithms that rank search results. We completely agree! It’s important to test all aspects of your site such as search algorithm changes, feature launches, complete site redesigns, and checkout flow processes. Testing how your site looks can reap rewards, but it’s also important to start thinking about other areas of your site that may lead to increased conversions.

Another point speaks to the very nature of testing. The ideas or concepts that you thought would be successful may not always work out. The author cites an example where the test results were the complete opposite of the original expectation. While your gut can lead you to discover new testing opportunities, it’s not always right. Only through testing can you establish data-driven reasons to make the best decisions for your site.

The post also mentions the importance of analyzing results by segment. Shutterstock discovered the importance of analyzing test segments so that you don’t miss out on key opportunities. In aggregate, while the overall test results may indicate no significant uplift, different customer segments (subsets) may show combinations of big wins and losses that have been averaged out from the global results.

Finally, never underestimate the importance of site speed. The author acknowledges its importance and widely cited Akamai research from 2010 indicates that “57% of online shoppers will wait 3 seconds or less before abandoning the site.” There are several factors that impact the speed of your site including the presence of JavaScript tags utilized by some optimization vendors.

As a software provider in the website optimization space, these same ideas inform a lot of our decisions. It’s good to see that others are discovering the benefits of a testing culture. More and more companies are discovering the need to make testing an integral part of how they deliver an optimal customer experience. We hope you can be one of them. If you’d like to learn more about the benefits of an experimentation culture, please feel free to contact us today.

Tags: A/B & Multivariate Testing