On-Site Search: A/B Test the Back-End

My last few blog posts have been dedicated to the importance of visual and functional A/B tests for on-site search. Before we wrap up the series, let’s review a few more ideas related to the back-end areas of your site not noticeable to the visitor:

Algorithm: A/B test the underlying algorithm that powers your search. The algorithm determines what the search returns. A/B testing a new or updated algorithm highlights opportunities to improve. You can A/B test sending users to different versions of a search algorithm, your own vs. one provided by a vendor.
Vendor: Compare two different vendors in head-to-head competition. Take the guesswork out of which vendor tool works best for your site. You can use SiteSpect to compare searches provided by two different vendors.

Bringing It All Together

Now that you have the list of suggested A/B tests, how do you move forward? As a first step, identify all the stakeholders on your optimization team. Then bring in other teams: cross-team collaboration is a critical element of prioritizing your roadmap and determining how to begin. Note the areas of search that are affected by each A/B test so they don’t collide. For example, you don’t want to A/B test relevancy when another part of the organization is A/B testing the algorithm. The two are so closely related that changes to one may affect the results for the other.

We started the series with a refresher about the importance of A/B testing on-site search. Then we provided some specific examples on what to A/B test. Hopefully this has sparked a few ideas that you can put to work in your own organization. For your reference, here is the list of A/B test ideas:

Visual A/B Tests: