Kim Ann King

Optimization Vendor Checklist: 12 Must-Haves in a Testing & Targeting Solution

Posted on

  • email
  •  
  • twitter
  • facebook
  • share this

share this

:: By Kim Ann King, SiteSpect ::


If you are looking for a new Web optimization vendor, or checking out testing technology for the first time, make sure you are asking all the right questions. Optimizing the entire user experience can be tricky, and you’ll want to be sure the technology you are selecting can do everything you need. You’ll want to make sure the technology you select can test every part of your site (front-end and back-end) and enable you to run more tests in less time. 

If you are in charge of selecting optimization technology, you are also likely responsible for advancing your company’s optimization culture and creating measurable wins for your marketers, product managers, and developers. Here are some of the questions our customers have asked us over the years; it’s a helpful list to get you started.

1) Can it test all types of content? It’s simply not enough to be able to test copy, fonts, layout, and images these days. Not only do you want to be able to test static HTML content, but also dynamic content and mobile content, among other things. 

2) Can it test mobile apps? App stores have a defined approval process, and you have little control over the approval timeline once your app is submitted. Companies that test their mobile apps know they are right from the start -- saving a great deal of time and resources.

3) Can it test every part of the conversion funnel on your Web or mobile site – from home page and landing pages to product pages and checkout pages?

4) Can it optimize navigation? For example, how the user experience is sequenced, what each item is labeled (e.g., "My profile" vs. "Profile" vs. "Settings"), how inline text links look, and link style, among other things.

5) Can it test site functionality? For example:

a. On-site search: On-site search is the typically the second-most visited area on a website and our clients have found the following test ideas extremely useful. So you’ll want to know if  the optimization solution help you optimize on-site search functionality, such as:

i. search results display

ii. search metadata (such as ratings) display

iii. search result filtering options

iv. number of search results displayed

v. search results layout (grid versus list, for example)

vi. how product type impacts search display preference

vii. underlying search algorithms

viii. ancillary search features (such as spelling correction)

b. Checkout process: Can it test how many form fields (and which ones) and number of pages in the checkout process are ideal to maximize conversions on your site? Can it experiment with the checkout process for logged-in versus non-logged in visitors? Can it test plug-ins used during checkout, such as chat?

c. Pricing/Shipping Offers: Can it test optimal free shipping thresholds, for example?

d. New features: There are ways to both mitigate risk and learn valuable insights without negatively impacting your conversion when launching new features. The best practice is to test these new features to a subset of your audience and measure behavior upfront, before a launch to all site visitors/customers. This also offers the ability to quickly measure KPIs while also checking for misplaced code or a poor experience. The size of your test audience for the new feature depends on traffic volume. If your site receives a significant amount of traffic, we recommend rolling out the feature to 1-3 percent of your traffic. This can always be adjusted if you aren’t getting the necessary traffic to reach statistical significance. Can the optimization solutions you are considering do all that?

e. Whole site releases: In other words, can it test a new site or new site sections versus the previous site as a control?

6) Can you use it to quantify the impact of your Web development efforts on your KPIs?

- See more at: http://www.websitemagazine.com/content/blogs/posts/archive/2014/05/23/op...

Website Magazine recently published an article written by SiteSpect and we thought you'd be interested in this information. Happy reading!

If you are looking for a new web optimization vendor, or checking out testing technology for the first time, make sure you are asking all the right questions. Optimizing the entire user experience can be tricky, and you’ll want to be sure the technology you are selecting can do everything you need. You’ll want to make sure the technology you select can test every part of your site (front-end and back-end) and enable you to run more tests in less time.

If you are in charge of selecting optimization technology, you are also likely responsible for advancing your company’s optimization culture and creating measurable wins for your marketers, product managers, and developers. Here are some of the questions our customers have asked us over the years; it’s a helpful list to get you started.

  1. Can it test all types of content? It’s simply not enough to be able to test copy, fonts, layout, and images these days. Not only do you want to be able to test static HTML content, but also dynamic content and mobile content, among other things.
  2. Can it test mobile apps? App stores have a defined approval process, and you have little control over the approval timeline once your app is submitted. Companies that test their mobile apps know they are right from the start -- saving a great deal of time and resources.
  3. Can it test every part of the conversion funnel on your web or mobile site – from home page and landing pages to product pages and checkout pages?
  4. Can it optimize navigation? For example, how the user experience is sequenced, what each item is labeled (e.g., "My profile" vs. "Profile" vs. "Settings"), how inline text links look, and link style, among other things.
  5. Can it test site functionality? For example:
    1. On-site search: On-site search is the typically the second-most visited area on a website and our clients have found the following test ideas extremely useful. So you’ll want to know if  the optimization solution help you optimize on-site search functionality, such as:
      • search results display
      • search metadata (such as ratings) display
      • search result filtering options
      • number of search results displayed
      • search results layout (grid versus list, for example)
      • how product type impacts search display preference
      • underlying search algorithms
      • ancillary search features (such as spelling correction)
    2. Checkout process: Can it test how many form fields (and which ones) and number of pages in the checkout process are ideal to maximize conversions on your site? Can it experiment with the checkout process for logged-in versus non-logged in visitors? Can it test plug-ins used during checkout, such as chat?
    3.  Pricing/Shipping Offers: Can it test optimal free shipping thresholds, for example?
    4. New features: There are ways to both mitigate risk and learn valuable insights without negatively impacting your conversion when launching new features. The best practice is to test these new features to a subset of your audience and measure behavior upfront, before a launch to all site visitors/customers. This also offers the ability to quickly measure KPIs while also checking for misplaced code or a poor experience. The size of your test audience for the new feature depends on traffic volume. If your site receives a significant amount of traffic, we recommend rolling out the feature to 1-3% of your traffic. This can always be adjusted if you aren’t getting the necessary traffic to reach statistical significance. Can the optimization solutions you are considering do all that?
    5. Whole site releases: In other words, can it test a new site or new site sections versus the previous site as a control?
  6. Can you use it to quantify the impact of your web development efforts on your KPIs?

Read the rest of the article on Website Magazine's website.

Tags: A/B & Multivariate Testing Product Features