A/B Testing as a Mindset

By SiteSpect Marketing

April 17, 2015


There’s something really liberating about being able to A/B test all of your beliefs. If you want to build a culture of optimization through A/B testing, that’s what you have to do. Adopting experimentation as a mindset will enable you to A/B test more ideas, brainstorm more alternatives, and maximize what’s possible.

So, how might companies actually put this new mindset into action? First, they can apply experimentation and optimization to their digital initiatives. By carefully examining what they believe to be true and then A/B testing alternatives, they can discover where the real truth lies.

Website Optimization Ideas

Consider the content, features, and functionality of your site to discover what works and what doesn’t. For example, you could experiment with the following:


  • A lot versus a little
  • Style and tone, such as chatty versus formal
  • Positioning: in other words, which value proposition, features, and benefits work best?
  • Call-to-action text

Font, color, and size

  • Think about your target audience (e.g., 10% of men are colorblind).
  • If your audience is older, A/B test larger fonts.
  • Consider readability; for example, serif fonts are often easier to read than sans serif fonts.


  • Color
  • Size
  • Design
  • Location
  • Copy


  • Sequence of items
  • Labels (e.g., “My profile” vs. “Profile” vs. “Settings”)
  • Inline text links
  • Link style: underline, bold, color, etc.


  • Content of image (e.g., man vs. woman vs. group)
  • Size
  • Location on page
  • Design treatment (e.g., shadow background, outlined, etc.)


  • Optimal number of required fields versus optional fields
  • Design
  • Location
  • Size


  • Location and size of areas/boxes on the page
  • Attention focus: what layouts help focus vs. create distraction?
  • Where should you place buttons and links?

Features and Functionality

  • New feature releases. Soft-launch new features on a subset of users and learn how they respond, in real time. The additional customer insight can help you iterate on features more quickly and deliver enhancements that actually move the needle.
  • Feature segmentation. Present different features to different audiences—based on customer value, device, geography, new vs. existing customers, etc.—to optimize KPIs.
  • Search algorithm optimization. Make head-to-head comparisons of the effectiveness of different versions of your site-search algorithms or even multiple search vendor software solutions.
  • Pricing experiments. A/B test your “free shipping” threshold or offer different price promotions to distinct sets of targeted users; for example, A/B test the impact of changing the threshold to obtain free shipping from $50 to $35.
  • Complete site redesigns. A/B test radical changes to your site experience targeted to specific users.
  • Checkout flow and functionality. Optimize your checkout flow through experimentation even on secure https pages. For example, you could A/B test making your five-step checkout process only three steps for logged in users and see what a difference that makes.

Mobile Optimization

Customers are no longer tethered to their desktops, yet they expect an optimized browsing experience on their mobile device; so, you’ll want to A/B test all aspects of your mobile site, including load times, search, and checkout, to deliver an optimal experience.

Experiment with different mobile experiences such as…

  • Traditional websites
  • Responsive design sites
  • M.dot, t.dot, and other dedicated mobile sites
  • Angular, single-page applications, and hybrid apps
  • Native applications

A/B test how different types of mobile content influence user behavior, and ultimately, site profitability. A/B test, measure, and deliver the content, layout, and promotional offers that are most effective for each mobile device category.

Also, consider targeting your tested content using a combination of static and dynamic criteria, including the following, for example:

  • Device type, including tablets and smartphones
  • Mobile operating system
  • Screen dimensions and rotation support
  • Browser capabilities
  • Cellular network data speeds
  • Preferred markup language
  • Keyboard type

Experimentation does not end with A/B testing; it also extends to segmentation and targeting strategies to enable you to target your A/B tests. Target visitors based on criteria including the following:

  • User behavior, such as adding an item to a cart, bookmarking a favorite, or viewing a product category page
  • Geo-targeting
  • History, such as recency, frequency, and monetary value of previous purchases and transactions
  • Mobile device and platform
  • Browser
  • Referral source of visit, such as Google or PPC
  • Day-parting and time-parting
  • Language
  • Externally defined criteria from third-party CRM or SFA databases

Those are just a few ideas to create experimentation as a mindset to build a culture of optimization. What other things can you A/B test? Let us know in the comments below!

To learn more about SiteSpect, visit our website.



SiteSpect Marketing

SiteSpect Marketing

SiteSpect Marketing focuses on authoring content for the web and social media.

Suggested Posts

Subscribe to our blog: