A/B Testing Mistakes and How to Avoid Them Part 5 – Not Segmenting Data

Data analysis is the crux of A/B testing and optimization. But many of us who do A/B testing and optimization aren’t data analysts by trade; we’re learning as we go.

One of the most common mistakes we see in A/B testing is not segmenting data. To understand how your A/B tests are performing and how they impact shopper behavior, you need to segment your data effectively.

Have you missed my other posts in this series?

• Part 1
• Part 2
• Part 3
• Part 4

What is Data Segmentation?

“Segments identify a subset of visitors who exhibit similar behavior or share similar attributes while visiting your website.”

Data segmentation is the process of taking your data and breaking it into smaller groups based on set parameters. For example, let’s say you’re tracking the following metrics:

1. Clicks on a featured item image.
2. Adds to cart of a featured item.
3. The total adds to cart.
4. Purchases.
5. Revenue per shopper.

Segmenting this data would mean looking at specific groups of shoppers, for example, only shoppers who came from a marketing email, or only shoppers on mobile devices, and looking at how your metrics perform just among those shoppers.

You can segment according to any parameter. You can look just at shoppers who completed a certain action, such as completing a purchase or clicking on a specific link. Or you can look at shoppers from a specific region, or on a certain type of device. You can even segment based on combinations of factors, like shoppers in a particular state who completed a purchase and clicked on the featured item image.

Why Data Segmentation is Important

Segmentation is critical because it gives you a much fuller picture of the factors that influence shopper behavior. Let’s say that in your variation with a new featured item image, you see purchases are higher than in the control group. You could easily call this a win and end it at that, but there are still unanswered questions. Did the shoppers who saw the featured image click it? And vice versa, did the shoppers who clicked the featured image purchase an item? Is revenue per visit higher among shoppers who clicked the featured image?

If you’re new to data analysis, break your goals into smaller pieces and tackle them one at a time. For example, if you want to know what the purchase rate looked like for shoppers who clicked the featured image, create a segment based on that one behavior. If you want to know how many shoppers who completed a purchase saw the featured image, create a segment just for shoppers who completed a purchase. This way, you’ll start to gain some clarity not just on which variation performed better, but also on why it did.

Summary

To gain a comprehensive understanding of the performance of your A/B tests and their influence on shopper behavior, it is crucial to appropriately segment your data. This practice of segmentation will help you make important decisions to help improve your conversions, therefore, increasing revenue.

Speak with an experimentation expert to see a personalized product demo. It will be time well spent!