5 Reasons You Should Be Testing Feature Releases

Feature releases can run the gamut from introducing a new payment option to replatforming an entire website. No matter the scale of your new release it can be a challenge to quantify success across organizational metrics. How does a new website platform affect website visitor metrics? Does an updated payment option increase purchases? Does improved site speed correlate with increased revenue? Without the ability to connect these dots, your teams will be stuck in silos and you’ll inevitably reach a plateau in all of your key metrics. So, why should you A/B test your feature releases and how do you do it successfully? In this blog we’ll walk through five reasons to A/B test feature releases.

1. Discover how well your release works, beyond standard QA.

Before you release a new feature it’s gone through your rigorous QA process. You’ve caught any bugs and made sure your release functions properly, but how do your live users actually respond to it? By A/B testing your new feature on a controlled subset of traffic, you get additional data that not only eliminates the risk of an unsuccessful release or unexpected consequence, but also helps your team evangelize your work across the organization. Plus, you may discover some unexpected benefits to your new feature. For example, does a new platform chosen primarily for its development capabilities also increase site speed and correlate with increased conversions?

2. Maintain a cleaner code base.

Releasing a new feature only to find out that you need to make significant changes to it takes a major toll on your team’s resources. But, by A/B testing through a release rollouts tool first you can make adjustments and perfect your release — while getting data from real traffic — without interfering with your code base. Once you’re ready, you can roll out your completed feature without the mess.

3. Control who sees new features, when, and how often.

If you have no A/B testing tool or process for release rollouts, it can be a challenge to control which users see new features and when. But, not every release is right for every user at every time. With controlled release testing, you designate audiences based on any number of factors, including geolocation, whether they are a new or returning visitor, browser or device type, previous behaviors, and more. This control lets you fine tune features for each audience, as well as ensure that features work as they should across the board.

4. Get more creative with less risk.

When every release impacts your code base, your site visitors, and your bottom line, you end up taking a risk every time you release something new. This means that to some extent, you have to play it safe. This saves you from unexpected negative results, but it also limits your successes. By A/B testing your feature releases at every stage you have the freedom to get a lot more creative. If you take a big risk and it fails, it only fails on a small subset of traffic and you can pause any A/B test in a matter of seconds. This gives you the room to pause a campaign and work on any issues without causing any fire drills, and you never have to roll back a code release. This also gives you the potential to invent creative solutions and innovative features for your customer base.

5. Get full analytics and reporting for every new feature.

With a good rollouts testing tool, you should also be able to get full analytics for hundreds of metrics. This is important not only because the data itself is important, but also because it allows you to get, in one bird’s eye view, a report on feature performance that integrates all stakeholders and interests. By A/B testing your new releases, you have the time and ability to gather a wider range of metrics and data before you roll out your feature to all traffic.

To learn more about SiteSpect Rollouts, visit our website.